Commits · f7512877e465e3c1fefd02927adc3e4d2fad0c56 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "5b833be49e02fec2542396a01c9575dfe6e1def2"

22 Oct, 2024 1 commit
- remove notes · f7512877
  zhuwenwen authored Oct 22, 2024
  
  f7512877
21 Oct, 2024 1 commit
- Merge tag 'v0.6.3.post1' into v0.6.3.post1-dev · 6d2051cc
  zhuwenwen authored Oct 21, 2024
  
  6d2051cc
19 Oct, 2024 2 commits
- update _custom_ops.py · 2c7f740a
  zhuwenwen authored Oct 19, 2024
  
  2c7f740a
- update _custom_ops.py · 5289f7c4
  zhuwenwen authored Oct 19, 2024
  
  5289f7c4
18 Oct, 2024 2 commits
- 删除v2 8192限制 · a7e3c9d6
  zhangshao authored Oct 18, 2024
  
  a7e3c9d6
- 解决pa v2 bug · e2bd7e16
  zhangshao authored Oct 18, 2024
  
  e2bd7e16
17 Oct, 2024 14 commits
- [CI/Build] remove .github from .dockerignore, add dirty repo check (#9375) · a2c71c54
  Daniele authored Oct 17, 2024
  
  a2c71c54
- [Core] Deprecating block manager v1 and make block manager v2 default (#8704) · 81ede99c
  Kuntai Du authored Oct 17, 2024
```
Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).
```
  81ede99c
- [Hardware][CPU] compressed-tensor INT8 W8A8 AZP support (#9344) · 5eda21e7
  Li, Jiang authored Oct 18, 2024
  
  5eda21e7
- [TPU] Call torch._sync(param) during weight loading (#9437) · 8e1cddcd
  Woosuk Kwon authored Oct 17, 2024
  
  8e1cddcd
- [Bugfix] Allow prefill of assistant response when using `mistral_common` (#9446) · 5e443b59
  sasha0552 authored Oct 17, 2024
  
  5e443b59
- [misc] CUDA Time Layerwise Profiler (#8337) · 9d30a056
  Lucas Wilkinson authored Oct 17, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  9d30a056
- [Misc] Print stack trace using `logger.exception` (#9461) · 390be746
  Cyrus Leung authored Oct 17, 2024
  
  390be746
- [Kernel] Add Exllama as a backend for compressed-tensors (#9395) · e312e52b
  Lucas Wilkinson authored Oct 17, 2024
  
  e312e52b
- update pa tc interfaces · 304e2bab
  zhuwenwen authored Oct 17, 2024
  
  304e2bab
- update attention_kernels_opt_tc.cu · 1c2aa04c
  zhuwenwen authored Oct 17, 2024
  
  1c2aa04c
- Add notes on the use of Slack (#9442) · dbfa8d31
  Yuan Tang authored Oct 17, 2024
  
  dbfa8d31
- update attention_kernels_opt_tc.cu · 83bb8f5a
  zhuwenwen authored Oct 17, 2024
  
  83bb8f5a
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
- [BugFix] [Kernel] Fix GPU SEGV occurring in int8 kernels (#9391) · 92d86da2
  rasmith authored Oct 16, 2024
  
  92d86da2
16 Oct, 2024 16 commits
- [Bugfix][Kernel] Prevent integer overflow in fp8 dynamic per-token quantize kernel (#9425) · c3fab5f7
  Tyler Michael Smith authored Oct 16, 2024
  
  c3fab5f7
- [CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) · 776dbd74
  Russell Bryant authored Oct 16, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  776dbd74
- [Performance][Spec Decode] Optimize ngram lookup performance (#9333) · 83450458
  Lily Liu authored Oct 16, 2024
  
  83450458
- [Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCPM-S-1B-sft (#9396) · 5b8a1fde
  Junhao Li authored Oct 16, 2024
  
  5b8a1fde
- [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) · fb60ae9b
  Mor Zusman authored Oct 17, 2024
  
  fb60ae9b
- Support mistral interleaved attn (#9414) · 415f76a9
  Patrick von Platen authored Oct 16, 2024
  
  415f76a9
- [Model] Support SDPA attention for Molmo vision backbone (#9410) · cf1d62a6
  Isotr0py authored Oct 16, 2024
  
  cf1d62a6
- [Misc] Consolidate example usage of OpenAI client for multimodal models (#9412) · 59230ef3
  Roger Wang authored Oct 16, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59230ef3
- [Core] Rename input data types (#8688) · cee711fd
  Cyrus Leung authored Oct 16, 2024
  
  cee711fd
- [CI/Build] Test VLM embeddings (#9406) · 1de76a0e
  Cyrus Leung authored Oct 16, 2024
  
  1de76a0e
- [Model] VLM2Vec, the first multimodal embedding model in vLLM (#9303) · 7abba39e
  Cyrus Leung authored Oct 16, 2024
  
  7abba39e
- [Misc] Standardize RoPE handling for Qwen2-VL (#9250) · 7e7eae33
  Cyrus Leung authored Oct 16, 2024
  
  7e7eae33
- [Bugfix] Molmo text-only input bug fix (#9397) · ed920135
  Reza Salehi authored Oct 15, 2024
```
Co-authored-by: sanghol <sanghol@allenai.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  ed920135
- add LM_TN for bloom lm_head weight · 2a9c497e
  zhuwenwen authored Oct 16, 2024
  
  2a9c497e
- update paged_attn.py · 85e8224c
  zhuwenwen authored Oct 16, 2024
  
  85e8224c
- [Bugfix][CI/Build] Fix CUDA 11.8 Build (#9386) · 717a5f82
  Lucas Wilkinson authored Oct 15, 2024
  
  717a5f82
15 Oct, 2024 4 commits
- [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) · ba309422
  Chang Su authored Oct 15, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ba309422
- [Misc] Directly use compressed-tensors for checkpoint definitions (#8909) · 22f8a695
  Michael Goin authored Oct 15, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  22f8a695
- pass ignore_eos parameter to all benchmark_serving calls (#9349) · 5d264f4a
  Grace Ho authored Oct 15, 2024
  
  5d264f4a
- support baichuan awq and skip _rocm_C · b3ab1cdc
  zhuwenwen authored Oct 15, 2024
  
  b3ab1cdc