Commits · 6d2051cc498f78cd60b4c1da054a499d6a7594ff · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "e2db2b42347ae27da3083c530c71b251861ed220"

19 Oct, 2024 2 commits
- update _custom_ops.py · 2c7f740a
  zhuwenwen authored Oct 19, 2024
  
  2c7f740a
- update _custom_ops.py · 5289f7c4
  zhuwenwen authored Oct 19, 2024
  
  5289f7c4
18 Oct, 2024 1 commit
- 删除v2 8192限制 · a7e3c9d6
  zhangshao authored Oct 18, 2024
  
  a7e3c9d6
17 Oct, 2024 7 commits
- [Core] Deprecating block manager v1 and make block manager v2 default (#8704) · 81ede99c
  Kuntai Du authored Oct 17, 2024
```
Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).
```
  81ede99c
- [TPU] Call torch._sync(param) during weight loading (#9437) · 8e1cddcd
  Woosuk Kwon authored Oct 17, 2024
  
  8e1cddcd
- [Bugfix] Allow prefill of assistant response when using `mistral_common` (#9446) · 5e443b59
  sasha0552 authored Oct 17, 2024
  
  5e443b59
- [misc] CUDA Time Layerwise Profiler (#8337) · 9d30a056
  Lucas Wilkinson authored Oct 17, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  9d30a056
- [Misc] Print stack trace using `logger.exception` (#9461) · 390be746
  Cyrus Leung authored Oct 17, 2024
  
  390be746
- [Kernel] Add Exllama as a backend for compressed-tensors (#9395) · e312e52b
  Lucas Wilkinson authored Oct 17, 2024
  
  e312e52b
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
16 Oct, 2024 12 commits
- [CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) · 776dbd74
  Russell Bryant authored Oct 16, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  776dbd74
- [Performance][Spec Decode] Optimize ngram lookup performance (#9333) · 83450458
  Lily Liu authored Oct 16, 2024
  
  83450458
- [Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCPM-S-1B-sft (#9396) · 5b8a1fde
  Junhao Li authored Oct 16, 2024
  
  5b8a1fde
- [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) · fb60ae9b
  Mor Zusman authored Oct 17, 2024
  
  fb60ae9b
- Support mistral interleaved attn (#9414) · 415f76a9
  Patrick von Platen authored Oct 16, 2024
  
  415f76a9
- [Model] Support SDPA attention for Molmo vision backbone (#9410) · cf1d62a6
  Isotr0py authored Oct 16, 2024
  
  cf1d62a6
- [Core] Rename input data types (#8688) · cee711fd
  Cyrus Leung authored Oct 16, 2024
  
  cee711fd
- [Model] VLM2Vec, the first multimodal embedding model in vLLM (#9303) · 7abba39e
  Cyrus Leung authored Oct 16, 2024
  
  7abba39e
- [Misc] Standardize RoPE handling for Qwen2-VL (#9250) · 7e7eae33
  Cyrus Leung authored Oct 16, 2024
  
  7e7eae33
- [Bugfix] Molmo text-only input bug fix (#9397) · ed920135
  Reza Salehi authored Oct 15, 2024
```
Co-authored-by: sanghol <sanghol@allenai.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  ed920135
- add LM_TN for bloom lm_head weight · 2a9c497e
  zhuwenwen authored Oct 16, 2024
  
  2a9c497e
- update paged_attn.py · 85e8224c
  zhuwenwen authored Oct 16, 2024
  
  85e8224c
15 Oct, 2024 7 commits
- [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) · ba309422
  Chang Su authored Oct 15, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  ba309422
- [Misc] Directly use compressed-tensors for checkpoint definitions (#8909) · 22f8a695
  Michael Goin authored Oct 15, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  22f8a695
- support baichuan awq and skip _rocm_C · b3ab1cdc
  zhuwenwen authored Oct 15, 2024
  
  b3ab1cdc
- [BugFix] Fix chat API continuous usage stats (#9357) · e9d517f2
  Nick Hill authored Oct 15, 2024
  
  e9d517f2
- [Bugfix] Update InternVL input mapper to support image embeds (#9351) · 55e081fb
  hhzhang16 authored Oct 14, 2024
  
  55e081fb
- [Frontend] Clarify model_type error messages (#9345) · 44eaa5a5
  Steve Grubb authored Oct 15, 2024
  
  44eaa5a5
- [Bugfix] Clean up some cruft in mamba.py (#9343) · 169b5306
  Tyler Michael Smith authored Oct 14, 2024
  
  169b5306
14 Oct, 2024 5 commits
- [Model] Make llama3.2 support multiple and interleaved images (#9095) · f0fe4fe8
  Xiang Xu authored Oct 14, 2024
  
  f0fe4fe8
- [Frontend] merge beam search implementations (#9296) · 4d31cd42
  Brendan Wong authored Oct 14, 2024
  
  4d31cd42
- [TPU] Fix TPU SMEM OOM by Pallas paged attention kernel (#9350) · 473e7b36
  Woosuk Kwon authored Oct 14, 2024
  
  473e7b36
- [Hardware][intel GPU] add async output process for xpu (#8897) · 4141608c
  Kunshang Ji authored Oct 15, 2024
  
  4141608c
- [Model] Molmo vLLM Integration (#9016) · dfe43a20
  Reza Salehi authored Oct 14, 2024
```
Co-authored-by: sanghol <sanghol@allenai.org>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  dfe43a20
13 Oct, 2024 2 commits
- [Bugfix] Bandaid fix for speculative decoding tests (#9327) · 16b24e7d
  Tyler Michael Smith authored Oct 13, 2024
  
  16b24e7d
- [CI] Fix merge conflict (#9317) · f519902c
  Lily Liu authored Oct 12, 2024
  
  f519902c
12 Oct, 2024 4 commits
- [Bugfix]Fix MiniCPM's LoRA bug (#9286) · 250e26a6
  Jee Jee Li authored Oct 13, 2024
  
  250e26a6
- restore qwen-vl to default layout · 422af727
  zhuwenwen authored Oct 12, 2024
  
  422af727
- [Bugfix] Fix bug of xformer prefill for encoder-decoder (#9026) · 00298e09
  Xiang Xu authored Oct 12, 2024
  
  00298e09
- [SpecDec] Remove Batch Expansion (2/3) (#9298) · 89feb4c8
  Lily Liu authored Oct 11, 2024
  
  89feb4c8