Commits · 47c7126213163e454019efc7d913125c22df9d6e · OpenDAS / vllm_cscc

"vllm/entrypoints/openai/engine/protocol.py" did not exist on "6a9c583e73c75c8eab10a9c607cb096750b751a0"

21 Mar, 2025 21 commits
- [Misc] Add attention mask pre-computation optimization back to Qwen2.5-VL (#15273) · 47c71262
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  47c71262
- [Bugfix] Add int8 torch dtype for KVCache (#15260) · a989ca2b
  Shanshan Shen authored Mar 21, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  a989ca2b
- [Feature] specify model in config.yaml (#14855) · 0fa3970d
  Wei Zeng authored Mar 21, 2025
```
Signed-off-by: weizeng <weizeng@roblox.com>
```
  0fa3970d
- [V1] Avoid redundant input processing in n>1 case (#14985) · da6ea29f
  Nick Hill authored Mar 20, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  da6ea29f
- [Doc] Update LWS docs (#15163) · 7297941b
  Edwin Hernandez authored Mar 20, 2025
```
Signed-off-by: Edwinhr716 <Edandres249@gmail.com>
```
  7297941b
- [V1] Enable Triton(ROCm) Attention backend for Nvidia GPUs (#14071) · f8a08cb9
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f8a08cb9
- [Hardware][TPU] Add check for no additional graph compilation during runtime (#14710) · b15fd2be
  Siyuan Liu authored Mar 20, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  b15fd2be
- Add an example for reproducibility (#15262) · e588ac23
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e588ac23
- [Misc] Better RayExecutor and multiprocessing compatibility (#14705) · 5df2da5b
  Cody Yu authored Mar 20, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  5df2da5b
- [Docs] Trim the latest news in README (#15261) · 11b986b3
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  11b986b3
- [Model] RE: Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14857) · 296f927f
  Chih-Chieh Yang authored Mar 20, 2025
```
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
```
  296f927f
- [Bugfix] detect alibi and revert to FA2 (#15231) · 0032903a
  Travis Johnson authored Mar 20, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0032903a
- [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) · 47195057
  Hyesoo Yang authored Mar 20, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  47195057
- Mention `extra_body` as a way top pass vLLM only parameters using the OpenAI client (#15240) · 6edbfa92
  Harry Mellor authored Mar 21, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6edbfa92
- [Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation (#15200) · 1e508343
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1e508343
- [ROCM] Upgrade torch to 2.6 (#15244) · 2e0b4cfd
  Sage Moore authored Mar 20, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  2e0b4cfd
- [Misc] Clean up the BitsAndBytes arguments (#15140) · 10f55fe6
  Jee Jee Li authored Mar 21, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  10f55fe6
- Fix CUDA kernel index data type in... · d3ccbd63
  Lu Fang authored Mar 20, 2025
```
Fix CUDA kernel index data type in vllm/csrc/quantization/fused_kernels/layernorm_utils.cuh +10 (#15159)
Signed-off-by: Lu Fang <lufang@fb.com>
Co-authored-by: Richard Barnes <rbarnes@meta.com>
```
  d3ccbd63
- [CI/Build] LoRA : make add_lora_test safer (#15181) · 0cfe7d38
  Varun Sundar Rabindranath authored Mar 20, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  0cfe7d38
- [V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) · 0c6f5023
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  0c6f5023
- Enforce that TP > 1 is not supported for Mamba2 if Quantization is Enabled. (#14617) · 06dd0825
  Yu Chin Fabian Lim authored Mar 21, 2025
```
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
```
  06dd0825
20 Mar, 2025 19 commits
- [V1] Add flag to disable cascade attention (#15243) · 2b22290c
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2b22290c
- [Bugfix] fix V1 Engine crash while handling requests with duplicate request id (#15043) · d8e82bc0
  Jason authored Mar 21, 2025
```
Signed-off-by: Jiahui Sun <jhsun2020@gmail.com>
```
  d8e82bc0
- [ci] feat: make the test_torchrun_example run with tp=2, external_dp=2 (#15172) · 086b5682
  Chi Zhang authored Mar 21, 2025
```
Signed-off-by: Chi Zhang <zhangchi.usc1992@bytedance.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  086b5682
- Replace `misc` issues with link to forum (#15226) · 5a0905ba
  Harry Mellor authored Mar 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  5a0905ba
- Fix env vars for running Ray distributed backend on GKE (#15166) · a8f12a63
  Richard Liu authored Mar 20, 2025
```
Signed-off-by: Richard Liu <ricliu@google.com>
```
  a8f12a63
- Add user forum to README (#15220) · 69ae2380
  Harry Mellor authored Mar 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  69ae2380
- [Bugfix] Multi-video inference on LLaVA-Onevision (#15082) · 27261e40
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  27261e40
- [macOS] Ugrade pytorch to 2.6.0 (#15129) · e3f813c3
  Quang-Linh LE authored Mar 20, 2025
  
  e3f813c3
- Fixing Imprecise Type Annotations (#15192) · c607a265
  Wang Ran (汪然) authored Mar 20, 2025
  
  c607a265
- [release] Tag vllm-cpu with latest upon new version released (#15193) · 3d45e3d7
  Kevin H. Luu authored Mar 20, 2025
  
  3d45e3d7
- [Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824) · 742369d3
  billishyahao authored Mar 20, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
```
  742369d3
- typo: Update config.py (#15189) · bfe2fe0a
  Wang Ran (汪然) authored Mar 20, 2025
  
  bfe2fe0a
- Enable CUDA graph support for llama 3.2 vision (#14917) · a8652f4f
  Matt Ritter authored Mar 19, 2025
```
Signed-off-by: Matt Ritter <100659061+mritterfigma@users.noreply.github.com>
```
  a8652f4f
- [Doc] Update README.md (#15187) · 2f726b24
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2f726b24
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
- [Misc]fixed disable these http request logs (#14754) · ae65f3e2
  Chauncey authored Mar 20, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ae65f3e2
- [Doc] Update Mistral Small 3.1/Pixtral example (#15184) · 34868b10
  Roger Wang authored Mar 19, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  34868b10
- [Core][V0] Add guidance backend for structured output (#14589) · 1f16b7fe
  Russell Bryant authored Mar 20, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <lohuynh@microsoft.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
```
  1f16b7fe
- [Benchmark] Allow oversample request in benchmark dataset (#15170) · b88be221
  Jennifer Zhao authored Mar 19, 2025
```
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  b88be221