Commits · b877031d806e3d9ebc834e0191ae64de40c4ddc2 · OpenDAS / vllm_cscc · GitLab

22 Mar, 2025 10 commits
- Remove openvino support in favor of external plugin (#15339) · b877031d
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b877031d
- [BugFix][Typing] Fix Imprecise Type Annotations (#15208) · dd861b99
  Wang Ran (汪然) authored Mar 23, 2025
```
Signed-off-by: Wang Ran (汪然) <wrran@outlook.com>
```
  dd861b99
- [V1] Add `disable-any-whitespace` option support for xgrammar (#15316) · eb63ea1e
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  eb63ea1e
- [Model] Support Tele-FLM Model (#15023) · 2f4bd358
  Naitong Yu authored Mar 22, 2025
```
Signed-off-by: Naitong Yu <ntyu@baai.ac.cn>
Signed-off-by: jiangxin <horizon94@outlook.com>
Co-authored-by: Jason Fang <jasonfang3900@gmail.com>
Co-authored-by: jiangxin <horizon94@outlook.com>
```
  2f4bd358
- [Bugfix] LoRA V0 - Fix case where `max_num_seqs` is between cudagraph capture sizes (#15308) · 8a8b30ea
  Varun Sundar Rabindranath authored Mar 22, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  8a8b30ea
- [Bugfix] Fix torch.compile raise FileNotFoundError (#15278) · 2fa0e139
  Jee Jee Li authored Mar 22, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  2fa0e139
- [Doc] add load_format items in docs (#14804) · 1c2bec0f
  wwl2755 authored Mar 22, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  1c2bec0f
- [FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) · ec870fba
  TJian authored Mar 22, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  ec870fba
- [Bugfix][V0] Multi-sequence logprobs streaming edge case (#15259) · df143026
  Andy Lo authored Mar 22, 2025
```
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  df143026
- [Misc] Increase RayDistributedExecutor RAY_CGRAPH_get_timeout (#15301) · 4c69e228
  Rui Qiao authored Mar 21, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  4c69e228
21 Mar, 2025 30 commits
- [Build/CI] Fix env var typo (#15305) · 790b7975
  Russell Bryant authored Mar 21, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  790b7975
- [TPU][V1] MHA Pallas backend (#15288) · cfbb8c93
  Nicolò Lucchesi authored Mar 21, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  cfbb8c93
- Revert "[Feature] specify model in config.yaml (#14855)" (#15293) · baec0d4d
  Cyrus Leung authored Mar 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  baec0d4d
- [Bugfix][VLM] fix llava processor (#15285) · c21b99b9
  Mengqing Cao authored Mar 21, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  c21b99b9
- [v1] Refactor KVCacheConfig (#14079) · 93a00d7d
  Chen Zhang authored Mar 21, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  93a00d7d
- [Misc] Add cProfile helpers (#15074) · 61e8c183
  Russell Bryant authored Mar 21, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  61e8c183
- [Bugfix] Fix broken kernel test due to missing rename for v1 Triton backend (#15282) · 8afcd0f6
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  8afcd0f6
- [V1] Fix wrong import path of get_flash_attn_version (#15280) · 91ca929d
  Lehua Ding authored Mar 21, 2025
```
Signed-off-by: Lehua Ding <lehuading@tencent.com>
```
  91ca929d
- [Bugfix] Fix incorrect resolving order for transformers fallback (#15279) · 84e00adc
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  84e00adc
- [Misc] Add attention mask pre-computation optimization back to Qwen2.5-VL (#15273) · 47c71262
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  47c71262
- [Bugfix] Add int8 torch dtype for KVCache (#15260) · a989ca2b
  Shanshan Shen authored Mar 21, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  a989ca2b
- [Feature] specify model in config.yaml (#14855) · 0fa3970d
  Wei Zeng authored Mar 21, 2025
```
Signed-off-by: weizeng <weizeng@roblox.com>
```
  0fa3970d
- [V1] Avoid redundant input processing in n>1 case (#14985) · da6ea29f
  Nick Hill authored Mar 20, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  da6ea29f
- [Doc] Update LWS docs (#15163) · 7297941b
  Edwin Hernandez authored Mar 20, 2025
```
Signed-off-by: Edwinhr716 <Edandres249@gmail.com>
```
  7297941b
- [V1] Enable Triton(ROCm) Attention backend for Nvidia GPUs (#14071) · f8a08cb9
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f8a08cb9
- [Hardware][TPU] Add check for no additional graph compilation during runtime (#14710) · b15fd2be
  Siyuan Liu authored Mar 20, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  b15fd2be
- Add an example for reproducibility (#15262) · e588ac23
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e588ac23
- [Misc] Better RayExecutor and multiprocessing compatibility (#14705) · 5df2da5b
  Cody Yu authored Mar 20, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  5df2da5b
- [Docs] Trim the latest news in README (#15261) · 11b986b3
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  11b986b3
- [Model] RE: Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14857) · 296f927f
  Chih-Chieh Yang authored Mar 20, 2025
```
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
```
  296f927f
- [Bugfix] detect alibi and revert to FA2 (#15231) · 0032903a
  Travis Johnson authored Mar 20, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  0032903a
- [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) · 47195057
  Hyesoo Yang authored Mar 20, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  47195057
- Mention `extra_body` as a way top pass vLLM only parameters using the OpenAI client (#15240) · 6edbfa92
  Harry Mellor authored Mar 21, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6edbfa92
- [Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation (#15200) · 1e508343
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1e508343
- [ROCM] Upgrade torch to 2.6 (#15244) · 2e0b4cfd
  Sage Moore authored Mar 20, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  2e0b4cfd
- [Misc] Clean up the BitsAndBytes arguments (#15140) · 10f55fe6
  Jee Jee Li authored Mar 21, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  10f55fe6
- Fix CUDA kernel index data type in... · d3ccbd63
  Lu Fang authored Mar 20, 2025
```
Fix CUDA kernel index data type in vllm/csrc/quantization/fused_kernels/layernorm_utils.cuh +10 (#15159)
Signed-off-by: Lu Fang <lufang@fb.com>
Co-authored-by: Richard Barnes <rbarnes@meta.com>
```
  d3ccbd63
- [CI/Build] LoRA : make add_lora_test safer (#15181) · 0cfe7d38
  Varun Sundar Rabindranath authored Mar 20, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  0cfe7d38
- [V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) · 0c6f5023
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  0c6f5023
- Enforce that TP > 1 is not supported for Mamba2 if Quantization is Enabled. (#14617) · 06dd0825
  Yu Chin Fabian Lim authored Mar 21, 2025
```
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
```
  06dd0825