Commits · b1cc4dfef57ac0f3e27575215f771f0c25706b81 · OpenDAS / vllm_cscc · GitLab

13 Mar, 2025 13 commits
- [VLM] Support loading InternVideo2.5 models as original InternVLChatModel (#14738) · b1cc4dfe
  Isotr0py authored Mar 13, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b1cc4dfe
- [VLM] Support pan-and-scan for Gemma3 multi-modal processor (#14672) · 38240392
  Cyrus Leung authored Mar 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  38240392
- [Bugfix] fix benchmark moe (#14653) · a73122de
  Jee Jee Li authored Mar 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a73122de
- [CI/Build] Delete ultravox LoRA test (#14730) · bd44b812
  Jee Jee Li authored Mar 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  bd44b812
- [Bugfix] Fix chunked prefill for GGUF (#14666) · 55211b01
  Szymon Ożóg authored Mar 13, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  55211b01
- [Quant] Bamba SupportsQuant (#14698) · 5d043c16
  Kyle Sayers authored Mar 13, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  5d043c16
- [Quant] BartModel SupportsQuant (#14699) · 36d1ccb2
  Kyle Sayers authored Mar 13, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  36d1ccb2
- [V1][TPU] Add assertion on multi-step-scheduler (#14707) · 1bc3b739
  Siyuan Liu authored Mar 12, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  1bc3b739
- [Config][Disaggregated] Add timeout configuration for the torch.store and add... · 1bd32bc8
  Mathis Felardos authored Mar 13, 2025
```
[Config][Disaggregated] Add timeout configuration for the torch.store and add KVTransferConfig.kv_connector_extra_config (#14367)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
```
  1bd32bc8
- [BugFix][TritonMLA] Process weights after model loading for GGUF (#14555) · 128bf752
  TY-AMD authored Mar 13, 2025
```
Signed-off-by: TianyuanWu <Tianyuan.Wu@amd.com>
```
  128bf752
- [ROCm][FP8] Fix for adjustments needed only for fnuz (#14689) · a94a699c
  Gregory Shtrasberg authored Mar 12, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  a94a699c
- Add ray[data] as tpu dependency (#14691) · ab426ec9
  Richard Liu authored Mar 12, 2025
```
Signed-off-by: <ricliu@google.com>
Signed-off-by: Richard Liu <ricliu@google.com>
```
  ab426ec9
- [bugfix] fixup warning message for plugged schedulers for v1 (#14700) · 165290d3
  Joe Runde authored Mar 12, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  165290d3
12 Mar, 2025 19 commits
- [release] Add force remove for TPU logs (#14697) · ce201246
  Kevin H. Luu authored Mar 12, 2025
  
  ce201246
- [V1] Allow sliding window + prefix caching (#13069) · 53be4a86
  Woosuk Kwon authored Mar 12, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  53be4a86
- [BugFix][V1] Fix parallel sampling finishing/aborts (#14512) · f5d3acd4
  Nick Hill authored Mar 12, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  f5d3acd4
- [FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash... · 916836bb
  TJian authored Mar 13, 2025
```
[FEAT] [ROCm] [Embedding] Add encoder-only model support into ROCm Flash Attention to enable embedding models. (#14664)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  916836bb
- [ROCm] Enable chunked prefill/paged attention in MLA on ROCm (#14316) · d9f83d62
  Sage Moore authored Mar 12, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  d9f83d62
- [Bugfix] Missing thumbnail from NVLM-D processor (#14633) · 4a754fcf
  ameyanjarlekar authored Mar 12, 2025
```
Signed-off-by: ameyanjarlekar <aanjarlekar@nvidia.com>
```
  4a754fcf
- [Model] Add support for Gemma 3 (#14660) · c0c25e25
  Woosuk Kwon authored Mar 12, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c0c25e25
- [ROCm][Bugfix] Ensure that the moe_wna16_gemm kernel is not built on ROCm platforms. (#14629) · 45f3f3f5
  Sage Moore authored Mar 12, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  45f3f3f5
- [CPU] Upgrade CPU backend to torch-2.6 (#13381) · ff47aab0
  Li, Jiang authored Mar 12, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  ff47aab0
- [Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) · debd6bbf
  Pavani Majety authored Mar 11, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  debd6bbf
- [V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due... · 5c538c37
  Benjamin Chislett authored Mar 12, 2025
```
[V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due to batch indexing (#14645)
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  5c538c37
- [Kernel] GGUF MoE kernel (#14613) · e22ee1e7
  Szymon Ożóg authored Mar 12, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  e22ee1e7
- [Core] Refactor `QKVCrossParallelLinear` implementation to support BNB 4-bit quantization (#14545) · e392d858
  Isotr0py authored Mar 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e392d858
- [V1][Core] Support MistralTokenizer for Structured Output (#14625) · 77a318bd
  Aaron Pham authored Mar 11, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  77a318bd
- [Model] Extend Ultravox to accept audio longer than 30s (#13631) · 80e78d02
  Farzad Abdolhosseini authored Mar 11, 2025
```
Signed-off-by: Farzad Abdolhosseini <farzad@fixie.ai>
```
  80e78d02
- [Doc] Update benchmarks README (#14646) · 4a42b9f5
  Jennifer Zhao authored Mar 11, 2025
```
Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  4a42b9f5
- [core][V1] pluggable scheduler (#14466) · 47532cd9
  Joe Runde authored Mar 11, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  47532cd9
- [Feature] Add `vllm bench` CLI (#13993) · 36e0c8f7
  Randy Chen authored Mar 11, 2025
```
Signed-off-by: Randy Chen <acad.randyjhc@gmail.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  36e0c8f7
- [release] Add commands to clean up logs on TPU release node (#14642) · 9f583e36
  Kevin H. Luu authored Mar 11, 2025
  
  9f583e36
11 Mar, 2025 8 commits
- [Bugfix][V1][PP] Only warmup sampler at last PP rank (#14643) · b706d898
  Cody Yu authored Mar 11, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  b706d898
- [V1][TPU] Pad the block_table.shape[1] so the ragged paged attention can handle correctly (#14597) · 863d315c
  iefgnoix authored Mar 11, 2025
  
  863d315c
- Fix run_tpu_test (#14641) · d374f04a
  Richard Liu authored Mar 11, 2025
```
Signed-off-by: <ricliu@google.com>
Signed-off-by: Richard Liu <ricliu@google.com>
```
  d374f04a
- [V1] Delay all xgrammar usage until needed (#14616) · 61a01b27
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  61a01b27
- fix some typos : supported_head_sizes (#14627) · 53056731
  Yang.Tao authored Mar 12, 2025
  
  53056731
- [V1] Remove cache from StructuredOutputManager (#14622) · 4cbf2867
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4cbf2867
- [Hardware][Intel GPU] upgrade IPEX dependency to 2.6.10. (#14564) · c6e14a61
  Kunshang Ji authored Mar 11, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  c6e14a61
- [BugFix/Build] Fix sparse kernels not getting built on hopper (#14572) · 07b4b7a3
  Lucas Wilkinson authored Mar 11, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  07b4b7a3