Commits · 54cc46f3ebcde68f072c37f2142ff37d60077ff0 · OpenDAS / vllm_cscc · GitLab

14 Mar, 2025 19 commits
- [Bugfix] Fix small typo in the example of Streaming delimiter (#14793) · 54cc46f3
  WeiCheng authored Mar 14, 2025
  
  54cc46f3
- [Misc] Clean up type annotation for `SupportsMultiModal` (#14794) · 601bd326
  Cyrus Leung authored Mar 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  601bd326
- [BugFix]Fix performance serving benchmark when enable profiling (#14737) · 09269b31
  Li Wang authored Mar 14, 2025
```
Signed-off-by: wangli <wangli858794774@gmail.com>
```
  09269b31
- [Bugfix][Kernel][CPU] Fix num_tokens in CPU rotary embedding kernel (#14667) · 27b50f1f
  Thien Tran authored Mar 14, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  27b50f1f
- [Attention] MLA get rid of materialization (#14770) · 9532c498
  Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9532c498
- [CI] Fix missing example model id in processor test (#14787) · 0c2af17c
  Roger Wang authored Mar 13, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  0c2af17c
- [Feature] Add visionarena offline support for benchmark_throughput (#14654) · a6e0d096
  Jennifer Zhao authored Mar 13, 2025
```
Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Jennifer Zhao <JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  a6e0d096
- [Neuron] flatten test parameterization for neuron attention kernels (#14712) · d3d49562
  Liangfu Chen authored Mar 13, 2025
  
  d3d49562
- [Misc][Minor] Simplify `SamplingParams.__post_init__()` (#14772) · 4059adc3
  Nick Hill authored Mar 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  4059adc3
- [ci] Reduce number of tests in fastcheck (#14782) · f1f632d9
  Kevin H. Luu authored Mar 13, 2025
  
  f1f632d9
- [Bugfix][IPEX] Add `VLLM_CPU_MOE_PREPACK` to allow disabling MoE prepack when... · 95d680b8
  Thien Tran authored Mar 14, 2025
```
[Bugfix][IPEX] Add `VLLM_CPU_MOE_PREPACK` to allow disabling MoE prepack when CPU does not support it (#14681)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  95d680b8
- [Kernel] [V1] Further optimizations to ROCm (Triton) Backend to better handle GQA. (#14431) · fb4c7f8e
  Thomas Parnell authored Mar 14, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Jan van Lunteren <jvl@zurich.ibm.com>
Co-authored-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
```
  fb4c7f8e
- [Kernel] LoRA - Enable CUDAGraphs for V1 (#14626) · 0b1cfa61
  Varun Sundar Rabindranath authored Mar 13, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  0b1cfa61
- [V1] Temporarily disable FlashInfer Rejection Sampler (#14788) · 32ef4983
  Woosuk Kwon authored Mar 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  32ef4983
- [V1] Move OOM check into sampler run (#14728) · ad19c8a0
  Roger Wang authored Mar 13, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  ad19c8a0
- forward fix PR 14245, restore build on ROCm 6.2 (#14709) · 2a602b05
  Jeff Daily authored Mar 13, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  2a602b05
- [V1] TPU - Enable prefix caching by default (#14773) · 7888e1d0
  Alexander Matveev authored Mar 13, 2025
  
  7888e1d0
- [Doc] Fix small typo in Transformers fallback (#14791) · 60c872d4
  Chen Zhang authored Mar 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  60c872d4
- [Doc] Fix typo in documentation (#14783) · 3fb17d26
  yasu52 authored Mar 13, 2025
```
Signed-off-by: yasu52 <tsuguro4649@gmail.com>
```
  3fb17d26
13 Mar, 2025 19 commits
- [Attention] Remove slow setattr in MLA (#14769) · d47807ba
  Lucas Wilkinson authored Mar 13, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  d47807ba
- [V1] Detokenizer: Respect Stop Tokens + not include_stop_str_in_output (#14624) · 02fcaa3d
  afeldman-nm authored Mar 13, 2025
```
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
```
  02fcaa3d
- [V1][Core] using cached vocab_size for Structured Outputs (#14630) · 8a4a2efc
  Aaron Pham authored Mar 13, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  8a4a2efc
- [Misc] Clean up processor tests (#14771) · 8e9ffd37
  Cyrus Leung authored Mar 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8e9ffd37
- [V1][Minor] Minor enhancements on scheduler (#14732) · 01b3fd0a
  Woosuk Kwon authored Mar 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  01b3fd0a
- [Bugfix] Fix prompt format of GLM4V (#14539) · f53a0586
  Cyrus Leung authored Mar 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f53a0586
- [VLM] Support loading InternVideo2.5 models as original InternVLChatModel (#14738) · b1cc4dfe
  Isotr0py authored Mar 13, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b1cc4dfe
- [VLM] Support pan-and-scan for Gemma3 multi-modal processor (#14672) · 38240392
  Cyrus Leung authored Mar 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  38240392
- [Bugfix] fix benchmark moe (#14653) · a73122de
  Jee Jee Li authored Mar 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a73122de
- [CI/Build] Delete ultravox LoRA test (#14730) · bd44b812
  Jee Jee Li authored Mar 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  bd44b812
- [Bugfix] Fix chunked prefill for GGUF (#14666) · 55211b01
  Szymon Ożóg authored Mar 13, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  55211b01
- [Quant] Bamba SupportsQuant (#14698) · 5d043c16
  Kyle Sayers authored Mar 13, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  5d043c16
- [Quant] BartModel SupportsQuant (#14699) · 36d1ccb2
  Kyle Sayers authored Mar 13, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  36d1ccb2
- [V1][TPU] Add assertion on multi-step-scheduler (#14707) · 1bc3b739
  Siyuan Liu authored Mar 12, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  1bc3b739
- [Config][Disaggregated] Add timeout configuration for the torch.store and add... · 1bd32bc8
  Mathis Felardos authored Mar 13, 2025
```
[Config][Disaggregated] Add timeout configuration for the torch.store and add KVTransferConfig.kv_connector_extra_config (#14367)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
```
  1bd32bc8
- [BugFix][TritonMLA] Process weights after model loading for GGUF (#14555) · 128bf752
  TY-AMD authored Mar 13, 2025
```
Signed-off-by: TianyuanWu <Tianyuan.Wu@amd.com>
```
  128bf752
- [ROCm][FP8] Fix for adjustments needed only for fnuz (#14689) · a94a699c
  Gregory Shtrasberg authored Mar 12, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  a94a699c
- Add ray[data] as tpu dependency (#14691) · ab426ec9
  Richard Liu authored Mar 12, 2025
```
Signed-off-by: <ricliu@google.com>
Signed-off-by: Richard Liu <ricliu@google.com>
```
  ab426ec9
- [bugfix] fixup warning message for plugged schedulers for v1 (#14700) · 165290d3
  Joe Runde authored Mar 12, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  165290d3
12 Mar, 2025 2 commits
- [release] Add force remove for TPU logs (#14697) · ce201246
  Kevin H. Luu authored Mar 12, 2025
  
  ce201246
- [V1] Allow sliding window + prefix caching (#13069) · 53be4a86
  Woosuk Kwon authored Mar 12, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  53be4a86