Commits · 5eeabc2a4400fde9b030f2f72746a2b03db059bd · OpenDAS / vllm_cscc · GitLab

17 Mar, 2025 18 commits
- [Bugfix] Fix bnb quantization for models with both HF-format and Mistral-format weights (#14950) · 5eeabc2a
  Tristan Leclercq authored Mar 18, 2025
  
  5eeabc2a
- [V1] Guard Against Main Thread Usage (#14972) · e41e1602
  Robert Shaw authored Mar 17, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  e41e1602
- [Bugfix] Fix interface for Olmo2 on V1 (#14976) · 5340b0e2
  Roger Wang authored Mar 17, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  5340b0e2
- [Bugfix] Make Gemma3 MM V0 only for now (#14971) · 37e38061
  Roger Wang authored Mar 17, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  37e38061
- [Fix][Structured Output] using vocab_size to construct matcher (#14868) · c0efdd65
  Aaron Pham authored Mar 17, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
```
  c0efdd65
- [Bugfix][Model] Mixtral: use unused head_dim config argument (#14961) · aaaec52a
  Quentin authored Mar 17, 2025
```
Signed-off-by: Quentin Torroba <quentin.torroba@mistral.ai>
```
  aaaec52a
- [Bugfix] Fix precommit - line too long in pixtral.py (#14960) · e1eb45d3
  Tyler Michael Smith authored Mar 17, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e1eb45d3
- [V1] Default MLA to V1 (#14921) · 89fca671
  Simon Mo authored Mar 17, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  89fca671
- Add patch merger (#14957) · d20b0c13
  Patrick von Platen authored Mar 17, 2025
  
  d20b0c13
- [Doc] Fix misleading log during multi-modal profiling (#14955) · 166a168b
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  166a168b
- [Bugfix] Fix Ultravox on V1 (#14929) · 868a8c5b
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  868a8c5b
- [V1][TPU] Apply the ragged paged attention kernel fix and remove the padding. (#14846) · b4ad56c1
  iefgnoix authored Mar 17, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b4ad56c1
- [Misc][XPU] Use None as device capacity for XPU (#14932) · 9b87a579
  Yan Ma authored Mar 17, 2025
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  9b87a579
- [V1] Remove input cache client (#14864) · b539222d
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  b539222d
- [V1] [Spec Decode] Support random sampling for spec decode (#13933) · 8d6cf895
  Lily Liu authored Mar 16, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d6cf895
- [BugFix] Fix MLA + V1 + TP==1 causing reinitialization of cuda context (#14910) · 1e799b7e
  Lucas Wilkinson authored Mar 16, 2025
  
  1e799b7e
- [V1][Minor] Add __repr__ to ConstantList (#14907) · 7f6c5ee0
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7f6c5ee0
- [V1] Optimize the overhead of rewinding (#14905) · faa02757
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  faa02757
16 Mar, 2025 9 commits
- [V1][BugFix] Detect interleaved sliding window attention (#14896) · 31060b27
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  31060b27
- [BugFix][V1] Fix overhead related to bad_words sampling when not in use (#14894) · fc1f6771
  Nick Hill authored Mar 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  fc1f6771
- Revert "[Bugfix] Limit profiling run sequence length by max_model_len (#14785) (#14892) · f6137adb
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f6137adb
- [Bugfix] Explicitly disable Phi-4-multimodal in V1 (#14889) · e53b1350
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e53b1350
- [Bugfix] Limit profiling run sequence length by max_model_len (#14785) · d30aa7e9
  Kyle Sayers authored Mar 16, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  d30aa7e9
- [BugFix] Fix torch distributed stateless PG backend init (#14870) · b82662d9
  Nick Hill authored Mar 15, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b82662d9
- [Kernel] Add more tuned configs (#14877) · 71c1e071
  Simon Mo authored Mar 15, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  71c1e071
- [V1] Remove V0 fallback for mistral-tokenizer (#14873) · b30c75dd
  Roger Wang authored Mar 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  b30c75dd
- [VLM] Clean up Phi-4-MM ViT implementation (#14812) · def232e1
  Isotr0py authored Mar 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  def232e1
15 Mar, 2025 13 commits
- [VLM] Merged multi-modal processor for Pixtral (#12211) · 61c6a5a7
  Rémi Delacourt authored Mar 15, 2025
```
Signed-off-by: remi <remi@mistral.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61c6a5a7
- [Core] Expose API endpoint `/is_sleeping` (#14312) · 74bc397b
  Jun Duan authored Mar 15, 2025
```
Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>
```
  74bc397b
- [VLM] Limit multimodal input cache by memory (#14805) · 3556a414
  Cyrus Leung authored Mar 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3556a414
- [Bugfix] EAGLE output norm bug (#14464) · 9ed6ee92
  Bryan Lu authored Mar 14, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  9ed6ee92
- [V1][Structured Output] calculate vocab_size eagerly (#14851) · 4c7629ca
  Aaron Pham authored Mar 15, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  4c7629ca
- [Attention] Get rid of mla cache alignment (#14842) · 5952d8ab
  Lucas Wilkinson authored Mar 15, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  5952d8ab
- [CPU] Support FP8 KV cache (#14741) · a2ae4965
  Li, Jiang authored Mar 15, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a2ae4965
- [V1] V1 Enablement Oracle (#13726) · d4d93db2
  Robert Shaw authored Mar 15, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  d4d93db2
- [Misc] Remove misleading message in gemma2 and gemma3 (#14850) · 97ac781c
  Isotr0py authored Mar 15, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  97ac781c
- Disable outlines cache by default (#14837) · 776dcec8
  Russell Bryant authored Mar 14, 2025
  
  776dcec8
- Revert "[Model] Mamba2 Prefill Performance Tweaks: Fixing Flurry of U… (#14848) · ccf02fcb
  Tyler Michael Smith authored Mar 14, 2025
  
  ccf02fcb
- [Bugfix][V1] Fix flashinfer sampling (#14815) · acaea3bb
  DefTruth authored Mar 15, 2025
  
  acaea3bb
- [Bugfix] Fix torch_xla in V0 which can't handle None seed introduced … (#14844) · dd344e03
  yarongmu-google authored Mar 14, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  dd344e03