Commits · d20b0c139cbf66fce265ed66d5f8c82ab73dff3a · OpenDAS / vllm_cscc

17 Mar, 2025 10 commits
- Add patch merger (#14957) · d20b0c13
  Patrick von Platen authored Mar 17, 2025
  
  d20b0c13
- [Doc] Fix misleading log during multi-modal profiling (#14955) · 166a168b
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  166a168b
- [Bugfix] Fix Ultravox on V1 (#14929) · 868a8c5b
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  868a8c5b
- [V1][TPU] Apply the ragged paged attention kernel fix and remove the padding. (#14846) · b4ad56c1
  iefgnoix authored Mar 17, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b4ad56c1
- [Misc][XPU] Use None as device capacity for XPU (#14932) · 9b87a579
  Yan Ma authored Mar 17, 2025
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  9b87a579
- [V1] Remove input cache client (#14864) · b539222d
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  b539222d
- [V1] [Spec Decode] Support random sampling for spec decode (#13933) · 8d6cf895
  Lily Liu authored Mar 16, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d6cf895
- [BugFix] Fix MLA + V1 + TP==1 causing reinitialization of cuda context (#14910) · 1e799b7e
  Lucas Wilkinson authored Mar 16, 2025
  
  1e799b7e
- [V1][Minor] Add __repr__ to ConstantList (#14907) · 7f6c5ee0
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7f6c5ee0
- [V1] Optimize the overhead of rewinding (#14905) · faa02757
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  faa02757
16 Mar, 2025 9 commits
- [V1][BugFix] Detect interleaved sliding window attention (#14896) · 31060b27
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  31060b27
- [BugFix][V1] Fix overhead related to bad_words sampling when not in use (#14894) · fc1f6771
  Nick Hill authored Mar 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  fc1f6771
- Revert "[Bugfix] Limit profiling run sequence length by max_model_len (#14785) (#14892) · f6137adb
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f6137adb
- [Bugfix] Explicitly disable Phi-4-multimodal in V1 (#14889) · e53b1350
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e53b1350
- [Bugfix] Limit profiling run sequence length by max_model_len (#14785) · d30aa7e9
  Kyle Sayers authored Mar 16, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  d30aa7e9
- [BugFix] Fix torch distributed stateless PG backend init (#14870) · b82662d9
  Nick Hill authored Mar 15, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b82662d9
- [Kernel] Add more tuned configs (#14877) · 71c1e071
  Simon Mo authored Mar 15, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  71c1e071
- [V1] Remove V0 fallback for mistral-tokenizer (#14873) · b30c75dd
  Roger Wang authored Mar 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  b30c75dd
- [VLM] Clean up Phi-4-MM ViT implementation (#14812) · def232e1
  Isotr0py authored Mar 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  def232e1
15 Mar, 2025 13 commits
- [VLM] Merged multi-modal processor for Pixtral (#12211) · 61c6a5a7
  Rémi Delacourt authored Mar 15, 2025
```
Signed-off-by: remi <remi@mistral.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61c6a5a7
- [Core] Expose API endpoint `/is_sleeping` (#14312) · 74bc397b
  Jun Duan authored Mar 15, 2025
```
Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>
```
  74bc397b
- [VLM] Limit multimodal input cache by memory (#14805) · 3556a414
  Cyrus Leung authored Mar 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3556a414
- [Bugfix] EAGLE output norm bug (#14464) · 9ed6ee92
  Bryan Lu authored Mar 14, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  9ed6ee92
- [V1][Structured Output] calculate vocab_size eagerly (#14851) · 4c7629ca
  Aaron Pham authored Mar 15, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  4c7629ca
- [Attention] Get rid of mla cache alignment (#14842) · 5952d8ab
  Lucas Wilkinson authored Mar 15, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  5952d8ab
- [CPU] Support FP8 KV cache (#14741) · a2ae4965
  Li, Jiang authored Mar 15, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a2ae4965
- [V1] V1 Enablement Oracle (#13726) · d4d93db2
  Robert Shaw authored Mar 15, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  d4d93db2
- [Misc] Remove misleading message in gemma2 and gemma3 (#14850) · 97ac781c
  Isotr0py authored Mar 15, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  97ac781c
- Disable outlines cache by default (#14837) · 776dcec8
  Russell Bryant authored Mar 14, 2025
  
  776dcec8
- Revert "[Model] Mamba2 Prefill Performance Tweaks: Fixing Flurry of U… (#14848) · ccf02fcb
  Tyler Michael Smith authored Mar 14, 2025
  
  ccf02fcb
- [Bugfix][V1] Fix flashinfer sampling (#14815) · acaea3bb
  DefTruth authored Mar 15, 2025
  
  acaea3bb
- [Bugfix] Fix torch_xla in V0 which can't handle None seed introduced … (#14844) · dd344e03
  yarongmu-google authored Mar 14, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  dd344e03
14 Mar, 2025 8 commits

Update to torch==2.6.0 (#12721) · 14f301b5

Michael Goin authored Mar 14, 2025


Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

14f301b5

[Model] Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14778) · fe66b347
Chih-Chieh Yang authored Mar 14, 2025
```
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
```
fe66b347
[Frontend] track server_load (#13950) · 73deea2f
daniel-salib authored Mar 14, 2025

73deea2f
[Frontend] Fix log message to use http vs https (#14774) · 0b0d6421
Russell Bryant authored Mar 14, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
0b0d6421
[V1] Fix vocab size calculation for structured output (#14826) · 1140991a
Russell Bryant authored Mar 14, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
1140991a
[BugFix]: properly catch templating error when preprocess input (#13976) · fd8e055f
Guillaume Calmettes authored Mar 14, 2025
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
fd8e055f
[VLM] Various cleanup and fixes (#14806) · ab93f136
Cyrus Leung authored Mar 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
ab93f136
[V1][Minor] Minor code cleanup for scheduling metrics (#14800) · c77620d2
Woosuk Kwon authored Mar 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
c77620d2