Commits · e9056056fbacecbac4318bd0323745fdd7fe55b6 · OpenDAS / vllm_cscc

"deploy/inference-gateway/epp/Dockerfile" did not exist on "b31b5b5663f7712d07df656b0b01510f1f10fb55"

22 Nov, 2025 4 commits
- [Model Runner V2] Limit cudagraph size to max decode batch size (#29221) · e9056056
  Woosuk Kwon authored Nov 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e9056056
- [Bugfix] Fix block size in block_table with PCP (#29094) · 5c8f2adf
  Jie Luo authored Nov 22, 2025
```
Signed-off-by: Livinfly <luojie3m@gmail.com>
```
  5c8f2adf
- [CI] Bug: Fix triton import issue (#29202) · 1d34eb11
  Wentao Ye authored Nov 21, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  1d34eb11
- [BugFix] Fix Eagle `IndexError: list index out of range` for even `num_speculative_tokens` (#29102) · 30d64662
  Lucas Wilkinson authored Nov 21, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  30d64662
21 Nov, 2025 15 commits
- [Model Runner V2] Optimize Gumbel Sampling Kernel (#29210) · e9af6ba6
  Woosuk Kwon authored Nov 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e9af6ba6
- [KV Connector] Fix async connector prefix cache metrics (#28585) · c6fa3895
  Mark McLoughlin authored Nov 21, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
```
  c6fa3895
- Default model load/config/tokenizer to `mistral` format if relevant files exist (#28659) · 57430fc9
  Julien Denize authored Nov 21, 2025
```
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
Signed-off-by: Julien Denize <40604584+juliendenize@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  57430fc9
- [Chore] Fix pre-commit error after #25266 (#29190) · 1bed891f
  Woosuk Kwon authored Nov 21, 2025
  
  1bed891f
- [Log] Optimize startup log (#28948) · a42ab317
  Wentao Ye authored Nov 21, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a42ab317
- GPU Model Runner V2 (#25266) · 30b44a15
  Woosuk Kwon authored Nov 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  30b44a15
- [Misc] Move dynamic seed initialization to `EngineArgs` (#29165) · d7219bcd
  Cyrus Leung authored Nov 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d7219bcd
- [Doc] Update plugin doc (#28532) · 4050bae4
  wangxiyuan authored Nov 21, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  4050bae4
- fix cross attention (#28346) · fc9f821d
  who who who authored Nov 21, 2025
```
Signed-off-by: fsx950223 <fsx950223@outlook.com>
```
  fc9f821d
- [Core] Align whisper closer to other multimodal models (#27292) · cca2d2cd
  Russell Bryant authored Nov 21, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cca2d2cd
- Revert "[Redo] #26368 (#28771)" (#29121) · 30b9c677
  Jialin Ouyang authored Nov 20, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  30b9c677
- [V0 Deprecation] Remove `best_of` (#29090) · 56e96b37
  Cyrus Leung authored Nov 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  56e96b37
- [CI] Fix mypy for `vllm/v1/worker` (#29037) · 56669c1f
  Wentao Ye authored Nov 20, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  56669c1f
- [AITER] [ROCm] Fix crash when loading llama4 model with old aiter version... · ed6ae1e3
  Xiao Li authored Nov 20, 2025
```
[AITER] [ROCm] Fix crash when loading llama4 model with old aiter version installed, fallback to forward_native implementation (#29124)
Signed-off-by: Xiao Li <ilx@meta.com>
```
  ed6ae1e3
- [LoRA][2/2]Remove LoRA extra vocab (#28545) · 9875be64
  Jee Jee Li authored Nov 21, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  9875be64
20 Nov, 2025 9 commits
- [KVConnector][Core] Support cross-layer KV blocks (#27743) · 64746471
  Or Ozeri authored Nov 20, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  64746471
- [Bugfix] Fix spec decode memory regression after #28549 (#28819) · 93c8672c
  Zhewen Li authored Nov 20, 2025
```
Signed-off-by: zhewenli <zhewenli@meta.com>
```
  93c8672c
- [RL] Add Pause and Resume Generation for Asynchronous RL Training (#28037) · 371b1d4c
  Samit authored Nov 20, 2025
```
Signed-off-by: SamitHuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Signed-off-by: samithuang <285365963@qq.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  371b1d4c
- [BugFix] kv_offloading: Fix bug in loading of partial cpu blocks (#28951) · c0c2dd1e
  Or Ozeri authored Nov 20, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  c0c2dd1e
- [ROCm] Add AMD GPU support on Deepseek v3.2 and SparseMLA (#26670) · 06c20c99
  Pleaplusone authored Nov 20, 2025
```
Signed-off-by: ganyi <ygan@amd.com>
```
  06c20c99
- [Bugfix] Revert custom attention mask for gemma3-mm (#28995) · 64192d56
  Isotr0py authored Nov 20, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  64192d56
- [Feat] Iteration-level profiling for Torch and CUDA profiler (#28987) · fcbcba6c
  Benjamin Chislett authored Nov 19, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  fcbcba6c
- [AMD] Use Decoupled Kernel Block Size to Support AITER MLA block_size=1 (#27715) · 3fb0d909
  Qiang Zhang authored Nov 20, 2025
```
Signed-off-by: chiangzhang <chiangzhang@tencent.com>
```
  3fb0d909
- [GC Debugger] Simply and improve GC Debugger Utils (#29029) · 537cc635
  Jialin Ouyang authored Nov 19, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  537cc635
19 Nov, 2025 8 commits

[BugFix] Ray with multiple nodes (#28873) · cdeec2e6
Julien Denize authored Nov 19, 2025
```
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
```
cdeec2e6

[Feature] Prefill Context Parallel (PCP) basic support (#28718) · 2fd893b4

Qiu authored Nov 20, 2025


Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Signed-off-by: FENP <yuanyongjie.yyj@antgroup.com>
Signed-off-by: LookAround <lixushi@huawei.com>
Signed-off-by: Jingchun Gao <gaojingchun1@huawei.com>
Signed-off-by: zhenwenqi2024 <zhenwenqi_2022@qq.com>
Co-authored-by: FENP <yuanyongjie.yyj@antgroup.com>
Co-authored-by: LookAround <lixushi@huawei.com>
Co-authored-by: Jingchun Gao <gaojingchun1@huawei.com>
Co-authored-by: zhenwenqi2024 <zhenwenqi_2022@qq.com>
Co-authored-by: Jingchun Gao <63247409+gjc0824@users.noreply.github.com>

2fd893b4

Eagle: MM Cuda Graphs with MRope (#28896) · 02f5903b

Izzy Putterman authored Nov 19, 2025


Signed-off-by: Izzy Putterman <iputterman@nvidia.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

02f5903b

Upstreaming aiter triton attention backend as a new backend (#28701) · ac10fd3c
Aleksandr Malyshev authored Nov 19, 2025
```
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
```
ac10fd3c
[Core] Reuse created spec tokens lists to mitigate GC cost (#28917) · 3319a493
Jialin Ouyang authored Nov 19, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
3319a493
[BugFix] Fix async-scheduling + FlashAttn MLA (#28990) · 48fc8b1e
Lucas Wilkinson authored Nov 19, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
48fc8b1e
[Attention] FlashAttention ViT support, make default backend (#28763) · 4c23690f
Matthew Bonanni authored Nov 18, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
4c23690f
[Core] Switch Flat logprob control from environment variable to SamplingParams (#28914) · 40b6b38f
Jialin Ouyang authored Nov 18, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
40b6b38f

18 Nov, 2025 4 commits
- Replace `torch.cuda.Event` with `torch.Event` for better hardware compatibility (#26985) · 2a2d5d27
  Kunshang Ji authored Nov 19, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  2a2d5d27
- [FEAT] [AITER] [ROCm] integrate aiter sampling ops (#26084) · 0af3d4f0
  vllmellm authored Nov 19, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  0af3d4f0
- [Minor] Rename `ec_producer` field to `is_ec_producer` (#28884) · da8dadf6
  Nick Hill authored Nov 18, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  da8dadf6
- [Model] Add Gemma3 GGUF multimodal support (#27772) · c2612371
  Luciano Martins authored Nov 18, 2025
```
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  c2612371