Commits · cec418b5df3bf032a83b6a6795e8026d39e199bd · OpenDAS / vllm_cscc

24 Nov, 2025 14 commits

[Model Runner V2] Change Numba AoT to JIT (#29328) · cec418b5
Woosuk Kwon authored Nov 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
cec418b5
[Model Runner V2] Implement Single-step Eagle 1 (#29300) · cc313cb7
Woosuk Kwon authored Nov 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
cc313cb7
[NIXL] Use config to enable telemetry + NIXL version bump (#29305) · 26a46558
Nicolò Lucchesi authored Nov 24, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
26a46558

[Bugfix] properly handle nested json with llama3 tool parser (#27701) · 656516c3

Aydin Abiar authored Nov 24, 2025

Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Signed-off-by: Aydin Abiar <62435714+Aydin-ab@users.noreply.github.com>
Co-authored-by: Aydin Abiar <aydin@anyscale.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

656516c3

[Bugfix] [ROCm] [UX] Reorganize ROCm Backend Selection Logic (#26980) · e48b2e68
vllmellm authored Nov 24, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
e48b2e68

Add option to use unbacked, and backed size obl dynamic shapes for more sounds... · 7a228b53

Laith Sakka authored Nov 24, 2025


Add option to use unbacked, and backed size obl dynamic shapes for more sounds compilation. (#26199)
Signed-off-by: Laith Sakka <lsakka@meta.com>

7a228b53

[EPLB] Optimize EPLB for Async Rearrange Experts (#22179) · 2601f18a
WeiQing Chen authored Nov 24, 2025
```
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: SunChenxiang123 <1291824390@qq.com>
```
2601f18a

[Doc]: fix typos in various files (#29230) · eca7a8fb

Didier Durand authored Nov 24, 2025

Signed-off-by: Didier Durand <durand.didier@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

eca7a8fb

[Bugfix][Rocm] Fix shared expert weight loading failure in DeepSeek-MTP (#27563) · 8005e606
杰兮 authored Nov 24, 2025
```
Signed-off-by: zhyajie <yajizhan@amd.com>
Co-authored-by: zhyajie <yajizhan@amd.com>
```
8005e606
[Feature][Benchmark] add --link-vars can filter when serve_param equal bench_param (#28909) · 68dfe28e
rongfu.leng authored Nov 24, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
68dfe28e
[Core] Deprecate `xformers` (#29262) · 0ff70821
Roger Wang authored Nov 23, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
0ff70821
[ROCm] Support for Whisper v1 with Aiter Unified Attention and Aiter Flash Attention (#28376) · 5253f427
tongqiu authored Nov 24, 2025
```
Signed-off-by: apinge <Tong.Qiu2@amd.com>
```
5253f427

[Model] Add OpenCUA-7B support (#29068) · 30854783

Zero authored Nov 24, 2025

Signed-off-by: lim4349 <rockmanzero@naver.com>
Signed-off-by: Zero <rockmanzero@naver.com>
Co-authored-by: Cloud User <ubuntu@a100-80g-4.novalocal>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

30854783

[LoRA] Optimize 3D MoE logic (#29222) · 1073ba68
Jee Jee Li authored Nov 24, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
1073ba68

23 Nov, 2025 7 commits
- [Model Runner V2] Add apply_temperature option to gumbel_sample (#29276) · 3e1ad406
  Woosuk Kwon authored Nov 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3e1ad406
- [Model Runner V2] Optimize CUDA graph capture time (#29275) · 62d54ba4
  Woosuk Kwon authored Nov 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  62d54ba4
- [Model Runner V2] Support spec decoding [1/N] (#29274) · b004c004
  Woosuk Kwon authored Nov 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b004c004
- [Model Runner V2] Change bookkeeping logic in preparation for spec decoding (#29194) · 7f12c82f
  Woosuk Kwon authored Nov 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7f12c82f
- [Bugfix] Use lazy string reference for DeepseekV3Config in config registry (#28958) · 6fb0215e
  Luke authored Nov 23, 2025
```
Signed-off-by: Luke <yq0536@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  6fb0215e
- [Model Runner V2] Minor fix for cudagraph_utils (#29256) · 20ee418a
  Woosuk Kwon authored Nov 22, 2025
  
  20ee418a
- [Fix] Add SM check to flashinfer MOE backend (#29144) · 5f96c00c
  jiahanc authored Nov 22, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  5f96c00c
22 Nov, 2025 19 commits
- Refactor: Move CUDA graph dispatch logic earlier (#27382) · df78aeef
  Yizhou authored Nov 23, 2025
```
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
```
  df78aeef
- [BugFix] Fix chunked prompt logprobs + preemption (#29071) · 7df331c6
  Nick Hill authored Nov 22, 2025
  
  7df331c6
- [Bugfix] Use HF config fields as fallback when loading Mistral config (#29239) · d1cf8214
  Cyrus Leung authored Nov 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d1cf8214
- [perf][cpu] Accelerate paged attention GEMMs (QK, PV) on Arm CPUs with NEON (#29193) · 730bd353
  Fadi Arafeh authored Nov 22, 2025
```
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
```
  730bd353
- chore: add RTX_PRO_6000 GLM4.6-FP8 kernel tuning (#29240) · f55c76c2
  Federico authored Nov 22, 2025
  
  f55c76c2
- Fix EVS crash when using `video_embeds` inputs in Qwen2.5-VL (#29232) · d84d8f44
  ZiTian Zhao authored Nov 22, 2025
```
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  d84d8f44
- [Misc] Fix pre-commit (#29238) · ae668183
  Cyrus Leung authored Nov 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ae668183
- [BugFix] Fix returned logprobs with spec decode + prefill chunking (#29216) · d44a63c6
  Nick Hill authored Nov 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d44a63c6
- [Attention] Refactor FA `block_size` limitations to hybrid models only (#29084) · 066209a0
  Nicolò Lucchesi authored Nov 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  066209a0
- [tiny] Remove unsupported TRITON_MLA backend from batch invariance (#28832) · 5f7209a7
  Bram Wasti authored Nov 22, 2025
```
Signed-off-by: Bram Wasti <bwasti@meta.com>
Signed-off-by: Bram Wasti <bwasti@fb.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  5f7209a7
- Fix: Resolve circular import in model_loader/utils.py (#29189) · 6965a392
  Nandan Vallamdasu authored Nov 22, 2025
```
Signed-off-by: nandan2003 <nandan.vallamdasu@outlook.com>
Signed-off-by: Nandan Vallamdasu  <nandan.vallamdasu@outlook.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  6965a392
- [Misc] Further clean up chunked prefill and prefix caching init (#29186) · 5a480258
  Cyrus Leung authored Nov 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5a480258
- Handle triton kernel import exception (#29062) · 988ee66b
  jinghanhu authored Nov 22, 2025
  
  988ee66b
- [Frontend][Responses API] Multi-turn (with type: "output_text") support for... · ea38474a
  Mads Kildegård authored Nov 22, 2025
```
[Frontend][Responses API] Multi-turn (with type: "output_text") support for non-harmony requests (#29175)
Signed-off-by: Mads Kildegård <mkildegaard99@gmail.com>
```
  ea38474a
- [responsesAPI] parse reasoning item input (#28248) · 742e9ff6
  Andrew Xia authored Nov 21, 2025
```
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  742e9ff6
- [Model Runner V2] Limit cudagraph size to max decode batch size (#29221) · e9056056
  Woosuk Kwon authored Nov 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e9056056
- [LoRA] Cleanup FusedMoEWithLoRA (#29187) · 1489902b
  Jee Jee Li authored Nov 22, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1489902b
- [Bugfix]Fix a conditional to not check zero value (#28754) · 933f67ec
  Yanan Cao authored Nov 21, 2025
```
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
```
  933f67ec
- [chore][LMCache connector] Remove useless logs from lmcache connector (#29069) · 77e1c035
  Yihua Cheng authored Nov 21, 2025
```
Signed-off-by: ApostaC <yihua98@uchicago.edu>
```
  77e1c035