Commits · 5c1e496a75138f6b7663cf56d733b6f9e06d7d66 · OpenDAS / vllm_cscc · GitLab

24 Sep, 2025 40 commits
- [MISC] replace c10::optional with std::optional (#25602) · 5c1e496a
  Shiyan Deng authored Sep 24, 2025
```
Signed-off-by: Shiyan Deng <dsy842974287@meta.com>
```
  5c1e496a
- Improve `--help` for enhanced user experience (#24903) · e7f27ea6
  Harry Mellor authored Sep 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e7f27ea6
- [Refactor] Use DeepGEMM Col Major TMA Aligned Tensor (#25517) · 1f291412
  Wentao Ye authored Sep 24, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  1f291412
- feat: BF16 FlashInfer Fused Cutlass MOE for Hopper and Blackwell Expert Parallel (#25503) · 6160ba41
  Duncan Moss authored Sep 24, 2025
```
Signed-off-by: Duncan Moss <djm.moss@gmail.com>
```
  6160ba41
- [Logging] Improve log for when DeepEP HT disables CUDA Graphs (#25531) · fea80060
  Tyler Michael Smith authored Sep 24, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  fea80060
- [V0 Deprecation] Remove unused classes in attention (#25541) · e6750d0b
  Woosuk Kwon authored Sep 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  e6750d0b
- [Docs] Enable `fail_on_warning` for the docs build in CI (#25580) · 8c853050
  Harry Mellor authored Sep 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8c853050
- Suppress benign cuBLAS warning when capturing cudagraphs with DBO (#25596) · f84a472a
  Sage Moore authored Sep 24, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  f84a472a
- Support mnnvl all2allv from Flashinfer (#21003) · 54e42b72
  Shu Wang authored Sep 24, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
```
  54e42b72
- [Bugfix] add cache model when from object storage get model (#24764) · 2dda3e35
  rongfu.leng authored Sep 25, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  2dda3e35
- Fixes and updates to bench_per_token_quant_fp8 (#25591) · d83f3f7c
  Michael Goin authored Sep 24, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  d83f3f7c
- [ROCm][Build][Bugfix] Fix ROCm base docker whls installation order (#25415) · 302eb941
  Gregory Shtrasberg authored Sep 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  302eb941
- [ROCm][Bugfix] Only enable +rms_norm based on aiter if not explicitly disabled (#25275) · 487745ff
  Gregory Shtrasberg authored Sep 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  487745ff
- [Misc] Improve type annotations for jsontree (#25577) · 9313be50
  Cyrus Leung authored Sep 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9313be50
- Move `DeviceConfig`, `ObservabilityConfig`, `SpeechToTextConfig` to their own files (#25564) · 8938774c
  Harry Mellor authored Sep 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8938774c
- [Bugfix] Fix DeepSeekV31ToolParser to correctly parse multiple tools in... · e18b714b
  Tao Hui authored Sep 24, 2025
```
[Bugfix] Fix DeepSeekV31ToolParser to correctly parse multiple tools in non-streaming output (#25405)
Signed-off-by: taohui <taohui3@gmail.com>
```
  e18b714b
- [docs] fix nixl kv_connector_extra_config.backends key (#25565) · b1068903
  Peter Pan authored Sep 24, 2025
```
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  b1068903
- [Benchmark] Fix regression in structured output benchmark (#25500) · 16429950
  Russell Bryant authored Sep 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  16429950
- [Bug] fix import and unit test (#25558) · 58c360d9
  Jonas M. Kübler authored Sep 24, 2025
```
Signed-off-by: Jonas M. Kübler <44084297+jmkuebler@users.noreply.github.com>
```
  58c360d9
- [Bugfix] Fix dummy video number of frames calculation (#25553) · 42488dae
  Roger Wang authored Sep 24, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
  42488dae
- [misc] update the warning message (#25566) · b67dece2
  youkaichao authored Sep 24, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b67dece2
- [BugFix] Potential Fix for FA3 full-cudagraph IMA (#25490) · 2338daff
  Lucas Wilkinson authored Sep 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  2338daff
- [V0 Deprecation] Remove max_seq_len_to_capture (#25543) · 2e19a848
  Woosuk Kwon authored Sep 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2e19a848
- [CI/Build] add nightly prime-rl integration tests (#25207) · 77a7fce1
  Jackmin801 authored Sep 24, 2025
```
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  77a7fce1
- [Misc]] Move processing context to multimodal directory (#25548) · 6488f348
  Cyrus Leung authored Sep 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6488f348
- [CI/Build] Fix v1 OOT registration test (#25547) · 27ec3c78
  Isotr0py authored Sep 24, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  27ec3c78
- [Bugfix][CPU] Skip unsupported custom op register on CPU (#25534) · 1cbcfb94
  Li, Jiang authored Sep 24, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  1cbcfb94
- [Misc] Retry HF processing if "Already borrowed" error occurs (#25535) · fed8a9b1
  Cyrus Leung authored Sep 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  fed8a9b1
- [TPU][Bugfix] fix the missing apply_model in tpu worker (#25526) · 190c45a6
  Chengji Yao authored Sep 23, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  190c45a6
- [Bugfix] [Frontend] Cleanup gpt-oss non-streaming chat tool calls (#25514) · 5caaeb71
  Ben Browning authored Sep 23, 2025
```
Signed-off-by: Ben Browning <bbrownin@redhat.com>
```
  5caaeb71
- [Perf] Fix jit compiles at runtime of fla gated delta rule (#25432) · d747c2ef
  Corey Lowman authored Sep 23, 2025
```
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  d747c2ef
- [Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b
  Benjamin Chislett authored Sep 23, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
```
  c30b405b
- [KV sharing] Re-land Gemma3n model changes from #22628 (#24357) · 77d90699
  Yong Hoon Shin authored Sep 23, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  77d90699
- [fix]: add Arm 4bit fused moe support (#23809) · 359d2930
  Nikhil Gupta authored Sep 24, 2025
```
Signed-off-by: Nikhil Gupta <nikhil.gupta2@arm.com>
```
  359d2930
- [BugFix] Fix MLA assert with CUTLASS MLA (#25478) · 9df8da54
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9df8da54
- [Compile] Fix AMD Compile Error (#25518) · bf68fd76
  Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  bf68fd76
- [Core] Support weight_loader_v2 for `UnquantizedLinearMethod` (#23036) · de94289a
  Kyle Sayers authored Sep 24, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  de94289a
- [Bugfix] Use a separate FlashInfer workspace buffer for trtllm-gen (#25520) · 19836092
  Benjamin Chislett authored Sep 23, 2025
  
  19836092
- [V1][Metrics] Add per-request TPOT histogram (#24015) · d06b5a95
  baxingpiaochong authored Sep 24, 2025
```
Signed-off-by: baxingpiaochong <771405853@qq.com>
```
  d06b5a95
- [Model] Support SeedOss Reason Parser (#24263) · be0bb568
  0xNullPath authored Sep 24, 2025
```
Signed-off-by: Yan Lu <luyan@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  be0bb568