Commits · 3157aebb63a2e121da6de943754dc95dffd14caa · OpenDAS / vllm_cscc

09 Aug, 2025 5 commits

[Log] Add Warning for Deprecation of DeepGEMM old version (#22194) · 3157aebb
Wentao Ye authored Aug 09, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
3157aebb

Implicit language-model-only mode via limit-mm-per-prompt (#22299) · 08b751ba

Roger Wang authored Aug 08, 2025


Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Andy Xie <andy.xning@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Signed-off-by: Junhao Li <junhao@ubicloud.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Signed-off-by: Linkun <github@lkchen.net>
Co-authored-by: Ning Xie <andy.xning@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Andrew Sansom <andrew@protopia.ai>
Co-authored-by: Zhiyu <zhiyuc@nvidia.com>
Co-authored-by: Shu Wang <shuw@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
Co-authored-by: Junhao Li <streaver91@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Yuxuan Zhang <2448370773@qq.com>
Co-authored-by: ZiTian Zhao <zitian.zhao@tencentmusic.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Hong Hanh <hanh.usth@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: lkchen <github@lkchen.net>

08b751ba

[Bugfix] Fix ModernBert cuda graph capturing in v1 (#21901) · 429e4e2d
Isotr0py authored Aug 09, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
```
429e4e2d

[BugFix] [P/D] Handle lookahead token count edge-case with Eagle Spec Decoding and P/D (#22317) · 35afe1b3

Pradyun92 authored Aug 08, 2025


Signed-off-by: Pradyun Ramadorai <pradyunr@amazon.com>
Signed-off-by: Pradyun92 <142861237+Pradyun92@users.noreply.github.com>
Co-authored-by: Pradyun Ramadorai <pradyunr@amazon.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>

35afe1b3

[XPU] upgrade torch 2.8 on for XPU (#22300) · 81c57f60
Kunshang Ji authored Aug 09, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
81c57f60

08 Aug, 2025 22 commits
- Extract `CompilationConfig` from `config.py` (#22524) · e3edc0a7
  Harry Mellor authored Aug 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e3edc0a7
- [Frontend] Add unix domain socket support (#18097) · baece8c3
  yyweiss authored Aug 09, 2025
```
Signed-off-by: <yyweiss@gmail.com>
Signed-off-by: yyw <yyweiss@gmail.com>
```
  baece8c3
- [Misc] DeepGEMM : Avoid JIT generation in the hot-path (#22215) · f703b923
  Varun Sundar Rabindranath authored Aug 08, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  f703b923
- [BugFix] Fix IMA FlashMLA full cuda-graph and DP + Update FlashMLA (#21691) · cd9b9de1
  Lucas Wilkinson authored Aug 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  cd9b9de1
- [gpt-oss] Support tool call and implement MCP tool server (#22427) · fe6d8257
  Chen Zhang authored Aug 08, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  fe6d8257
- [gpt-oss] guard import when triton kernel is not installed (#22529) · f756a682
  Yongye Zhu authored Aug 08, 2025
```
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f756a682
- [gpt-oss] triton kernel mxfp4 (#22421) · e789cad6
  Yongye Zhu authored Aug 08, 2025
```
Signed-off-by: <zyy1102000@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
```
  e789cad6
- Remove exception for Python 3.8 typing from linter (#22506) · e5ebeeba
  Harry Mellor authored Aug 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e5ebeeba
- [BugFix] Don't cancel asyncio tasks directly from destructors (#22476) · ccdae737
  Nick Hill authored Aug 08, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ccdae737
- [Misc] Begin deprecation of `get_tensor_model_*_group` (#22494) · 43c4f3d7
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  43c4f3d7
- [CI/Build] Fix multimodal tests (#22491) · 1712543d
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1712543d
- [bench] Fix benchmark/serve.py to ignore unavailable results (#22382) · 808a7b69
  lkchen authored Aug 07, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  808a7b69
- [bugfix] Fix Llama3/4 issues caused by FlashInfer 0.2.10 (#22426) · af473f0a
  Po-Han Huang (NVIDIA) authored Aug 08, 2025
```
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
```
  af473f0a
- Fix pre-commit (#22487) · 157f9c13
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  157f9c13
- Optimize MiniCPMO mask creation with vectorized implementation (#22464) · 6f287915
  ZiTian Zhao authored Aug 08, 2025
```
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
```
  6f287915
- not tie_word_embeddings for glm-4.5 and glm-4.5v (#22460) · c152e2a8
  Yuxuan Zhang authored Aug 08, 2025
```
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
```
  c152e2a8
- [Bugfix] Fix RuntimeError: Index put requires the source and destination dtypes match (#22065) · 17eaaef5
  Chauncey authored Aug 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  17eaaef5
- Fix Flashinfer CUTLASS MOE Allgather (#21963) · b2c8ce57
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  b2c8ce57
- Support Tensorrt-LLM MoE fp4 for low-latency (#21331) · a3b9c17b
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
```
  a3b9c17b
- Add ModelOpt Qwen3 nvfp4 support (#20101) · d57dc236
  Zhiyu authored Aug 07, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  d57dc236
- [PERF] Use pybase64 to more quickly decode prompt embeddings (#22469) · e2c8f1ed
  Andrew Sansom authored Aug 07, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
```
  e2c8f1ed
- [ROCm] [V1] [SpecDec] Enable Speculative Decoding on ROCm V1 Engine (#21496) · 1ee5ead5
  TJian authored Aug 07, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  1ee5ead5
07 Aug, 2025 13 commits
- Remove `from_dict` from `SpeculativeConfig` (#22451) · 7e3a8dc9
  Harry Mellor authored Aug 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7e3a8dc9
- [Frontend] Use engine argument to control MM cache size (#22441) · 139d1557
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  139d1557
- [Core] Simplify mm processing cache (#22457) · 8c9da6be
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8c9da6be
- Fix pre-commit error in main (#22462) · 399d2a10
  Woosuk Kwon authored Aug 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  399d2a10
- [gpt-oss] Generate ResponseOutputItem from Harmony Message (#22410) · 4815b00f
  Chen Zhang authored Aug 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  4815b00f
- [Tool] Fix auto tool call (#22434) · 4da8bf20
  Chen Zhang authored Aug 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  4da8bf20
- [Bugfix] Add missing `packed_modules_mapping` to `DeepseekV2ForCausalLM` (#22352) · 7e0b1218
  fxmarty-amd authored Aug 07, 2025
```
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
```
  7e0b1218
- [Core] Store only the keys for multi-modal data in P0 (#22198) · 766bc816
  Cyrus Leung authored Aug 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  766bc816
- [Misc] Enhance code formatting in mxfp4.py (#22423) · 136825de
  Woosuk Kwon authored Aug 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  136825de
- Add H20-3e fused MoE kernel tuning configs for GLM-4.5 (#22433) · c2dba2db
  JaceyShao authored Aug 07, 2025
```
Signed-off-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
Co-authored-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
```
  c2dba2db
- feat: Add --enable-log-outputs flag for logging model generations (#20707) · 8e8e0b6a
  Adrián García García authored Aug 07, 2025
```
Signed-off-by: Adrian Garcia <adrian.garcia@inceptionai.ai>
```
  8e8e0b6a
- [Misc] Support routing logic simulation (#21990) · 82216dc2
  Ming Yang authored Aug 06, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  82216dc2
- [Frontend] Update OpenAI error response to upstream format (#22099) · 37066185
  Moritz Sanft authored Aug 07, 2025
```
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
```
  37066185