Commits · f703b923f3885157cf02b951c42f967c25329b01 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "5f1af97f86021cf2819e5ab2d84722dac53c2257"

08 Aug, 2025 20 commits
- [Misc] DeepGEMM : Avoid JIT generation in the hot-path (#22215) · f703b923
  Varun Sundar Rabindranath authored Aug 08, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  f703b923
- [BugFix] Fix IMA FlashMLA full cuda-graph and DP + Update FlashMLA (#21691) · cd9b9de1
  Lucas Wilkinson authored Aug 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  cd9b9de1
- [gpt-oss] Support tool call and implement MCP tool server (#22427) · fe6d8257
  Chen Zhang authored Aug 08, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  fe6d8257
- [gpt-oss] guard import when triton kernel is not installed (#22529) · f756a682
  Yongye Zhu authored Aug 08, 2025
```
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f756a682
- [gpt-oss] triton kernel mxfp4 (#22421) · e789cad6
  Yongye Zhu authored Aug 08, 2025
```
Signed-off-by: <zyy1102000@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
```
  e789cad6
- Remove exception for Python 3.8 typing from linter (#22506) · e5ebeeba
  Harry Mellor authored Aug 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e5ebeeba
- [BugFix] Don't cancel asyncio tasks directly from destructors (#22476) · ccdae737
  Nick Hill authored Aug 08, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ccdae737
- [Misc] Begin deprecation of `get_tensor_model_*_group` (#22494) · 43c4f3d7
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  43c4f3d7
- [CI/Build] Fix multimodal tests (#22491) · 1712543d
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1712543d
- [bench] Fix benchmark/serve.py to ignore unavailable results (#22382) · 808a7b69
  lkchen authored Aug 07, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  808a7b69
- [bugfix] Fix Llama3/4 issues caused by FlashInfer 0.2.10 (#22426) · af473f0a
  Po-Han Huang (NVIDIA) authored Aug 08, 2025
```
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
```
  af473f0a
- Fix pre-commit (#22487) · 157f9c13
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  157f9c13
- Optimize MiniCPMO mask creation with vectorized implementation (#22464) · 6f287915
  ZiTian Zhao authored Aug 08, 2025
```
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
```
  6f287915
- not tie_word_embeddings for glm-4.5 and glm-4.5v (#22460) · c152e2a8
  Yuxuan Zhang authored Aug 08, 2025
```
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
```
  c152e2a8
- [Bugfix] Fix RuntimeError: Index put requires the source and destination dtypes match (#22065) · 17eaaef5
  Chauncey authored Aug 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  17eaaef5
- Fix Flashinfer CUTLASS MOE Allgather (#21963) · b2c8ce57
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  b2c8ce57
- Support Tensorrt-LLM MoE fp4 for low-latency (#21331) · a3b9c17b
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
```
  a3b9c17b
- Add ModelOpt Qwen3 nvfp4 support (#20101) · d57dc236
  Zhiyu authored Aug 07, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  d57dc236
- [PERF] Use pybase64 to more quickly decode prompt embeddings (#22469) · e2c8f1ed
  Andrew Sansom authored Aug 07, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
```
  e2c8f1ed
- [ROCm] [V1] [SpecDec] Enable Speculative Decoding on ROCm V1 Engine (#21496) · 1ee5ead5
  TJian authored Aug 07, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  1ee5ead5
07 Aug, 2025 20 commits
- Remove `from_dict` from `SpeculativeConfig` (#22451) · 7e3a8dc9
  Harry Mellor authored Aug 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7e3a8dc9
- [Frontend] Use engine argument to control MM cache size (#22441) · 139d1557
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  139d1557
- [Core] Simplify mm processing cache (#22457) · 8c9da6be
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8c9da6be
- Fix pre-commit error in main (#22462) · 399d2a10
  Woosuk Kwon authored Aug 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  399d2a10
- [gpt-oss] Generate ResponseOutputItem from Harmony Message (#22410) · 4815b00f
  Chen Zhang authored Aug 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  4815b00f
- [Tool] Fix auto tool call (#22434) · 4da8bf20
  Chen Zhang authored Aug 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  4da8bf20
- [Bugfix] Add missing `packed_modules_mapping` to `DeepseekV2ForCausalLM` (#22352) · 7e0b1218
  fxmarty-amd authored Aug 07, 2025
```
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
```
  7e0b1218
- [Core] Store only the keys for multi-modal data in P0 (#22198) · 766bc816
  Cyrus Leung authored Aug 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  766bc816
- [Misc] Enhance code formatting in mxfp4.py (#22423) · 136825de
  Woosuk Kwon authored Aug 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  136825de
- Add H20-3e fused MoE kernel tuning configs for GLM-4.5 (#22433) · c2dba2db
  JaceyShao authored Aug 07, 2025
```
Signed-off-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
Co-authored-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
```
  c2dba2db
- feat: Add --enable-log-outputs flag for logging model generations (#20707) · 8e8e0b6a
  Adrián García García authored Aug 07, 2025
```
Signed-off-by: Adrian Garcia <adrian.garcia@inceptionai.ai>
```
  8e8e0b6a
- [Misc] Support routing logic simulation (#21990) · 82216dc2
  Ming Yang authored Aug 06, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  82216dc2
- [Frontend] Update OpenAI error response to upstream format (#22099) · 37066185
  Moritz Sanft authored Aug 07, 2025
```
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
```
  37066185
- [Model] Switch to Fused RMS norm in Qwen2.5_VL model. (#22184) · cbc8457b
  vllmellm authored Aug 07, 2025
```
Signed-off-by: kf <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaavllm <tunjian.tan@amd.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: kf <kuanfu.liu@embeddedllm.com>
```
  cbc8457b
- [Bench] Split serve.py:main into async/async versions (#22405) · 4d4297e8
  lkchen authored Aug 06, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  4d4297e8
- [Bugfix] EPLB load statistics problem (#22167) · 4be02a37
  WeiQing Chen authored Aug 07, 2025
```
Signed-off-by: ycyaw66 <497410282@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>
```
  4be02a37
- [gpt-oss] Convert user input to harmony format (#22402) · f6278b62
  Chen Zhang authored Aug 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f6278b62
- preload heavy modules when mp method is forkserver (#22214) · ad6c655d
  Lionel Villard authored Aug 06, 2025
```
Signed-off-by: Lionel Villard <villard@us.ibm.com>
```
  ad6c655d
- Optimize logger init performance by using module-level constants (#22373) · 14bcf93a
  ZiTian.Zhao authored Aug 07, 2025
```
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
```
  14bcf93a
- [Bugfix] Add proper comparison for package versions (#22314) · 609b533c
  Syed Muhammad Bin Asif authored Aug 07, 2025
```
Signed-off-by: Syed Muhammad Bin Asif <syedmba7@connect.hku.hk>
```
  609b533c