Commits · 43c4f3d77c3c03f67385201e1b1725a6ba6bcc7a · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "e31ae3de366dcf1898069dd57eebcb2983b5ab1d"

08 Aug, 2025 16 commits
- [Misc] Begin deprecation of `get_tensor_model_*_group` (#22494) · 43c4f3d7
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  43c4f3d7
- [CI/Build] Fix multimodal tests (#22491) · 1712543d
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1712543d
- [bench] Fix benchmark/serve.py to ignore unavailable results (#22382) · 808a7b69
  lkchen authored Aug 07, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  808a7b69
- [Doc] Sleep mode documentation (#22310) · 099c0464
  iAmir97 authored Aug 08, 2025
```
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Hong Hanh <hanh.usth@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  099c0464
- [bugfix] Fix Llama3/4 issues caused by FlashInfer 0.2.10 (#22426) · af473f0a
  Po-Han Huang (NVIDIA) authored Aug 08, 2025
```
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
```
  af473f0a
- Fix pre-commit (#22487) · 157f9c13
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  157f9c13
- Optimize MiniCPMO mask creation with vectorized implementation (#22464) · 6f287915
  ZiTian Zhao authored Aug 08, 2025
```
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
```
  6f287915
- not tie_word_embeddings for glm-4.5 and glm-4.5v (#22460) · c152e2a8
  Yuxuan Zhang authored Aug 08, 2025
```
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
```
  c152e2a8
- [Bugfix] Fix RuntimeError: Index put requires the source and destination dtypes match (#22065) · 17eaaef5
  Chauncey authored Aug 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  17eaaef5
- [Kernel] Add support for block FP8 on SM120 (NVIDIA 5090 and RTX PRO 6000) (#22131) · 3303f134
  Junhao Li authored Aug 07, 2025
```
Signed-off-by: Junhao Li <junhao@ubicloud.com>
```
  3303f134
- Fix Flashinfer CUTLASS MOE Allgather (#21963) · b2c8ce57
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  b2c8ce57
- Support Tensorrt-LLM MoE fp4 for low-latency (#21331) · a3b9c17b
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
```
  a3b9c17b
- Add ModelOpt Qwen3 nvfp4 support (#20101) · d57dc236
  Zhiyu authored Aug 07, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  d57dc236
- [PERF] Use pybase64 to more quickly decode prompt embeddings (#22469) · e2c8f1ed
  Andrew Sansom authored Aug 07, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
```
  e2c8f1ed
- [ROCm] [V1] [SpecDec] Enable Speculative Decoding on ROCm V1 Engine (#21496) · 1ee5ead5
  TJian authored Aug 07, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  1ee5ead5
- [Misc] normalize multiprocessing Queue usage (#22371) · acf8aeb7
  Ning Xie authored Aug 08, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  acf8aeb7
07 Aug, 2025 24 commits
- Remove `from_dict` from `SpeculativeConfig` (#22451) · 7e3a8dc9
  Harry Mellor authored Aug 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7e3a8dc9
- [Frontend] Use engine argument to control MM cache size (#22441) · 139d1557
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  139d1557
- [Core] Simplify mm processing cache (#22457) · 8c9da6be
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8c9da6be
- Fix pre-commit error in main (#22462) · 399d2a10
  Woosuk Kwon authored Aug 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  399d2a10
- [gpt-oss] Generate ResponseOutputItem from Harmony Message (#22410) · 4815b00f
  Chen Zhang authored Aug 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  4815b00f
- [Tool] Fix auto tool call (#22434) · 4da8bf20
  Chen Zhang authored Aug 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  4da8bf20
- [Bugfix] Add missing `packed_modules_mapping` to `DeepseekV2ForCausalLM` (#22352) · 7e0b1218
  fxmarty-amd authored Aug 07, 2025
```
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
```
  7e0b1218
- [Core] Store only the keys for multi-modal data in P0 (#22198) · 766bc816
  Cyrus Leung authored Aug 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  766bc816
- [Docs] Update features/disagg_prefill, add v1 examples and development (#22165) · 289b18e6
  WeiQing Chen authored Aug 07, 2025
```
Signed-off-by: David Chen <530634352@qq.com>
```
  289b18e6
- [Doc] update docs for nightly benchmarks (#12022) · 35171b11
  Andrew Chan authored Aug 07, 2025
```
Signed-off-by: Andrew Chan <andrewkchan.akc@gmail.com>
```
  35171b11
- [Docs] Factor out troubleshooting to its own guide; add section for Ray Observability (#21578) · a2c6696b
  Ricardo Decal authored Aug 07, 2025
```
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
```
  a2c6696b
- [Doc] Fix link to prefix caching design (#22384) · 5e839880
  Yong Hoon Shin authored Aug 07, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  5e839880
- [Misc] Enhance code formatting in mxfp4.py (#22423) · 136825de
  Woosuk Kwon authored Aug 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  136825de
- Add H20-3e fused MoE kernel tuning configs for GLM-4.5 (#22433) · c2dba2db
  JaceyShao authored Aug 07, 2025
```
Signed-off-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
Co-authored-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
```
  c2dba2db
- [Docs] Add missing dependency for docs build (#22435) · 434d2f3f
  Harry Mellor authored Aug 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  434d2f3f
- feat: Add --enable-log-outputs flag for logging model generations (#20707) · 8e8e0b6a
  Adrián García García authored Aug 07, 2025
```
Signed-off-by: Adrian Garcia <adrian.garcia@inceptionai.ai>
```
  8e8e0b6a
- [Misc] Support routing logic simulation (#21990) · 82216dc2
  Ming Yang authored Aug 06, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  82216dc2
- [Frontend] Update OpenAI error response to upstream format (#22099) · 37066185
  Moritz Sanft authored Aug 07, 2025
```
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
```
  37066185
- [Model] Switch to Fused RMS norm in Qwen2.5_VL model. (#22184) · cbc8457b
  vllmellm authored Aug 07, 2025
```
Signed-off-by: kf <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaavllm <tunjian.tan@amd.com>
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: kf <kuanfu.liu@embeddedllm.com>
```
  cbc8457b
- [Bench] Split serve.py:main into async/async versions (#22405) · 4d4297e8
  lkchen authored Aug 06, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  4d4297e8
- [CI] Skip the pooling models that do not support transformers v4.55 (#22411) · 2a4c8255
  wang.yuqi authored Aug 07, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  2a4c8255
- [Bugfix] EPLB load statistics problem (#22167) · 4be02a37
  WeiQing Chen authored Aug 07, 2025
```
Signed-off-by: ycyaw66 <497410282@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>
```
  4be02a37
- [gpt-oss] Convert user input to harmony format (#22402) · f6278b62
  Chen Zhang authored Aug 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f6278b62
- preload heavy modules when mp method is forkserver (#22214) · ad6c655d
  Lionel Villard authored Aug 06, 2025
```
Signed-off-by: Lionel Villard <villard@us.ibm.com>
```
  ad6c655d