Commits · f756a682d96ba1824b6a759017f9d27a7f5f0182 · OpenDAS / vllm_cscc

"examples/vscode:/vscode.git/clone" did not exist on "da855b42d2005d9b701758e9d59836131ee8c6fb"

08 Aug, 2025 23 commits
- [gpt-oss] guard import when triton kernel is not installed (#22529) · f756a682
  Yongye Zhu authored Aug 08, 2025
```
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f756a682
- [Benchmark] Add benchmark tool for multi turn conversations (#20267) · f0964e29
  Daniel Serebrenik authored Aug 08, 2025
  
  f0964e29
- [gpt-oss] triton kernel mxfp4 (#22421) · e789cad6
  Yongye Zhu authored Aug 08, 2025
```
Signed-off-by: <zyy1102000@gmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
```
  e789cad6
- Remove exception for Python 3.8 typing from linter (#22506) · e5ebeeba
  Harry Mellor authored Aug 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e5ebeeba
- [Docs] Improve API docs (+small tweaks) (#22459) · 7be7f382
  Harry Mellor authored Aug 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7be7f382
- [BugFix] Don't cancel asyncio tasks directly from destructors (#22476) · ccdae737
  Nick Hill authored Aug 08, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ccdae737
- [Misc] fix openai version (#22485) · 90406390
  rongfu.leng authored Aug 08, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  90406390
- [Misc] Begin deprecation of `get_tensor_model_*_group` (#22494) · 43c4f3d7
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  43c4f3d7
- [CI/Build] Fix multimodal tests (#22491) · 1712543d
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1712543d
- [bench] Fix benchmark/serve.py to ignore unavailable results (#22382) · 808a7b69
  lkchen authored Aug 07, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  808a7b69
- [Doc] Sleep mode documentation (#22310) · 099c0464
  iAmir97 authored Aug 08, 2025
```
Signed-off-by: iAmir97 <Amir.balwel@embeddedllm.com>
Signed-off-by: iAmir97 <71513472+iAmir97@users.noreply.github.com>
Co-authored-by: iAmir97 <Amir.balwel@embeddedllm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Hong Hanh <hanh.usth@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  099c0464
- [bugfix] Fix Llama3/4 issues caused by FlashInfer 0.2.10 (#22426) · af473f0a
  Po-Han Huang (NVIDIA) authored Aug 08, 2025
```
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
```
  af473f0a
- Fix pre-commit (#22487) · 157f9c13
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  157f9c13
- Optimize MiniCPMO mask creation with vectorized implementation (#22464) · 6f287915
  ZiTian Zhao authored Aug 08, 2025
```
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
Signed-off-by: zitian zhao <zitian.zhao@tencentmusic.com>
```
  6f287915
- not tie_word_embeddings for glm-4.5 and glm-4.5v (#22460) · c152e2a8
  Yuxuan Zhang authored Aug 08, 2025
```
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
```
  c152e2a8
- [Bugfix] Fix RuntimeError: Index put requires the source and destination dtypes match (#22065) · 17eaaef5
  Chauncey authored Aug 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  17eaaef5
- [Kernel] Add support for block FP8 on SM120 (NVIDIA 5090 and RTX PRO 6000) (#22131) · 3303f134
  Junhao Li authored Aug 07, 2025
```
Signed-off-by: Junhao Li <junhao@ubicloud.com>
```
  3303f134
- Fix Flashinfer CUTLASS MOE Allgather (#21963) · b2c8ce57
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  b2c8ce57
- Support Tensorrt-LLM MoE fp4 for low-latency (#21331) · a3b9c17b
  Shu Wang authored Aug 07, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Po-Han Huang <pohanh@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: XIn Li <xinli@nvidia.com>
Co-authored-by: XIn Li <xinli@nvidia.com>
```
  a3b9c17b
- Add ModelOpt Qwen3 nvfp4 support (#20101) · d57dc236
  Zhiyu authored Aug 07, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
  d57dc236
- [PERF] Use pybase64 to more quickly decode prompt embeddings (#22469) · e2c8f1ed
  Andrew Sansom authored Aug 07, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
```
  e2c8f1ed
- [ROCm] [V1] [SpecDec] Enable Speculative Decoding on ROCm V1 Engine (#21496) · 1ee5ead5
  TJian authored Aug 07, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  1ee5ead5
- [Misc] normalize multiprocessing Queue usage (#22371) · acf8aeb7
  Ning Xie authored Aug 08, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  acf8aeb7
07 Aug, 2025 17 commits
- Remove `from_dict` from `SpeculativeConfig` (#22451) · 7e3a8dc9
  Harry Mellor authored Aug 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7e3a8dc9
- [Frontend] Use engine argument to control MM cache size (#22441) · 139d1557
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  139d1557
- [Core] Simplify mm processing cache (#22457) · 8c9da6be
  Cyrus Leung authored Aug 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8c9da6be
- Fix pre-commit error in main (#22462) · 399d2a10
  Woosuk Kwon authored Aug 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  399d2a10
- [gpt-oss] Generate ResponseOutputItem from Harmony Message (#22410) · 4815b00f
  Chen Zhang authored Aug 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  4815b00f
- [Tool] Fix auto tool call (#22434) · 4da8bf20
  Chen Zhang authored Aug 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  4da8bf20
- [Bugfix] Add missing `packed_modules_mapping` to `DeepseekV2ForCausalLM` (#22352) · 7e0b1218
  fxmarty-amd authored Aug 07, 2025
```
Signed-off-by: Felix Marty <Felix.Marty@amd.com>
```
  7e0b1218
- [Core] Store only the keys for multi-modal data in P0 (#22198) · 766bc816
  Cyrus Leung authored Aug 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  766bc816
- [Docs] Update features/disagg_prefill, add v1 examples and development (#22165) · 289b18e6
  WeiQing Chen authored Aug 07, 2025
```
Signed-off-by: David Chen <530634352@qq.com>
```
  289b18e6
- [Doc] update docs for nightly benchmarks (#12022) · 35171b11
  Andrew Chan authored Aug 07, 2025
```
Signed-off-by: Andrew Chan <andrewkchan.akc@gmail.com>
```
  35171b11
- [Docs] Factor out troubleshooting to its own guide; add section for Ray Observability (#21578) · a2c6696b
  Ricardo Decal authored Aug 07, 2025
```
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
```
  a2c6696b
- [Doc] Fix link to prefix caching design (#22384) · 5e839880
  Yong Hoon Shin authored Aug 07, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  5e839880
- [Misc] Enhance code formatting in mxfp4.py (#22423) · 136825de
  Woosuk Kwon authored Aug 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  136825de
- Add H20-3e fused MoE kernel tuning configs for GLM-4.5 (#22433) · c2dba2db
  JaceyShao authored Aug 07, 2025
```
Signed-off-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
Co-authored-by: shaojunqi <shaojunqi.sjq@alibaba-inc.com>
```
  c2dba2db
- [Docs] Add missing dependency for docs build (#22435) · 434d2f3f
  Harry Mellor authored Aug 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  434d2f3f
- feat: Add --enable-log-outputs flag for logging model generations (#20707) · 8e8e0b6a
  Adrián García García authored Aug 07, 2025
```
Signed-off-by: Adrian Garcia <adrian.garcia@inceptionai.ai>
```
  8e8e0b6a
- [Misc] Support routing logic simulation (#21990) · 82216dc2
  Ming Yang authored Aug 06, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  82216dc2