Commits · 2a4c825523d5715068bf3ec373f662e113c66f45 · OpenDAS / vllm_cscc

07 Aug, 2025 21 commits
- [CI] Skip the pooling models that do not support transformers v4.55 (#22411) · 2a4c8255
  wang.yuqi authored Aug 07, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  2a4c8255
- [Bugfix] EPLB load statistics problem (#22167) · 4be02a37
  WeiQing Chen authored Aug 07, 2025
```
Signed-off-by: ycyaw66 <497410282@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>
```
  4be02a37
- [gpt-oss] Convert user input to harmony format (#22402) · f6278b62
  Chen Zhang authored Aug 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f6278b62
- preload heavy modules when mp method is forkserver (#22214) · ad6c655d
  Lionel Villard authored Aug 06, 2025
```
Signed-off-by: Lionel Villard <villard@us.ibm.com>
```
  ad6c655d
- Optimize logger init performance by using module-level constants (#22373) · 14bcf93a
  ZiTian.Zhao authored Aug 07, 2025
```
Signed-off-by: zitian.zhao <zitian.zhao@tencentmusic.com>
```
  14bcf93a
- Update `hf_xet` pin to resolve hangs (#22356) · ecbea55c
  Harry Mellor authored Aug 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  ecbea55c
- [Bugfix] Add proper comparison for package versions (#22314) · 609b533c
  Syed Muhammad Bin Asif authored Aug 07, 2025
```
Signed-off-by: Syed Muhammad Bin Asif <syedmba7@connect.hku.hk>
```
  609b533c
- [Bugfix]: Fix the streaming output for function calls in the minimax (#22015) · 5e9455ae
  qscqesze authored Aug 07, 2025
```
Signed-off-by: QscQ <qscqesze@gmail.com>
Signed-off-by: qingjun <qingjun@minimaxi.com>
```
  5e9455ae
- Use float32 for test_completion.py (#22385) · a00d8b23
  Michael Goin authored Aug 06, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  a00d8b23
- [Bugfix] Fix wrong method name in Intern-S1 image processor (#22417) · 04cf435d
  Cyrus Leung authored Aug 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  04cf435d
- [Qwen3] Enable dual-chunk-attention support for Qwen3 models. (#21924) · 7377131a
  Tao He authored Aug 07, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
```
  7377131a
- [XPU]Fix `flash_attn_varlen_func` interface on xpu (#22350) · 6b47ef24
  Kunshang Ji authored Aug 07, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  6b47ef24
- [Attention] Support multiple attention metadata builders per kv_cache_spec +... · 1dc8a70b
  Lucas Wilkinson authored Aug 06, 2025
```
[Attention] Support multiple attention metadata builders per kv_cache_spec  + proper local attention no hybrid kv cache fix (#21588)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  1dc8a70b
- Support encoder_only attention for FlexAttention (#22273) · f825c6bd
  Maximilien de Bayser authored Aug 06, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  f825c6bd
- [model] Support MiniCPM-V 4.0 (#22166) · 41b67f42
  tc-mb authored Aug 07, 2025
```
Co-authored-by: imning3 <hbning@pku.edu.cn>
```
  41b67f42
- Update `flashinfer-python==0.2.10` (#22389) · e8961e96
  Michael Goin authored Aug 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  e8961e96
- Fix trtllm-gen attention env and add attention sink (#22378) · 9a3835aa
  Lain authored Aug 06, 2025
```
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: Lain <fusiyuan2000@hotmail.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
```
  9a3835aa
- [gpt-oss] fix model config with hf_config (#22401) · 5c7cc33f
  Yongye Zhu authored Aug 06, 2025
```
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
```
  5c7cc33f
- [gpt-oss] add demo tool server (#22393) · 19c9365a
  Chen Zhang authored Aug 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  19c9365a
- [Bug] Fix B200 DeepGEMM E8M0 Accuracy Issue (#22399) · eec890c1
  Wentao Ye authored Aug 06, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  eec890c1
- [v1] - Mamba1 Attention Metadata (#21249) · 46a13949
  Asaf Joseph Gardin authored Aug 07, 2025
```
Signed-off-by: asafg <asafg@ai21.com>
Co-authored-by: asafg <asafg@ai21.com>
```
  46a13949
06 Aug, 2025 19 commits

[gpt-oss] flashinfer mxfp4 (#22339) · 31f09c61

Yongye Zhu authored Aug 06, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

31f09c61

[gpt-oss] Enhance error msg on attention sink init (#22335) · 31f5dc5b

Yongye Zhu authored Aug 06, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

31f5dc5b

[gpt-oss] Add loop for built-in tool call (#22374) · ec7cb192

Woosuk Kwon authored Aug 06, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

ec7cb192

[Bugfix] Make condition in triton kernel constexpr (#22370) · 2435ea7e
Gregory Shtrasberg authored Aug 06, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
2435ea7e

[BugFix] Fix triton compile error in `kernel_unified_attention_2/3d` caused by... · 4a6b72c2

Lucas Wilkinson authored Aug 06, 2025


[BugFix] Fix triton compile error in `kernel_unified_attention_2/3d` caused by attention sinks (#22368)
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>

4a6b72c2

add the codes to check AMD Instinct GPU number (#22367) · b4b9813b
Zhang Jason authored Aug 06, 2025
```
Signed-off-by: Zhang Jason <ning.zhang2@amd.com>
```
b4b9813b
[BugFix] Fix FA2 RuntimeError when sinks is provided (#22365) · 2cb6ef89
Lucas Wilkinson authored Aug 06, 2025
```
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
```
2cb6ef89
[Minor] Fix type (#22347) · 9edd1db0
Woosuk Kwon authored Aug 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
9edd1db0
[gpt-oss] Support chat completion api (#22342) · f263a4b5
Woosuk Kwon authored Aug 06, 2025

f263a4b5
[gpt-oss] add model to supported models doc (#22336) · 54991c54
Roger Wang authored Aug 06, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
```
54991c54

[gpt-oss] Add Tool/ConversationContext classes and harmony_utils (#22340) · 178d03fb

Woosuk Kwon authored Aug 06, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

178d03fb

[Misc] Clean up duplicated hf overrides (#22311) · fa00c5d7
Isotr0py authored Aug 06, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
fa00c5d7

[gpt-oss] Add openai-harmony as default dependency (#22332) · 134a8ee8

Woosuk Kwon authored Aug 06, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

134a8ee8

[gpt-oss] flashinfer attention sink init (#22330) · 90ec0069

Yongye Zhu authored Aug 05, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>

90ec0069

[GptOss] Add GptOss reasoning parser to support structure output (#22322) · a47e6ffe

Chen Zhang authored Aug 05, 2025


Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

a47e6ffe

[ROCm] Add attention sink to use_rocm_custom_paged_attention (#22329) · 98a3a810

Woosuk Kwon authored Aug 05, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

98a3a810

Add GPT-OSS model code and config [1/N] (#22327) · de98252f
Woosuk Kwon authored Aug 05, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
de98252f

Update transformers to `v4.55` (#21931) · 796bae07

Harry Mellor authored Aug 06, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

796bae07

Add attention sink in attention backends (#22320) · 6e209243

Woosuk Kwon authored Aug 05, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

6e209243