Commits · 31f5dc5b2a5da18bc17240c7a67e8770d00901d8 · OpenDAS / vllm_cscc

06 Aug, 2025 26 commits

[gpt-oss] Enhance error msg on attention sink init (#22335) · 31f5dc5b

Yongye Zhu authored Aug 06, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

31f5dc5b

[gpt-oss] Add loop for built-in tool call (#22374) · ec7cb192

Woosuk Kwon authored Aug 06, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

ec7cb192

[Bugfix] Make condition in triton kernel constexpr (#22370) · 2435ea7e
Gregory Shtrasberg authored Aug 06, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
2435ea7e

[BugFix] Fix triton compile error in `kernel_unified_attention_2/3d` caused by... · 4a6b72c2

Lucas Wilkinson authored Aug 06, 2025


[BugFix] Fix triton compile error in `kernel_unified_attention_2/3d` caused by attention sinks (#22368)
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>

4a6b72c2

add the codes to check AMD Instinct GPU number (#22367) · b4b9813b
Zhang Jason authored Aug 06, 2025
```
Signed-off-by: Zhang Jason <ning.zhang2@amd.com>
```
b4b9813b
[BugFix] Fix FA2 RuntimeError when sinks is provided (#22365) · 2cb6ef89
Lucas Wilkinson authored Aug 06, 2025
```
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
```
2cb6ef89
[Minor] Fix type (#22347) · 9edd1db0
Woosuk Kwon authored Aug 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
9edd1db0
[gpt-oss] Support chat completion api (#22342) · f263a4b5
Woosuk Kwon authored Aug 06, 2025

f263a4b5
[gpt-oss] add model to supported models doc (#22336) · 54991c54
Roger Wang authored Aug 06, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
```
54991c54

[gpt-oss] Add Tool/ConversationContext classes and harmony_utils (#22340) · 178d03fb

Woosuk Kwon authored Aug 06, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

178d03fb

[Misc] Clean up duplicated hf overrides (#22311) · fa00c5d7
Isotr0py authored Aug 06, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
fa00c5d7

[gpt-oss] Add openai-harmony as default dependency (#22332) · 134a8ee8

Woosuk Kwon authored Aug 06, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

134a8ee8

[gpt-oss] flashinfer attention sink init (#22330) · 90ec0069

Yongye Zhu authored Aug 05, 2025


Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>

90ec0069

[GptOss] Add GptOss reasoning parser to support structure output (#22322) · a47e6ffe

Chen Zhang authored Aug 05, 2025


Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

a47e6ffe

[ROCm] Add attention sink to use_rocm_custom_paged_attention (#22329) · 98a3a810

Woosuk Kwon authored Aug 05, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

98a3a810

Add GPT-OSS model code and config [1/N] (#22327) · de98252f
Woosuk Kwon authored Aug 05, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
de98252f

Update transformers to `v4.55` (#21931) · 796bae07

Harry Mellor authored Aug 06, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

796bae07

Add attention sink in attention backends (#22320) · 6e209243

Woosuk Kwon authored Aug 05, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
Co-authored-by: Minseok Lee <47620120+minseokl@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>

6e209243

Increase openai-python version (#22316) · dd16bdc7
Woosuk Kwon authored Aug 05, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
dd16bdc7
Upgrade FA3 for attention sink (#22313) · e3c876dc
Woosuk Kwon authored Aug 05, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
e3c876dc
[Bugfix][CI/Build][ROCm] Make sure to use the headers from the build folder on ROCm (#22264) · 5d5d419c
Gregory Shtrasberg authored Aug 05, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
5d5d419c
[Bugfix] Skip dead and non-GPU nodes for Ray DP engine allocation (#22275) · 302962e8
Rui Qiao authored Aug 05, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
302962e8
[Perf] Parallelize fill_bitmask to accelerate high-throughput guided decoding (#21862) · 7e6544c7
Benjamin Chislett authored Aug 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
7e6544c7
[Bugfix] Fix MoE BNB version (#22260) · 8e6c7e87
Jee Jee Li authored Aug 06, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
8e6c7e87
[Bugfix] Fix 3D input passed into cutlass_scaled_mm (#22278) · 6a515304
Michael Goin authored Aug 05, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
6a515304
[Bugfix] Remove faulty test for oot attention backend (#22286) · 35509fc5
Michael Goin authored Aug 05, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
35509fc5

05 Aug, 2025 14 commits
- [CI][TPU] Fix docker clean up (#22271) · 4b29d278
  Siyuan Liu authored Aug 05, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  4b29d278
- [bugfix] fix blackwell deepep installation (#22255) · 59a0b855
  youkaichao authored Aug 06, 2025
  
  59a0b855
- [V1] port xformers backend to v1 (#21342) · 469b3ffa
  Giancarlo Delfin authored Aug 05, 2025
```
Signed-off-by: Giancarlo Delfin <gdelfin@meta.com>
```
  469b3ffa
- [Refactor] Remove Unused Environment Variable `VLLM_NO_DEPRECATION_WARNING` (#22199) · ae87ddd0
  Wentao Ye authored Aug 05, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  ae87ddd0
- [CI/Build] Update flashinfer to 0.2.9 (#22233) · a7cb6101
  Michael Goin authored Aug 05, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a7cb6101
- Use UV_LINK_MODE=copy in Dockerfile to avoid hardlink fail (#22128) · c494f96f
  Michael Goin authored Aug 05, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  c494f96f
- [V0 Deprecation][TPU] Remove V1 flag check from tests (#22248) · 0c275ad5
  Nicolò Lucchesi authored Aug 05, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  0c275ad5
- [Misc] correct static type check for GroupCoordinator (#21946) · 74333ae2
  Ning Xie authored Aug 05, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  74333ae2
- [NVIDIA] Support Flashinfer TRT-LLM Prefill Attention Kernel (#22095) · 83156c7b
  elvischenv authored Aug 05, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  83156c7b
- [Feature] Non-contiguous Support for FP8 Quantization (#21961) · 4771df7b
  Wentao Ye authored Aug 05, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  4771df7b
- Migrate KimiVLImagePixelInputs to TensorSchema (#21769) · 05fae021
  Benji Beck authored Aug 05, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  05fae021
- [Docs][TPU] Highlight TPU Software version selection (#22242) · d1bf1b97
  Nicolò Lucchesi authored Aug 05, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d1bf1b97
- [Model] Pooling model activation supports per request control by PoolingParams (#20538) · 586f2867
  wang.yuqi authored Aug 05, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  586f2867
- [Core] Factor out common logic for MM budget calculation (#22228) · 811ac13d
  Cyrus Leung authored Aug 05, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  811ac13d