Commits · a388252ac49ec642bec5a8a63b226d3058e64a54 · OpenDAS / vllm_cscc

30 Sep, 2025 31 commits

Add explicit pooling classes for the Transformers backend (#25322) · a388252a

Harry Mellor authored Sep 30, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

a388252a

[V1] [P/D] Add Support for KV Load Failure Recovery (#19330) · 9a9f48df
David Ben-David authored Oct 01, 2025
```
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
```
9a9f48df
[Bench] Add DeepSeekV32 to MoE benchmark (#25962) · 67f3fb08
Jee Jee Li authored Oct 01, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
67f3fb08

[Llama4] [multimodal] Fix misplaced dtype cast of `cos_sin_cache` in... · 43b752c3

cjackal authored Oct 01, 2025


[Llama4] [multimodal] Fix misplaced dtype cast of `cos_sin_cache` in `Llama4VisionRotaryEmbedding` (#25889)
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>

43b752c3

OffloadingConnector: Fix GPU block tracking bug (#25856) · cfd302db
Or Ozeri authored Sep 30, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
cfd302db

[Docs] Add moe kernel features doc (#25297) · fb610ae6

bnellnm authored Sep 30, 2025


Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

fb610ae6

[Doc] Improve MM Pooling model documentation (#25966) · 2f652e6c
Cyrus Leung authored Oct 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
2f652e6c
[Bug] Fix AttributeError: 'QKVParallelLinear' object has no attribute 'orig_dtype' (#25958) · e6a226ef
Wentao Ye authored Sep 30, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
e6a226ef
[bugfix][deepseek] fix flashmla kernel selection (#25956) · a2e6fa7e
youkaichao authored Oct 01, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
a2e6fa7e
[Bugfix] Token type and position embeddings fail to be applied to `inputs_embeds` (#25922) · 9f1c4eca
Cyrus Leung authored Oct 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
9f1c4eca
[Bugfix] Fix accuracy issue of TRTLLM FP8 MOE and improve logging (#25895) · ef283548
Pavani Majety authored Sep 30, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
ef283548

[Bugfix][Model] Fix inference for Hunyuan dense models (#25354) · f4db5e6d

Anion authored Sep 30, 2025


Signed-off-by: anion <1005128408@qq.com>
Signed-off-by: Anion <123177548+Anionex@users.noreply.github.com>

f4db5e6d

Add Hugging Face Inference Endpoints guide to Deployment docs (#25886) · 099aaee5

Sergio Paniego Blanco authored Sep 30, 2025

Signed-off-by: sergiopaniego <sergiopaniegoblanco@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

099aaee5

[Kernel][Moe Configs] Add more tuned triton configs for ExpertsInt8 and FP8 (#25858) · 35fe398c
Asaf Joseph Gardin authored Sep 30, 2025
```
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
```
35fe398c
[Fix] Improve CPU backend compatibility for RISC-V (#25816) · bb6d4304
ihb2032 authored Sep 30, 2025
```
Signed-off-by: lyd1992 <liuyudong@iscas.ac.cn>
Signed-off-by: ihb2032 <1355790728@qq.com>
```
bb6d4304

[CI] Move applicable tests to CPU (#24080) · bc546f76

Reza Barazesh authored Sep 30, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

bc546f76

[NIXL] Add support for MLA caches with different latent dim (#25902) · 80608ba5

Nicolò Lucchesi authored Sep 30, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>

80608ba5

[perf] Use CPU tensor to reduce GPU->CPU sync (#25884) · e184c9c5
Lehua Ding authored Sep 30, 2025
```
Signed-off-by: Lehua Ding <lehuading@tencent.com>
```
e184c9c5
[Model] Move `vision_feature_select_strategy` into `resolve_visual_encoder_outputs` (#25938) · d7e34b42
Cyrus Leung authored Sep 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
d7e34b42
[Bugfix][Model]fix ernie45 moe gate&bias dtype to float32 (#25936) · ef6e0e71
CSWYF3634076 authored Sep 30, 2025
```
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
```
ef6e0e71

Updated TRL integration docs (#25684) · 1ad3aca6

Sergio Paniego Blanco authored Sep 30, 2025

Signed-off-by: sergiopaniego <sergiopaniegoblanco@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Sergio Paniego Blanco <sergiopaniegoblanco@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

1ad3aca6

[Doc] Add Cambricon MLU support (#25942) · 8d0afa9b
a120092009 authored Sep 30, 2025
```
Signed-off-by: a120092009 <zhaoty0121@gmail.com>
```
8d0afa9b

[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) · fa7e254a

Yongye Zhu authored Sep 30, 2025


Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>