Commits · 46ec6d71c71c50d088a03e0b04e6f49cf8637c00 · OpenDAS / vllm_cscc

"tests/models/language/generation/test_models.py" did not exist on "64172a976c8d975b3aec946f1675716d2532d94f"

28 Jan, 2026 1 commit

[Model Runner V2] Use a different stream for grammar bitmask h2d copy (#33059) · 46ec6d71

Woosuk Kwon authored Jan 27, 2026


Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Nick Hill <nhill@redhat.com>

46ec6d71

27 Jan, 2026 28 commits

[torch.compile] Speed up MOE handling in forward_context (#33184) · d9aa39a3
Richard Zou authored Jan 27, 2026
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
d9aa39a3
[Perf] Optimize dcp allocate tensor (#33102) · 3a6d5cbe
Wentao Ye authored Jan 27, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
3a6d5cbe
[Bugfix] Fix display error (inconsistent with context) (#33020) · f5d7049c
linhaifeng authored Jan 28, 2026
```
Signed-off-by: linhaifeng <1371675203@qq.com>
```
f5d7049c
[Attention] Use `has_flashinfer` helper (#33177) · 1cbccb6d
Matthew Bonanni authored Jan 27, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
1cbccb6d

feature: support eagle3 for HunyuanVL & Hunyuan (#33035) · bd92089d

Iris authored Jan 28, 2026


Signed-off-by: irisliu10 <601012173@qq.com>
Signed-off-by: Iris <38269816+irisliu10@users.noreply.github.com>

bd92089d

Support compress-tensors with nvfp4 or fp8 weights and modelopt with nvfp4... · 66e601ef

IriKa authored Jan 28, 2026


Support compress-tensors with nvfp4 or fp8 weights and modelopt with nvfp4 weights on Turing (#33076)
Signed-off-by: IriKa Qiu <qiujie.jq@gmail.com>

66e601ef

[BugFix] Fix P/D with non-MoE DP (#33037) · 0cd259b2
Nick Hill authored Jan 27, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
0cd259b2

Support heterogeneous NemotronHPuzzle model (#32549) · 83fb2d09

danielafrimi authored Jan 27, 2026



Signed-off-by: <dafrimi@nvidia.com>
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com>
Signed-off-by: root <dafrimi@nvidia.com>

83fb2d09

[LoRA][Spec Decode] Support LoRA for Nemotron-H MTP models (#32265) · f3a5ee70

danisereb authored Jan 27, 2026


Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

f3a5ee70

[Frontend] Cleanup api server (#33158) · 7cbbca9a

wang.yuqi authored Jan 27, 2026


Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>

7cbbca9a

[Metrics][MFU] Fix UnembedMetrics FLOP overcounting for prefill (#33045) (#33045) · 5ec44056

omkhalil authored Jan 27, 2026

Fix UnembedMetrics to correctly count FLOPs for the unembedding (LM head) layer.

The bug: UnembedMetrics used total_num_tokens() which counts all tokens in the
batch for projection flops, vocab projections are run on just the last token for the
autoregressive use case.
Co-authored-by: Omar Mohamed Khalil <omarkhalil@meta.com>

5ec44056

[Bugfix] Fix DeepseekV32 `AssertionError: num_kv_heads == 1` (#33090) · 492a7983
Nicolò Lucchesi authored Jan 27, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
492a7983
[5/N][Attention] Finish eliminating `vllm/attention` folder (#32064) · a608b4c6
Matthew Bonanni authored Jan 27, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
a608b4c6
[Bugfix] Disable CG for Whisper+FA2 (#33164) · 1f3a2c29
Nicolò Lucchesi authored Jan 27, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
1f3a2c29

[Metrics] [KVConnector] Add Offloading Connector metrics (#27942) · 7227d061

omerpaz95 authored Jan 27, 2026



Added queries and hits metrics for the Offloading Connector.

Also added timing metrics for store and load operations, which take the
average time it takes to load/store, per-token.

The metrics are available from Prometheus and from the StatLogger.
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Omer Paz <Omer.Paz@ibm.com>

7227d061

Fix weight mapping test for Transfomers v5 (#33162) · 14385c80
Harry Mellor authored Jan 27, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
14385c80

[Frontend] Frontend will only attach supported tasks corresponding entrypoints. (#33139) · 76139d08

wang.yuqi authored Jan 27, 2026


Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

76139d08

[AMD][QWEN3-NEXT] FP8 Tunings (#32042) · da8d0c44
Lifan Shen authored Jan 27, 2026
```
Signed-off-by: Lifan Shen <lifans@meta.com>
```
da8d0c44

[Models] Kimi-K2.5 (#33131) · b539f988

Roger Wang authored Jan 26, 2026


Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: wanglinian <wanglinian@stu.pku.edu.cn>
Co-authored-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

b539f988

[code clean] remove duplicate code (#33135) · b781eeaa
Ning Xie authored Jan 27, 2026
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
b781eeaa
[Frontend] Cleanup serving engine (#33103) · e0b005d9
Cyrus Leung authored Jan 27, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
e0b005d9
[torch.compile] Stop assuming 32 bit indexing (#33113) · 3b8f0fe5
Richard Zou authored Jan 26, 2026
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
3b8f0fe5
[Frontend] Reduce mixin usage in serving pooling (#33101) · c831911b
Cyrus Leung authored Jan 27, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
c831911b
[Perf] avoid duplicate mem_get_info() call in get_current_memory_usage (#33064) · 157caf51
Paco Xu authored Jan 27, 2026
```
Signed-off-by: Paco Xu <paco.xu@daocloud.io>
```
157caf51
Fix IndexError with encoder-decoder models when using Custom Paged Attention (#33112) · c568581f
Strahinja Stamenkovic authored Jan 27, 2026
```
Signed-off-by: sstamenk <strahinja.stamenkovic@amd.com>
```
c568581f

fix: preserve native tool call ID in multi-turn tool calling (#32768) · 2d705343

wangln19 authored Jan 27, 2026


Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <2037008807@qq.com>

2d705343

[MoE Refactor] Integrate Naive Prepare Finalize into MK (#32567) · 5a93b916

Robert Shaw authored Jan 26, 2026


Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: amirkl94 <203507526+amirkl94@users.noreply.github.com>

5a93b916

[Model Runner V2] Remove UvaBufferPool for cpu->gpu copy (#33055) · 6d86fde0

Woosuk Kwon authored Jan 26, 2026


Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Nick Hill <nhill@redhat.com>

6d86fde0

26 Jan, 2026 11 commits

[Bugfix][TPU] Return a Default fp8 MoE Backend (#32908) · 510ed1e8

XiongfeiWei authored Jan 26, 2026


Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

510ed1e8

[Bugfix][MXFP4] Call `trtllm_fp4_block_scale_moe` with kwargs (#33104) · 8caffd92
Pengchao Wang authored Jan 26, 2026
```
Signed-off-by: Pengchao Wang <wpc@fb.com>
```
8caffd92
[fix] CPUDNNLGEMMHandler pointer baked into inductor artifact (#32913) · 58a05b0c
dolpm authored Jan 26, 2026
```
Signed-off-by: dolpm <34420038+dolpm@users.noreply.github.com>
```
58a05b0c

[Logging] add `--disable-access-log-for-endpoints` CLI option (#30011) · 6ee7f18f

Jared Wen authored Jan 27, 2026



Add a new CLI option --disable-access-log-for-endpoints to suppress
uvicorn access logs for specified endpoints (e.g., /health, /metrics, /ping).

This addresses the need to reduce log noise in production environments
where health check endpoints are frequently polled by load balancers or
monitoring systems, generating excessive log entries that obscure
meaningful request logs.

Fixes #29982
Signed-off-by: JaredforReal <w13431838023@gmail.com>

6ee7f18f

[Refactor] Remove unused `_moe_permute` function (#33108) · 8f987883
Wentao Ye authored Jan 26, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
8f987883

[Bugfix] Fix Dtypes for Pynccl Wrapper (#33030) · 43a013c3

Robert Shaw authored Jan 26, 2026


Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>

43a013c3

[Model] Bump transformers version for test registry (#33100) · c25dbee4
Cyrus Leung authored Jan 27, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
c25dbee4
[Bugfix] Fix Voxtral streaming slot_mapping (#33073) · 19ab0f7c
Nicolò Lucchesi authored Jan 26, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
19ab0f7c

[FIX] Always support TP > 4 for FP4 Gemm (#31099) · 67fe677c

danielafrimi authored Jan 26, 2026


Signed-off-by: dafrimi <dafrimi@nvidia.com>
Co-authored-by: root <root@gpu-51.slurm-workers-slurm.slurm.svc.cluster.local>

67fe677c

Remove unused logic in `models/mistral.py` (#33095) · d56afd45
Andy Lo authored Jan 26, 2026
```
Signed-off-by: Andy Lo <andy@mistral.ai>
```
d56afd45
[ROCm][Bugfix] Fix ptpc scale load issue for fused shared expert path in deepseek mtp (#33018) · be6931ee
Pleaplusone authored Jan 26, 2026
```
Signed-off-by: ganyi <ygan@amd.com>
```
be6931ee