Commits · fb7abfc1d03c93d07813946d84368002b2cac833 · OpenDAS / vllm_cscc

28 Jan, 2026 4 commits
- [docs] Improve tlparse section (#33211) · fb7abfc1
  Angela Yi authored Jan 27, 2026
```
Signed-off-by: angelayi <yiangela7@gmail.com>
```
  fb7abfc1
- [CI] minor fixes to pipeline generator and tests (#33151) · 5d3d6e44
  Kevin H. Luu authored Jan 27, 2026
```
Signed-off-by: khluu <khluu000@gmail.com>
```
  5d3d6e44
- [Model Runner V2] Use a different stream for grammar bitmask h2d copy (#33059) · 46ec6d71
  Woosuk Kwon authored Jan 27, 2026
```
Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  46ec6d71
- Add attention benchmarking tools (#26835) · e82fa448
  Matthew Bonanni authored Jan 27, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
```
  e82fa448
27 Jan, 2026 33 commits

[torch.compile] Speed up MOE handling in forward_context (#33184) · d9aa39a3
Richard Zou authored Jan 27, 2026
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
d9aa39a3
[Perf] Optimize dcp allocate tensor (#33102) · 3a6d5cbe
Wentao Ye authored Jan 27, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
3a6d5cbe
[Bugfix] Fix display error (inconsistent with context) (#33020) · f5d7049c
linhaifeng authored Jan 28, 2026
```
Signed-off-by: linhaifeng <1371675203@qq.com>
```
f5d7049c

Enabling "2 node" distributed tests in the AMD CI pipeline. (#32719) · 3c3c547c

Alexei-V-Ivanov-AMD authored Jan 27, 2026

Signed-off-by: DCCS-4560 <alivanov@chi-mi325x-pod1-112.ord.vultr.cpe.ice.amd.com>
Co-authored-by: DCCS-4560 <alivanov@chi-mi325x-pod1-112.ord.vultr.cpe.ice.amd.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>

3c3c547c

[Attention] Use `has_flashinfer` helper (#33177) · 1cbccb6d
Matthew Bonanni authored Jan 27, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
1cbccb6d

feature: support eagle3 for HunyuanVL & Hunyuan (#33035) · bd92089d

Iris authored Jan 28, 2026


Signed-off-by: irisliu10 <601012173@qq.com>
Signed-off-by: Iris <38269816+irisliu10@users.noreply.github.com>

bd92089d

[Doc] Improve serve parameter documentation with meaningful defaults (#33082) · a6760f15

Karan Bansal authored Jan 27, 2026

Signed-off-by: Karan Bansal <karanb192@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

a6760f15

Support compress-tensors with nvfp4 or fp8 weights and modelopt with nvfp4... · 66e601ef

IriKa authored Jan 28, 2026


Support compress-tensors with nvfp4 or fp8 weights and modelopt with nvfp4 weights on Turing (#33076)
Signed-off-by: IriKa Qiu <qiujie.jq@gmail.com>

66e601ef

[BugFix] Fix P/D with non-MoE DP (#33037) · 0cd259b2
Nick Hill authored Jan 27, 2026
```
Signed-off-by: Nick Hill <nickhill123@gmail.com>
```
0cd259b2

Support heterogeneous NemotronHPuzzle model (#32549) · 83fb2d09

danielafrimi authored Jan 27, 2026



Signed-off-by: <dafrimi@nvidia.com>
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com>
Signed-off-by: root <dafrimi@nvidia.com>

83fb2d09

[LoRA][Spec Decode] Support LoRA for Nemotron-H MTP models (#32265) · f3a5ee70

danisereb authored Jan 27, 2026


Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

f3a5ee70

[Frontend] Cleanup api server (#33158) · 7cbbca9a

wang.yuqi authored Jan 27, 2026


Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>

7cbbca9a

[Metrics][MFU] Fix UnembedMetrics FLOP overcounting for prefill (#33045) (#33045) · 5ec44056

omkhalil authored Jan 27, 2026

Fix UnembedMetrics to correctly count FLOPs for the unembedding (LM head) layer.

The bug: UnembedMetrics used total_num_tokens() which counts all tokens in the
batch for projection flops, vocab projections are run on just the last token for the
autoregressive use case.
Co-authored-by: Omar Mohamed Khalil <omarkhalil@meta.com>

5ec44056

[Bugfix] Fix DeepseekV32 `AssertionError: num_kv_heads == 1` (#33090) · 492a7983
Nicolò Lucchesi authored Jan 27, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
492a7983
[5/N][Attention] Finish eliminating `vllm/attention` folder (#32064) · a608b4c6
Matthew Bonanni authored Jan 27, 2026
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
a608b4c6
[Bugfix] Disable CG for Whisper+FA2 (#33164) · 1f3a2c29
Nicolò Lucchesi authored Jan 27, 2026
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
1f3a2c29

[Metrics] [KVConnector] Add Offloading Connector metrics (#27942) · 7227d061

omerpaz95 authored Jan 27, 2026



Added queries and hits metrics for the Offloading Connector.

Also added timing metrics for store and load operations, which take the
average time it takes to load/store, per-token.

The metrics are available from Prometheus and from the StatLogger.
Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Omer Paz <Omer.Paz@ibm.com>

7227d061

Fix weight mapping test for Transfomers v5 (#33162) · 14385c80
Harry Mellor authored Jan 27, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
14385c80

[Frontend] Frontend will only attach supported tasks corresponding entrypoints. (#33139) · 76139d08

wang.yuqi authored Jan 27, 2026


Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

76139d08

[AMD][QWEN3-NEXT] FP8 Tunings (#32042) · da8d0c44
Lifan Shen authored Jan 27, 2026
```
Signed-off-by: Lifan Shen <lifans@meta.com>
```
da8d0c44

[AMD][Kernel][BugFix] Use correct scale in concat_and_cache_ds_mla_kernel when on gfx942 (#32976) · 58996f35

rasmith authored Jan 27, 2026


Signed-off-by: Randall Smith <ransmith@amd.com>
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>

58996f35

[Models] Kimi-K2.5 (#33131) · b539f988

Roger Wang authored Jan 26, 2026


Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: wanglinian <wanglinian@stu.pku.edu.cn>
Co-authored-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Co-authored-by: Zaida Zhou <58739961+zhouzaida@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

b539f988

[CI][Pooling] Stabilize ModernBERT test (#32909) · 6c006457
Andreas Karatzas authored Jan 26, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
6c006457
[code clean] remove duplicate code (#33135) · b781eeaa
Ning Xie authored Jan 27, 2026
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
b781eeaa
[Frontend] Cleanup serving engine (#33103) · e0b005d9
Cyrus Leung authored Jan 27, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
e0b005d9
[torch.compile] Stop assuming 32 bit indexing (#33113) · 3b8f0fe5
Richard Zou authored Jan 26, 2026
```
Signed-off-by: Richard Zou <zou3519@gmail.com>
```
3b8f0fe5
[Frontend] Reduce mixin usage in serving pooling (#33101) · c831911b
Cyrus Leung authored Jan 27, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
c831911b
[Perf] avoid duplicate mem_get_info() call in get_current_memory_usage (#33064) · 157caf51
Paco Xu authored Jan 27, 2026
```
Signed-off-by: Paco Xu <paco.xu@daocloud.io>
```
157caf51

[DOC]: Add warning about max_num_batched_tokens and max_model_len when chunked... · 0b53bec6

Vincent Gimenes authored Jan 27, 2026


[DOC]: Add warning about max_num_batched_tokens and max_model_len when chunked prefill is disabled (#33109)
Signed-off-by: Vincent Gimenes <147169146+VincentG1234@users.noreply.github.com>

0b53bec6

Fix IndexError with encoder-decoder models when using Custom Paged Attention (#33112) · c568581f
Strahinja Stamenkovic authored Jan 27, 2026
```
Signed-off-by: sstamenk <strahinja.stamenkovic@amd.com>
```
c568581f

fix: preserve native tool call ID in multi-turn tool calling (#32768) · 2d705343

wangln19 authored Jan 27, 2026


Signed-off-by: wanglinian <wanglinian@stu.pku.edu.cn>
Signed-off-by: wangln19 <96399074+wangln19@users.noreply.github.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Isotr0py <2037008807@qq.com>

2d705343

[MoE Refactor] Integrate Naive Prepare Finalize into MK (#32567) · 5a93b916

Robert Shaw authored Jan 26, 2026


Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: amirkl94 <203507526+amirkl94@users.noreply.github.com>

5a93b916

[Model Runner V2] Remove UvaBufferPool for cpu->gpu copy (#33055) · 6d86fde0

Woosuk Kwon authored Jan 26, 2026


Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Nick Hill <nhill@redhat.com>

6d86fde0

26 Jan, 2026 3 commits

[Bugfix][TPU] Return a Default fp8 MoE Backend (#32908) · 510ed1e8

XiongfeiWei authored Jan 26, 2026


Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

510ed1e8

[Bugfix][MXFP4] Call `trtllm_fp4_block_scale_moe` with kwargs (#33104) · 8caffd92
Pengchao Wang authored Jan 26, 2026
```
Signed-off-by: Pengchao Wang <wpc@fb.com>
```
8caffd92
[fix] CPUDNNLGEMMHandler pointer baked into inductor artifact (#32913) · 58a05b0c
dolpm authored Jan 26, 2026
```
Signed-off-by: dolpm <34420038+dolpm@users.noreply.github.com>
```
58a05b0c