Commits · 64e39d667cb5b550e6ce148acd3d4dcd1654eace · OpenDAS / vllm_cscc

17 Nov, 2025 1 commit
- [Bugfix][Model] Prevent special token leakage in KimiK2ToolParser streaming mode (#28543) · 6f374192
  Jay Caldwell authored Nov 16, 2025
```
Signed-off-by: Jscaldwell55 <jay.s.caldwell@gmail.com>
```
  6f374192
16 Nov, 2025 4 commits

[BugFix] Fix async scheduling + chunked prefill + preemption (#28787) · 80b6080d
Nick Hill authored Nov 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
80b6080d
Feature: Support Relu2 in FusedMoE fp8 cutlass path (#27261) · 03ee4811
amirkl94 authored Nov 16, 2025

03ee4811

[V1] Support MP Executor for multi node distributed inference (#23691) · b316ac65

Lucia Fang authored Nov 16, 2025


Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

b316ac65

[Model] Allow users to control skip reading cache per request. (#28194) · a55b6463
wang.yuqi authored Nov 16, 2025
```
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
```
a55b6463

15 Nov, 2025 9 commits

Add support for Eagle with separate lm-head and embed_tokens layers (#28549) · e439c784
Eldar Kurtić authored Nov 15, 2025
```
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
```
e439c784

[compile] Enable sequence parallelism matching w/o custom ops enabled (#27126) · f36292db

Angela Yi authored Nov 15, 2025


Signed-off-by: angelayi <yiangela7@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: ProExpertProg <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <luka.govedic@gmail.com>

f36292db

[Misc] Make `SchedulerConfig.max_model_len` init-only (#28733) · 638e4196
Cyrus Leung authored Nov 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
638e4196

[Redo] #26368 (#28771) · 98b4d389

Cyrus Leung authored Nov 15, 2025


Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>

98b4d389

[Performance][DeepGEMM] Estimate expected_m (#28694) · 6965ef43

Varun Sundar Rabindranath authored Nov 15, 2025


Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>

6965ef43

[NIXL] heterogeneous block_size support (#26759) · c9e66585

Chendi.Xue authored Nov 14, 2025


Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>

c9e66585

Revert "[Core] Performance: Use list[np.ndarray] instead of list[list… (#28773) · ac86bff8
Nick Hill authored Nov 14, 2025

ac86bff8

[Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output... · 186352b2

Jialin Ouyang authored Nov 14, 2025


[Core] Performance: Use list[np.ndarray] instead of list[list[int]] for output tokens for GC optimization (#26368)
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>

186352b2

[Test] Rework e2e async scheduling tests (#28744) · 58e61e56
Nick Hill authored Nov 14, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
58e61e56

14 Nov, 2025 18 commits

Avoid bytecode hook and simplify TorchCompileWrapperWithCustomDipatch (#25110) · 2e0ad629
Laith Sakka authored Nov 14, 2025
```
Signed-off-by: Laith Sakka <lsakka@meta.com>
```
2e0ad629
[Bugfix] TypeError: 'NoneType' object is not callable (#27410) · 0de4f217
Marcin Ostrowski authored Nov 14, 2025
```
Signed-off-by: Marcin Ostrowski <marcinx.ostrowski@intel.com>
```
0de4f217
[Chore] Rename `SchedulerConfig.chunked_prefill_enabled` (#28735) · e2741f6c
Cyrus Leung authored Nov 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
e2741f6c
[Bugfix] [ROCm] [AITER]: Fix aiter block quant not compatible with torch compile dynamo (#28716) · a425dc25
TJian authored Nov 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
a425dc25

[DisaggEverything] Tokens in<>out `/generate` endpoint (#24261) · 6f1e7f72

Nicolò Lucchesi authored Nov 14, 2025

Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

6f1e7f72

[Docs] Update the name of `Transformers backend` -> `Transformers modeling backend` (#28725) · 5f3cd7f7
Harry Mellor authored Nov 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
5f3cd7f7

[Fix] improve aspect ratio in dummy image generation and add common VLM tests... · c934caee

dongbo910220 authored Nov 15, 2025


[Fix] improve aspect ratio in dummy image generation and add common  VLM tests for PaddleOCR-VL (#28711)
Signed-off-by: dongbo910220 <1275604947@qq.com>

c934caee

[Config] Clean up SchedulerConfig initialization (#28665) · 511a6b61
Cyrus Leung authored Nov 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
511a6b61
[Bugfix][Nixl] Fix kernel physical<>logical block_size issue (#28677) · 96b23b8e
Nicolò Lucchesi authored Nov 14, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
96b23b8e
[BugFix] Fix FA3 IMA with FULL_AND_PIECEWISE and cascade attention (default) (#28702) · db56a599
Lucas Wilkinson authored Nov 14, 2025

db56a599
Fix KV sharing fast prefill with cudagraph enabled (#28537) · 9324e102
Yong Hoon Shin authored Nov 14, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
9324e102

[DCP] Support Decode Context Parallel (DCP) for GQA with Flashinfer (#25438) · 4516d44b

Jingchun Gao authored Nov 14, 2025


Signed-off-by: gaojc <1055866782@qq.com>
Signed-off-by: Jingchun Gao <gaojingchun1@huawei.com>
Signed-off-by: Jingchun Gao <63247409+gjc0824@users.noreply.github.com>
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: gaojingchun (A) <g00955623@china.huawei.com>
Co-authored-by: Jingchun Gao <gaojingchun1@huawei.com>
Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com>

4516d44b

[Frontend] Added chat-style multimodal support to /classify. (#27516) · 360bd876

Srreyansh Sethi authored Nov 14, 2025


Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Signed-off-by: vnadathur <glvikramn@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: vnadathur <236933696+vnadathur@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: vnadathur <glvikramn@gmail.com>
Co-authored-by: wang.yuqi <noooop@126.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>

360bd876

[Misc] fix comment in test_envs (#28529) · 8cfbe89b
Xing Liu authored Nov 14, 2025
```
Signed-off-by: Xing Liu <xingliu14@gmail.com>
```
8cfbe89b
[Minor] avoid register new custom and just import silly_attn (#28578) · fd75d3e8
Boyuan Feng authored Nov 14, 2025
```
Signed-off-by: Boyuan Feng <boyuan@meta.com>
```
fd75d3e8
Add output token counting to gsm8k eval (#28594) · c9a3a021
Michael Goin authored Nov 14, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
c9a3a021

[BugFix][CI/Build][ROCM] Fix import error and apply assert in appropriate case... · 93103575

rasmith authored Nov 14, 2025


[BugFix][CI/Build][ROCM] Fix import error and apply assert in appropriate case in test_struct_output_generate (#28311)
Signed-off-by: Randall Smith <ransmith@amd.com>
Co-authored-by: Randall Smith <ransmith@amd.com>

93103575

[quantization][config] enable override existing quant_config (#28510) · 4d5943bd
Hank_ authored Nov 14, 2025
```
Signed-off-by: Hank <hcc.mayday@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
4d5943bd

13 Nov, 2025 8 commits

[KV Connector] Test async mode in scheduler tests (#28550) · 6e25b1cd
Mark McLoughlin authored Nov 13, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
6e25b1cd
[Bugfix] Eliminate tuple inputs to submodules in graph partitioning (#28533) · 262d263f
Yanan Cao authored Nov 13, 2025
```
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
```
262d263f

[Perf] Support stream interval for reducing host overhead (#27869) · 5d6ce2b9

elvischenv authored Nov 14, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

5d6ce2b9

[Bugfix] Fix validate model input for decoder models (#27099) · 119c4927

Yannick Schnider authored Nov 13, 2025


Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

119c4927

[Performance][B200] silu_mul_quant: pack scales in int32 (#28358) · fe1cd770

Varun Sundar Rabindranath authored Nov 13, 2025


Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>

fe1cd770

[Misc] Turn off encoder torch compile by default (#28634) · d3387750
Roger Wang authored Nov 13, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
d3387750
[Bugfix] Prevent crash on empty grammar string (#28210) · 4504e802
tjandy98 authored Nov 13, 2025
```
Signed-off-by: tjandy98 <3953059+tjandy98@users.noreply.github.com>
```
4504e802

[Frontend][responsesAPI][1/n] convert responses API tool input to chat... · 1a0b157a

Andrew Xia authored Nov 12, 2025


[Frontend][responsesAPI][1/n] convert responses API tool input to chat completions tool format (#28231)
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

1a0b157a