Commits · e71b8e210db4b98ffa4398d25f8bdf280686ad78 · OpenDAS / vllm_cscc

25 Sep, 2025 9 commits
- [Spec Decode] Add Batch Parallel Ngram. Upto 8x lower overhead. (#24986) · e71b8e21
  Ekagra Ranjan authored Sep 25, 2025
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e71b8e21
- [Model] Define `merge_by_field_config` MM interface (#25676) · 0ea80c87
  Cyrus Leung authored Sep 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0ea80c87
- [BugFix] Fix DBO hang (#25625) · 13cc7f53
  Lucas Wilkinson authored Sep 25, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  13cc7f53
- [BUGFIX] Fix crash in Eagle Speculative Decoding models when exceedin… (#24662) · e04a1b6b
  AlonKejzman authored Sep 25, 2025
```
Signed-off-by: AlonKejzman <alonkeizman@gmail.com>
```
  e04a1b6b
- [Logging] Remove TORCH_NCCL_AVOID_RECORD_STREAMS to squash a warning (#25532) · 2e5df88c
  Tyler Michael Smith authored Sep 25, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2e5df88c
- [CPU] update torch 2.8 and fix missing fields in TorchSDPAMetadata (#25652) · eb32335e
  Li, Jiang authored Sep 25, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  eb32335e
- [Misc] Simplify PoolerOutput and move to `v1/outputs` (#25629) · 755ed7b0
  Cyrus Leung authored Sep 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  755ed7b0
- [Bugfix] fix apply_temperature to avoid nan in probs (#24734) · a676e668
  courage17340 authored Sep 25, 2025
```
Signed-off-by: courage17340 <courage17340@163.com>
```
  a676e668
- [Model] Add LongCat-Flash (#23991) · 845adb3e
  XuruiYang authored Sep 25, 2025
```
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
```
  845adb3e
24 Sep, 2025 4 commits
- Suppress benign cuBLAS warning when capturing cudagraphs with DBO (#25596) · f84a472a
  Sage Moore authored Sep 24, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  f84a472a
- [Bugfix] Fix dummy video number of frames calculation (#25553) · 42488dae
  Roger Wang authored Sep 24, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
  42488dae
- [TPU][Bugfix] fix the missing apply_model in tpu worker (#25526) · 190c45a6
  Chengji Yao authored Sep 23, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  190c45a6
- [BugFix] AssertionError: Do not capture num_reqs > max_num_reqs for uniform batch (#25505) · dc464a3d
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dc464a3d
23 Sep, 2025 10 commits

[BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) · abad204b
kourosh hakhamaneshi authored Sep 23, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
abad204b
[CI] Fix Pre-commit Issue (#25497) · 8b8a8afc
Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
8b8a8afc
[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) · d5944d51
jiahanc authored Sep 23, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
```
d5944d51
[Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEWISE (#25444) · 24fab45d
Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
24fab45d

[core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3

Amir Samani authored Sep 23, 2025


Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

8c1c81a3

[Log] Optimize kv cache memory log from Bytes to GiB (#25204) · 846197f5
Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
846197f5

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed

Lucas Wilkinson authored Sep 23, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

cc1dc7ed

[UX] Change kv-cache-memory log level to debug (#25479) · 2c58742d
Michael Goin authored Sep 23, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
2c58742d

[BugFix] [DP/EP] Fix slow execution when BS <= DP (#25407) · ac0048c0

Matthew Bonanni authored Sep 22, 2025

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Chris Bamford <chrisbam4d@gmail.com>

ac0048c0

[Bugfix] Fix missing `clear_connector_metadata` (#25397) · 09019703
Nicolò Lucchesi authored Sep 23, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
09019703

22 Sep, 2025 1 commit
- [TPU] Deprecate `xm.mark_step` in favor of ``torch_xla.sync` (#25254) · 4cf71cc8
  Nicolò Lucchesi authored Sep 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  4cf71cc8
21 Sep, 2025 1 commit
- [V0 Deprecation] Remove V0 sampling metadata (#25345) · 1c3ffdbe
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  1c3ffdbe
20 Sep, 2025 4 commits
- [Docs] Fix warnings in mkdocs build (continued) (#25042) · 032d661d
  Wenlong Wang authored Sep 20, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  032d661d
- [Hybrid Allocator] Support full attention with different hidden size (#25101) · 9607d5eb
  Chen Zhang authored Sep 19, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  9607d5eb
- [BugFix] Ensure appropriate guards in destructors (#25284) · a25ade5d
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  a25ade5d
- [Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) · 3da17c2c
  Lucas Kabela authored Sep 19, 2025
```
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
```
  3da17c2c
19 Sep, 2025 4 commits
- [BugFix] Fix async scheduling CPU tensor race take 2 (#25279) · 14c14327
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  14c14327
- [Core] Modify the initialization parameters of the lora manager (#25249) · 28219864
  Jee Jee Li authored Sep 20, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  28219864
- [P/D][Nixl] Introduce `KVTransferMetrics` and aggregation strategy (#22188) · a3d087ad
  Nicolò Lucchesi authored Sep 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a3d087ad
- [CORE] Prompt Embeddings Support for v1 Engine (#24278) · 9a4600e4
  Andrew Sansom authored Sep 18, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9a4600e4
18 Sep, 2025 5 commits

[feat]: Create interface for model-specific M-RoPE (#24194) · 38db529f

Aziz authored Sep 18, 2025

Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
Signed-off-by: Aziz <azizbenothman76@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

38db529f

[Structured Output][Refactor] Move `apply_grammar_bitmask()` method from... · 470484a4

Shanshan Shen authored Sep 18, 2025


[Structured Output][Refactor] Move `apply_grammar_bitmask()` method from `ModelRunner` to structured output utils (#21999)
Signed-off-by: shen-shanshan <467638484@qq.com>

470484a4

[Model] Improve Pooling Model (#25149) · 37970105
Jee Jee Li authored Sep 18, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
37970105
[Spec Decode] Efficient padded speculation (#24539) · b7433ca1
Benjamin Chislett authored Sep 18, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
b7433ca1
[XPU] Whisper model support on XPU Platform (#25123) · 3bc18127
Chaojun Zhang authored Sep 18, 2025
```
Signed-off-by: chzhang <chaojun.zhang@intel.com>
```
3bc18127

17 Sep, 2025 2 commits
- [Bugfix] Fix Stream usage in CPU model runner and OneDNN kernel check (#25046) · 9fccd04e
  Li, Jiang authored Sep 17, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  9fccd04e
- [XPU] Fix xpu model runner call torch.cuda APIs (#25011) · dd39baf7
  Kunshang Ji authored Sep 17, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  dd39baf7