Commits · e71b8e210db4b98ffa4398d25f8bdf280686ad78 · OpenDAS / vllm_cscc · GitLab

25 Sep, 2025 15 commits
- [Spec Decode] Add Batch Parallel Ngram. Upto 8x lower overhead. (#24986) · e71b8e21
  Ekagra Ranjan authored Sep 25, 2025
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e71b8e21
- [V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) · 3468f17e
  Matthew Bonanni authored Sep 25, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
  3468f17e
- [Model] Define `merge_by_field_config` MM interface (#25676) · 0ea80c87
  Cyrus Leung authored Sep 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0ea80c87
- [BugFix] Fix DBO hang (#25625) · 13cc7f53
  Lucas Wilkinson authored Sep 25, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  13cc7f53
- [BUGFIX] Fix crash in Eagle Speculative Decoding models when exceedin… (#24662) · e04a1b6b
  AlonKejzman authored Sep 25, 2025
```
Signed-off-by: AlonKejzman <alonkeizman@gmail.com>
```
  e04a1b6b
- [Logging] Remove TORCH_NCCL_AVOID_RECORD_STREAMS to squash a warning (#25532) · 2e5df88c
  Tyler Michael Smith authored Sep 25, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2e5df88c
- [ux] Switch a warning to debug about a pytorch fallback (#23750) · 532a6cfc
  Russell Bryant authored Sep 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  532a6cfc
- [CPU] update torch 2.8 and fix missing fields in TorchSDPAMetadata (#25652) · eb32335e
  Li, Jiang authored Sep 25, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  eb32335e
- [torch.compile] Make Query Quantization Fusable (#24914) · 69a8c8e9
  Jonas M. Kübler authored Sep 25, 2025
```
Signed-off-by: Jonas Kuebler <kuebj@amazon.com>
```
  69a8c8e9
- typo: remove duplicate `is` (#25641) · af4ee63e
  Nicole LiHui 🥜 authored Sep 25, 2025
```
Signed-off-by: nicole-lihui <nicole.li@daocloud.io>
```
  af4ee63e
- [Misc] Simplify PoolerOutput and move to `v1/outputs` (#25629) · 755ed7b0
  Cyrus Leung authored Sep 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  755ed7b0
- [Bugfix] fix apply_temperature to avoid nan in probs (#24734) · a676e668
  courage17340 authored Sep 25, 2025
```
Signed-off-by: courage17340 <courage17340@163.com>
```
  a676e668
- optimize: eliminate duplicate split_enc_dec_inputs calls (#25573) · c85be1f6
  Nicole LiHui 🥜 authored Sep 25, 2025
```
Signed-off-by: nicole-lihui <nicole.li@daocloud.io>
```
  c85be1f6
- [Model] Add LongCat-Flash (#23991) · 845adb3e
  XuruiYang authored Sep 25, 2025
```
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
```
  845adb3e
- [Kernel] Support DCP for Triton backend (#25132) · 05c19485
  Wei Wei authored Sep 24, 2025
```
Signed-off-by: Wei Wei <wwei6@meta.com>
```
  05c19485
24 Sep, 2025 11 commits
- [V0 Deprecation] Remove unused classes in attention (#25541) · e6750d0b
  Woosuk Kwon authored Sep 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  e6750d0b
- [Docs] Enable `fail_on_warning` for the docs build in CI (#25580) · 8c853050
  Harry Mellor authored Sep 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8c853050
- Suppress benign cuBLAS warning when capturing cudagraphs with DBO (#25596) · f84a472a
  Sage Moore authored Sep 24, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  f84a472a
- [Bugfix] Fix dummy video number of frames calculation (#25553) · 42488dae
  Roger Wang authored Sep 24, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
  42488dae
- [BugFix] Potential Fix for FA3 full-cudagraph IMA (#25490) · 2338daff
  Lucas Wilkinson authored Sep 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  2338daff
- [TPU][Bugfix] fix the missing apply_model in tpu worker (#25526) · 190c45a6
  Chengji Yao authored Sep 23, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  190c45a6
- [Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b
  Benjamin Chislett authored Sep 23, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
```
  c30b405b
- [BugFix] Fix MLA assert with CUTLASS MLA (#25478) · 9df8da54
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9df8da54
- [Bugfix] Use a separate FlashInfer workspace buffer for trtllm-gen (#25520) · 19836092
  Benjamin Chislett authored Sep 23, 2025
  
  19836092
- [V1][Metrics] Add per-request TPOT histogram (#24015) · d06b5a95
  baxingpiaochong authored Sep 24, 2025
```
Signed-off-by: baxingpiaochong <771405853@qq.com>
```
  d06b5a95
- [BugFix] AssertionError: Do not capture num_reqs > max_num_reqs for uniform batch (#25505) · dc464a3d
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dc464a3d
23 Sep, 2025 14 commits
- Add `VLLM_NVTX_SCOPES_FOR_PROFILING=1` to enable `nvtx.annotate` scopes (#25501) · c85d75cf
  Corey Lowman authored Sep 23, 2025
```
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
```
  c85d75cf
- [BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) · abad204b
  kourosh hakhamaneshi authored Sep 23, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
  abad204b
- [Core] Use KVCacheBlock as much as possible instead of dict[block_id, KVCacheBlock] (#24830) · 4f8c4b89
  Jialin Ouyang authored Sep 23, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  4f8c4b89
- [Build] Update Xgrammar to 0.1.25 (#25467) · 08275ec0
  Chauncey authored Sep 24, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  08275ec0
- [CI] Fix Pre-commit Issue (#25497) · 8b8a8afc
  Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  8b8a8afc
- [Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) · d5944d51
  jiahanc authored Sep 23, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
```
  d5944d51
- [Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEWISE (#25444) · 24fab45d
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  24fab45d
- [core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3
  Amir Samani authored Sep 23, 2025
```
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  8c1c81a3
- [V1][Kernel] Add triton implementation for `reshape_and_cache_flash` (#24503) · 100b630a
  Burkhard Ringlein authored Sep 23, 2025
```
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  100b630a
- [Log] Optimize kv cache memory log from Bytes to GiB (#25204) · 846197f5
  Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  846197f5
- [Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc1dc7ed
- [V1] Remove V0 code paths for Hybrid models (#25400) · a903669e
  Thomas Parnell authored Sep 23, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  a903669e
- [UX] Change kv-cache-memory log level to debug (#25479) · 2c58742d
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  2c58742d
- [Bug] Fix Long Context OOM Issue (#25290) · 4741239d
  Wentao Ye authored Sep 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  4741239d