Commits · f84a472a03cf86cc7362eedaf6c2f09cfa90982c · OpenDAS / vllm_cscc · GitLab

24 Sep, 2025 9 commits
- Suppress benign cuBLAS warning when capturing cudagraphs with DBO (#25596) · f84a472a
  Sage Moore authored Sep 24, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  f84a472a
- [Bugfix] Fix dummy video number of frames calculation (#25553) · 42488dae
  Roger Wang authored Sep 24, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
  42488dae
- [BugFix] Potential Fix for FA3 full-cudagraph IMA (#25490) · 2338daff
  Lucas Wilkinson authored Sep 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  2338daff
- [TPU][Bugfix] fix the missing apply_model in tpu worker (#25526) · 190c45a6
  Chengji Yao authored Sep 23, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  190c45a6
- [Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b
  Benjamin Chislett authored Sep 23, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
```
  c30b405b
- [BugFix] Fix MLA assert with CUTLASS MLA (#25478) · 9df8da54
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9df8da54
- [Bugfix] Use a separate FlashInfer workspace buffer for trtllm-gen (#25520) · 19836092
  Benjamin Chislett authored Sep 23, 2025
  
  19836092
- [V1][Metrics] Add per-request TPOT histogram (#24015) · d06b5a95
  baxingpiaochong authored Sep 24, 2025
```
Signed-off-by: baxingpiaochong <771405853@qq.com>
```
  d06b5a95
- [BugFix] AssertionError: Do not capture num_reqs > max_num_reqs for uniform batch (#25505) · dc464a3d
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dc464a3d
23 Sep, 2025 18 commits
- Add `VLLM_NVTX_SCOPES_FOR_PROFILING=1` to enable `nvtx.annotate` scopes (#25501) · c85d75cf
  Corey Lowman authored Sep 23, 2025
```
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
```
  c85d75cf
- [BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) · abad204b
  kourosh hakhamaneshi authored Sep 23, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
  abad204b
- [Core] Use KVCacheBlock as much as possible instead of dict[block_id, KVCacheBlock] (#24830) · 4f8c4b89
  Jialin Ouyang authored Sep 23, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  4f8c4b89
- [Build] Update Xgrammar to 0.1.25 (#25467) · 08275ec0
  Chauncey authored Sep 24, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  08275ec0
- [CI] Fix Pre-commit Issue (#25497) · 8b8a8afc
  Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  8b8a8afc
- [Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) · d5944d51
  jiahanc authored Sep 23, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
```
  d5944d51
- [Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEWISE (#25444) · 24fab45d
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  24fab45d
- [core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3
  Amir Samani authored Sep 23, 2025
```
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  8c1c81a3
- [V1][Kernel] Add triton implementation for `reshape_and_cache_flash` (#24503) · 100b630a
  Burkhard Ringlein authored Sep 23, 2025
```
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  100b630a
- [Log] Optimize kv cache memory log from Bytes to GiB (#25204) · 846197f5
  Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  846197f5
- [Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc1dc7ed
- [V1] Remove V0 code paths for Hybrid models (#25400) · a903669e
  Thomas Parnell authored Sep 23, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  a903669e
- [UX] Change kv-cache-memory log level to debug (#25479) · 2c58742d
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  2c58742d
- [Bug] Fix Long Context OOM Issue (#25290) · 4741239d
  Wentao Ye authored Sep 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  4741239d
- [Performance] Remove input pads in cutlass_mla and optimize v_proj output handling (#25184) · 0b7bed9c
  Alexander Matveev authored Sep 22, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  0b7bed9c
- [BugFix] [DP/EP] Fix slow execution when BS <= DP (#25407) · ac0048c0
  Matthew Bonanni authored Sep 22, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Chris Bamford <chrisbam4d@gmail.com>
```
  ac0048c0
- [Bugfix] Fix missing `clear_connector_metadata` (#25397) · 09019703
  Nicolò Lucchesi authored Sep 23, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  09019703
- [Core] Drop overly aggressive whisper assertion (#25408) · f31ff874
  Russell Bryant authored Sep 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  f31ff874
22 Sep, 2025 7 commits
- [KV offload][5/N] Add `CPUOffloadingSpec` (#24251) · 8db29392
  Or Ozeri authored Sep 22, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  8db29392
- [DP] support torchrun external launcher with Data Parallelism (#24899) · 922979bf
  Lucia Fang authored Sep 22, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  922979bf
- [CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) · cfbee3d0
  Daisy-Ma-coder authored Sep 22, 2025
```
Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  cfbee3d0
- [V1][Attention] Split triton_attn in triton-only and rocm specific backends (#24648) · 175811e3
  Burkhard Ringlein authored Sep 22, 2025
```
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
```
  175811e3
- [TPU] Deprecate `xm.mark_step` in favor of ``torch_xla.sync` (#25254) · 4cf71cc8
  Nicolò Lucchesi authored Sep 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  4cf71cc8
- Enable Eagle3 speculative decoding for GPT-OSS model (#25246) · 21467f9a
  Eldar Kurtić authored Sep 22, 2025
```
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
```
  21467f9a
- [V0 Deprecation] Remove `MultiModalPlaceholderMap` (#25366) · f92d9526
  Cyrus Leung authored Sep 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f92d9526
21 Sep, 2025 5 commits
- [V0 Deprecation] Remove V0 sampling metadata (#25345) · 1c3ffdbe
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  1c3ffdbe
- [V1] Add sliding window support to Flex Attention backend (#24089) · cf56cf78
  Isotr0py authored Sep 21, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  cf56cf78
- [V0 Deprecation] Remove V0 MP executor (#25329) · 7ed82d19
  Woosuk Kwon authored Sep 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7ed82d19
- [V0 Deprecation] Remove V0 core (#25321) · c99db8c8
  Woosuk Kwon authored Sep 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c99db8c8
- [V0 Deprecation] Remove V0 Output Processor (#25320) · 86647d1c
  Woosuk Kwon authored Sep 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  86647d1c
20 Sep, 2025 1 commit
- [Docs] Fix warnings in mkdocs build (continued) (#25042) · 032d661d
  Wenlong Wang authored Sep 20, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  032d661d