Commits · 4778b426605ec48b3eed90a94b29197835059090 · OpenDAS / vllm_cscc

"platforms/cuda-old/src/kernels/kCalculateObcGbsaBornSum.h" did not exist on "b9d12c46232671c10ade0e36ebd637a4f53bd50a"

26 Sep, 2025 6 commits

Reduce the Cuda Graph memory footprint when running with DBO (#25779) · 4778b426
Sage Moore authored Sep 26, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
4778b426

[V1] address post issues related to #20059 (part 1) (#23046) · f075693d

fhl2000 authored Sep 27, 2025


Signed-off-by: fhl2000 <63384265+fhl2000@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

f075693d

[BugFix] Fix using `dbo_decode_token_threshold` always (and ignoring... · 984d1849

Lucas Wilkinson authored Sep 26, 2025


[BugFix] Fix using `dbo_decode_token_threshold` always (and ignoring `dbo_prefill_token_threshold`) (#25622)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

984d1849

[Qwen3-Next][GDN] fixes cuda graph capturing bug in GDN metadata and a stride... · 99b3a504

Tao He authored Sep 26, 2025


[Qwen3-Next][GDN] fixes cuda graph capturing bug in GDN metadata and a stride bug in causal_conv_1d. (#25743)
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>

99b3a504

perf: Avoid copying inputs_embeds tensors to GPU unless prompt_embeds is enabled (#25739) · d48f4d6d
Andrew Sansom authored Sep 26, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
```
d48f4d6d

EVS Support (Video tokens pruning) (#22980) · 392edee3

Eugene Khvedchenya authored Sep 26, 2025


Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
Signed-off-by: Eugene Khvedchenya <ekhvedchenya@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

392edee3

25 Sep, 2025 9 commits
- [Spec Decode] Add Batch Parallel Ngram. Upto 8x lower overhead. (#24986) · e71b8e21
  Ekagra Ranjan authored Sep 25, 2025
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e71b8e21
- [Model] Define `merge_by_field_config` MM interface (#25676) · 0ea80c87
  Cyrus Leung authored Sep 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0ea80c87
- [BugFix] Fix DBO hang (#25625) · 13cc7f53
  Lucas Wilkinson authored Sep 25, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  13cc7f53
- [BUGFIX] Fix crash in Eagle Speculative Decoding models when exceedin… (#24662) · e04a1b6b
  AlonKejzman authored Sep 25, 2025
```
Signed-off-by: AlonKejzman <alonkeizman@gmail.com>
```
  e04a1b6b
- [Logging] Remove TORCH_NCCL_AVOID_RECORD_STREAMS to squash a warning (#25532) · 2e5df88c
  Tyler Michael Smith authored Sep 25, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2e5df88c
- [CPU] update torch 2.8 and fix missing fields in TorchSDPAMetadata (#25652) · eb32335e
  Li, Jiang authored Sep 25, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  eb32335e
- [Misc] Simplify PoolerOutput and move to `v1/outputs` (#25629) · 755ed7b0
  Cyrus Leung authored Sep 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  755ed7b0
- [Bugfix] fix apply_temperature to avoid nan in probs (#24734) · a676e668
  courage17340 authored Sep 25, 2025
```
Signed-off-by: courage17340 <courage17340@163.com>
```
  a676e668
- [Model] Add LongCat-Flash (#23991) · 845adb3e
  XuruiYang authored Sep 25, 2025
```
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
```
  845adb3e
24 Sep, 2025 4 commits
- Suppress benign cuBLAS warning when capturing cudagraphs with DBO (#25596) · f84a472a
  Sage Moore authored Sep 24, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  f84a472a
- [Bugfix] Fix dummy video number of frames calculation (#25553) · 42488dae
  Roger Wang authored Sep 24, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
  42488dae
- [TPU][Bugfix] fix the missing apply_model in tpu worker (#25526) · 190c45a6
  Chengji Yao authored Sep 23, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  190c45a6
- [BugFix] AssertionError: Do not capture num_reqs > max_num_reqs for uniform batch (#25505) · dc464a3d
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dc464a3d
23 Sep, 2025 10 commits

[BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) · abad204b
kourosh hakhamaneshi authored Sep 23, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
abad204b
[CI] Fix Pre-commit Issue (#25497) · 8b8a8afc
Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
8b8a8afc
[Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) · d5944d51
jiahanc authored Sep 23, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
```
d5944d51
[Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEWISE (#25444) · 24fab45d
Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
24fab45d

[core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3

Amir Samani authored Sep 23, 2025


Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

8c1c81a3

[Log] Optimize kv cache memory log from Bytes to GiB (#25204) · 846197f5
Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
846197f5

[Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed

Lucas Wilkinson authored Sep 23, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>