Commits · 6de3d431d911eff03a3e1b69233f68b5e7f8a472 · OpenDAS / vllm_cscc

28 Sep, 2025 2 commits
- [MM] Optimize memory profiling for scattered multimodal embeddings (#25810) · 6de3d431
  Roger Wang authored Sep 27, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  6de3d431
- Reduce the Cuda Graph memory footprint when running with DBO (#25779) · bb79c4da
  Sage Moore authored Sep 26, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  bb79c4da
26 Sep, 2025 11 commits

[ray][metrics] Replace ':' with '_' for OpenTelemetry compatibility in Ray (#25439) · 8d52f2b3

Seiji Eicher authored Sep 26, 2025

Signed-off-by: Seiji Eicher <seiji@anyscale.com>
Signed-off-by: Seiji Eicher <58963096+eicherseiji@users.noreply.github.com>
Co-authored-by: Rui Qiao <161574667+ruisearch42@users.noreply.github.com>

8d52f2b3

[BugFix] Fix using `dbo_decode_token_threshold` always (and ignoring... · 984d1849

Lucas Wilkinson authored Sep 26, 2025


[BugFix] Fix using `dbo_decode_token_threshold` always (and ignoring `dbo_prefill_token_threshold`) (#25622)
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

984d1849

[Bugfix] Properly abort pooling request. (#25734) · fe6b19c3

wang.yuqi authored Sep 26, 2025


Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

fe6b19c3

[Model] Mamba2 varlen refactor (#21467) · 2b6b1d78

Chih-Chieh Yang authored Sep 26, 2025


Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
Co-authored-by: RishiAstra <40644327+RishiAstra@users.noreply.github.com>

2b6b1d78

Remove cuda hard-code in compute_causal_conv1d_metadata (#25555) · dd70437a
Icey authored Sep 26, 2025
```
Signed-off-by: Icey <1790571317@qq.com>
```
dd70437a

[Qwen3-Next][GDN] fixes cuda graph capturing bug in GDN metadata and a stride... · 99b3a504

Tao He authored Sep 26, 2025


[Qwen3-Next][GDN] fixes cuda graph capturing bug in GDN metadata and a stride bug in causal_conv_1d. (#25743)
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>

99b3a504

perf: Avoid copying inputs_embeds tensors to GPU unless prompt_embeds is enabled (#25739) · d48f4d6d
Andrew Sansom authored Sep 26, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
```
d48f4d6d

fix: revert cast to cpu in `MsgpackEncoder._encode_tensor` to avoid hidden... · e84e0735

Andrew Sansom authored Sep 26, 2025


fix: revert cast to cpu in `MsgpackEncoder._encode_tensor` to avoid hidden performance regressions (#25738)
Signed-off-by: Andrew Sansom <andrew@protopia.ai>

e84e0735

[CI/Build] fix doc build warning: Failed to get 'name: description' pair (#25733) · 3edf87d2
yitingdc authored Sep 26, 2025
```
Signed-off-by: yiting.jiang <yiting.jiang@daocloud.io>
```
3edf87d2

EVS Support (Video tokens pruning) (#22980) · 392edee3

Eugene Khvedchenya authored Sep 26, 2025


Signed-off-by: Eugene Khvedchenia <ekhvedchenia@nvidia.com>
Signed-off-by: Eugene Khvedchenya <ekhvedchenya@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

392edee3

[Misc] Don't log shm dequeue delay warning on worker side (#25720) · 8b77328f
Nick Hill authored Sep 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
8b77328f

25 Sep, 2025 16 commits
- [Core] Enable command line logging for LLMEngine (#25610) · 8c435c9b
  Zhuohan Li authored Sep 25, 2025
```
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Zhuohan Li <zhuohan123@gmail.com>
```
  8c435c9b
- [Spec Decode] Add Batch Parallel Ngram. Upto 8x lower overhead. (#24986) · e71b8e21
  Ekagra Ranjan authored Sep 25, 2025
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e71b8e21
- [V0 deprecation] Remove _VLLM_V1 suffixes from attention backend names (#25489) · 3468f17e
  Matthew Bonanni authored Sep 25, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
  3468f17e
- [Model] Define `merge_by_field_config` MM interface (#25676) · 0ea80c87
  Cyrus Leung authored Sep 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0ea80c87
- [BugFix] Fix DBO hang (#25625) · 13cc7f53
  Lucas Wilkinson authored Sep 25, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  13cc7f53
- [BUGFIX] Fix crash in Eagle Speculative Decoding models when exceedin… (#24662) · e04a1b6b
  AlonKejzman authored Sep 25, 2025
```
Signed-off-by: AlonKejzman <alonkeizman@gmail.com>
```
  e04a1b6b
- [Logging] Remove TORCH_NCCL_AVOID_RECORD_STREAMS to squash a warning (#25532) · 2e5df88c
  Tyler Michael Smith authored Sep 25, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2e5df88c
- [ux] Switch a warning to debug about a pytorch fallback (#23750) · 532a6cfc
  Russell Bryant authored Sep 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  532a6cfc
- [CPU] update torch 2.8 and fix missing fields in TorchSDPAMetadata (#25652) · eb32335e
  Li, Jiang authored Sep 25, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  eb32335e
- [torch.compile] Make Query Quantization Fusable (#24914) · 69a8c8e9
  Jonas M. Kübler authored Sep 25, 2025
```
Signed-off-by: Jonas Kuebler <kuebj@amazon.com>
```
  69a8c8e9
- typo: remove duplicate `is` (#25641) · af4ee63e
  Nicole LiHui 🥜 authored Sep 25, 2025
```
Signed-off-by: nicole-lihui <nicole.li@daocloud.io>
```
  af4ee63e
- [Misc] Simplify PoolerOutput and move to `v1/outputs` (#25629) · 755ed7b0
  Cyrus Leung authored Sep 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  755ed7b0
- [Bugfix] fix apply_temperature to avoid nan in probs (#24734) · a676e668
  courage17340 authored Sep 25, 2025
```
Signed-off-by: courage17340 <courage17340@163.com>
```
  a676e668
- optimize: eliminate duplicate split_enc_dec_inputs calls (#25573) · c85be1f6
  Nicole LiHui 🥜 authored Sep 25, 2025
```
Signed-off-by: nicole-lihui <nicole.li@daocloud.io>
```
  c85be1f6
- [Model] Add LongCat-Flash (#23991) · 845adb3e
  XuruiYang authored Sep 25, 2025
```
Signed-off-by: yangxurui <yangxurui@meituan.com>
Co-authored-by: yangxurui <yangxurui@meituan.com>
```
  845adb3e
- [Kernel] Support DCP for Triton backend (#25132) · 05c19485
  Wei Wei authored Sep 24, 2025
```
Signed-off-by: Wei Wei <wwei6@meta.com>
```
  05c19485
24 Sep, 2025 11 commits
- [V0 Deprecation] Remove unused classes in attention (#25541) · e6750d0b
  Woosuk Kwon authored Sep 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  e6750d0b
- [Docs] Enable `fail_on_warning` for the docs build in CI (#25580) · 8c853050
  Harry Mellor authored Sep 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8c853050
- Suppress benign cuBLAS warning when capturing cudagraphs with DBO (#25596) · f84a472a
  Sage Moore authored Sep 24, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  f84a472a
- [Bugfix] Fix dummy video number of frames calculation (#25553) · 42488dae
  Roger Wang authored Sep 24, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
  42488dae
- [BugFix] Potential Fix for FA3 full-cudagraph IMA (#25490) · 2338daff
  Lucas Wilkinson authored Sep 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  2338daff
- [TPU][Bugfix] fix the missing apply_model in tpu worker (#25526) · 190c45a6
  Chengji Yao authored Sep 23, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  190c45a6
- [Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b
  Benjamin Chislett authored Sep 23, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>
```
  c30b405b
- [BugFix] Fix MLA assert with CUTLASS MLA (#25478) · 9df8da54
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9df8da54
- [Bugfix] Use a separate FlashInfer workspace buffer for trtllm-gen (#25520) · 19836092
  Benjamin Chislett authored Sep 23, 2025
  
  19836092
- [V1][Metrics] Add per-request TPOT histogram (#24015) · d06b5a95
  baxingpiaochong authored Sep 24, 2025
```
Signed-off-by: baxingpiaochong <771405853@qq.com>
```
  d06b5a95
- [BugFix] AssertionError: Do not capture num_reqs > max_num_reqs for uniform batch (#25505) · dc464a3d
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dc464a3d