Commits · 767cbb011dfa6e7685739dd29b68cde0727e459a · OpenDAS / vllm_cscc

03 Oct, 2025 36 commits
- [CI] Fix Pre-commit Mypy Error (#26181) · 767cbb01
  Wentao Ye authored Oct 03, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  767cbb01
- [BugFix] Fix de-functionalization pass for rotary_embedding (#23953) · 7cfa4b24
  Angela Yi authored Oct 03, 2025
```
Signed-off-by: angelayi <yiangela7@gmail.com>
```
  7cfa4b24
- [Misc] Add penalties sampling parameters to serve tool (#25974) · b71fcd49
  Sergei Skvortsov authored Oct 03, 2025
```
Signed-off-by: Sergei Skvortsov <sergeyskv@nebius.com>
Co-authored-by: Sergei Skvortsov <sergeyskv@nebius.com>
```
  b71fcd49
- [Bugfix] Relax tokenizer regex for mixtral to include 'tokenizer.model' (#25964) · 78b8015a
  Bowen Bao authored Oct 03, 2025
```
Signed-off-by: Bowen Bao <bowenbao@amd.com>
```
  78b8015a
- [responsesAPI] add better error messaging for long prompts (#25724) · 831b1241
  Andrew Xia authored Oct 03, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
```
  831b1241
- [Refactor] Optimize FP8 MOE Backend Choice and Log (#26044) · c1ffcb55
  Wentao Ye authored Oct 03, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  c1ffcb55
- [Perf] Remove hardcoded num_warps=1 (#26183) · 0879736a
  Corey Lowman authored Oct 03, 2025
```
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
```
  0879736a
- [Quantization/NVFP4] Speed up TRTLLM NVFP4 MOE weight loading and fix K/V... · a2691733
  Pavani Majety authored Oct 03, 2025
```
[Quantization/NVFP4] Speed up TRTLLM NVFP4 MOE weight loading and fix K/V scale loading for MLA Attn (#25968)
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  a2691733
- Fix V1 engine serialization error with Ray distributed executor (#26148) · cd9e5b83
  Nikhil G authored Oct 03, 2025
```
Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com>
```
  cd9e5b83
- [BugFix][QWEN-VL]fix wrong apply_rotary_emb_torch selection introduced by #24642 (#26123) · dd96465f
  Chendi.Xue authored Oct 03, 2025
```
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
```
  dd96465f
- Fix undefined symbol: cutlass_moe_mm_sm100 (#26098) · 4f8f47e8
  Jun Jiang authored Oct 03, 2025
```
Signed-off-by: Jun Jiang <jasl9187@hotmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  4f8f47e8
- [Renderer] Move Processor out of LLMEngine (#26165) · d78fda7c
  Cyrus Leung authored Oct 03, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d78fda7c
- [Model] Fixed stream generator for gpt-oss + spec-decoding (#26027) · 73a99cc2
  Aleksandr Samarin authored Oct 03, 2025
```
Signed-off-by: Aleksandr Samarin <astrlrd@nebius.com>
```
  73a99cc2
- [Model] Supplement to PR 24862: Pass param prefix to LLMHead (#25805) · cbf92219
  whx authored Oct 03, 2025
```
Signed-off-by: whx-sjtu <2952154980@qq.com>
```
  cbf92219
- [backends][short_conv] CUDA graph piecewise edits (#24215) · 5f42fc53
  Paul Pak authored Oct 03, 2025
```
Signed-off-by: Paul Pak <paulpak58@gmail.com>
```
  5f42fc53
- [Bugfix] Re-enable prefill of max model length (#24446) · 8ee846c2
  Yannick Schnider authored Oct 03, 2025
```
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
```
  8ee846c2
- [Renderer] Move Processor out of AsyncLLM (#24138) · 812b7f54
  Yang Liu authored Oct 03, 2025
```
Signed-off-by: Yang <lymailforjob@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  812b7f54
- Quick fix for IMA with the Prefix Prefill kernel during graph capture (#25983) · 5f2cacdb
  Sage Moore authored Oct 03, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  5f2cacdb
- [Doc] Fixed shape description for fused_batched_moe.py (#25668) · aa5053e3
  Egor authored Oct 03, 2025
```
Signed-off-by: Egor <e.a.krivov@gmail.com>
```
  aa5053e3
- [Multi Modal] Configurable MM Profiling (#25631) · 79aa2446
  Wenlong Wang authored Oct 03, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  79aa2446
- [openai] Fix missing tool usage check (system message) (#24768) · 2ed3f20d
  kyt authored Oct 03, 2025
```
Signed-off-by: kyt <eluban4532@gmail.com>
```
  2ed3f20d
- [NIXL][Misc] Expose metrics from NIXL for logging to CLI (#25388) · 48f30902
  Nicolò Lucchesi authored Oct 03, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  48f30902
- [Model] Use `merge_by_field_config` for MM models (InternVL family) (#26153) · f9a8084e
  Cyrus Leung authored Oct 03, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f9a8084e
- add(v1): RequestStatesStats to RequestOutput (#24947) · 3e70e3d4
  HUIJONG JEONG authored Oct 03, 2025
```
Signed-off-by: huijjj <huijong.jeong@squeezebits.com>
```
  3e70e3d4
- [Input] Remove unused `prompt` field (#26097) · 0ad9951c
  Cyrus Leung authored Oct 03, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0ad9951c
- [Misc] Remove typing.List (#26150) · 8c911718
  Varun Sundar Rabindranath authored Oct 03, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  8c911718
- [BUG] Reorder model config creation (#26124) · c4b48d3c
  ahao-anyscale authored Oct 02, 2025
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
  c4b48d3c
- `FusedMoE` support for the Transformers backend (#22650) · 10d76548
  Harry Mellor authored Oct 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  10d76548
- [Model] Use `merge_by_field_config` for MM models (G) (#26117) · 39b643dc
  Cyrus Leung authored Oct 03, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  39b643dc
- [Bugfix] Fix import `gemm_afp4wfp4` failure on AMD (#26068) · 711f4856
  Zhewen Li authored Oct 02, 2025
```
Signed-off-by: zhewenli <zhewenli@meta.com>
```
  711f4856
- [ROCm] [VL] [Bugfix] Fix vit flash attn dispatcher logic for ROCm (#26104) · 9c5ee91b
  TJian authored Oct 02, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  9c5ee91b
- [gpt-oss] disable tool server initialization if no tool in request (#25790) · e5017cd6
  Andrew Xia authored Oct 02, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>
```
  e5017cd6
- [Bug]: Limit num_reqs in dummy_run when max_num_seqs is small (#26144) · 6a7796e8
  Benjamin Chislett authored Oct 03, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  6a7796e8
- [Attention] Move Backend enum into registry (#25893) · 2aaa4238
  Matthew Bonanni authored Oct 02, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  2aaa4238
- [Bug][Benchmark] Fix duplicate req in oversampling (#26140) · ad2d7880
  Ekagra Ranjan authored Oct 02, 2025
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
```
  ad2d7880
- [Log] Optimize DeepGEMM Missing Log (#26106) · 36ce76c6
  Wentao Ye authored Oct 02, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  36ce76c6
02 Oct, 2025 4 commits

[Bugfix] Disable cascade attention with FlashInfer (#26130) · f1fc2107

Michael Goin authored Oct 02, 2025

Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

f1fc2107

Fix MTP with deepep_low_latency (#25904) · 13cdc021
Matthew Bonanni authored Oct 02, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
13cdc021

[Perf] Fix and reapply move apply w8a8 block fp8 linear to class (#25696) · 502640c3

ElizaWszola authored Oct 02, 2025


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

502640c3

[Mamba][KVCacheManager] Simplify kv cache manage logic for mamba + MTP (#25119) · 3d5f1c86
Chen Zhang authored Oct 02, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
3d5f1c86