Commits · 5e25b122366a639f6da9fa568b532d6932c45aff · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "8516999495114926c9838c2d6e0feb580d4d983f"

23 Sep, 2025 40 commits
- [Kernel] [Mamba] Remove BLOCK_H=1 from list of tuneable configurations for... · 5e25b122
  Thomas Parnell authored Sep 24, 2025
```
[Kernel] [Mamba] Remove BLOCK_H=1 from list of tuneable configurations for `_chunk_cumsum_fwd_kernel` (#25197)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh-Yang <chih.chieh.yang@ibm.com>
```
  5e25b122
- Add `VLLM_NVTX_SCOPES_FOR_PROFILING=1` to enable `nvtx.annotate` scopes (#25501) · c85d75cf
  Corey Lowman authored Sep 23, 2025
```
Signed-off-by: Corey Lowman <clowman1993@gmail.com>
```
  c85d75cf
- [BugFix] Fix OOM in vLLM replicas by ensuring consistent NCCL memory accounting (#25359) · abad204b
  kourosh hakhamaneshi authored Sep 23, 2025
```
Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>
```
  abad204b
- Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) · 7361ab37
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  7361ab37
- [gpt-oss][bugfix] remove logic to require resp_ in ResponseAPI (#25428) · 95bc60e4
  Andrew Xia authored Sep 23, 2025
```
Signed-off-by: Andrew Xia <axia@meta.com>
```
  95bc60e4
- Fix triton_reshape_and_cache_flash.py triton import (#25522) · 4f2954f7
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4f2954f7
- Add VLLM_ENABLE_INDUCTOR_MAX_AUTOTUNE & VLLM_ENABLE_INDUCTOR_COORDINA… (#25493) · eca7be90
  rouchenzi authored Sep 23, 2025
```
Signed-off-by: rouchenzi <ruochenwen@gmail.com>
Signed-off-by: rouchenzi <40842833+rouchenzi@users.noreply.github.com>
```
  eca7be90
- [V0 Deprecation] Remove placeholder attn (#25510) · 969b4da3
  Thomas Parnell authored Sep 24, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  969b4da3
- [Core] Use KVCacheBlock as much as possible instead of dict[block_id, KVCacheBlock] (#24830) · 4f8c4b89
  Jialin Ouyang authored Sep 23, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  4f8c4b89
- [CI/Build] Fix and re-enable v1 PP test on CI (#25496) · ae002924
  Isotr0py authored Sep 24, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  ae002924
- [Bugfix] Fix for the import error from #24588 (#25481) · 690f948e
  Gregory Shtrasberg authored Sep 23, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  690f948e
- [Build] Update Xgrammar to 0.1.25 (#25467) · 08275ec0
  Chauncey authored Sep 24, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  08275ec0
- [Bugfix] gpt-oss container tool output bug (#25485) · c828d1bf
  Alec S authored Sep 23, 2025
```
Signed-off-by: Alec Solder <alecs@fb.com>
Co-authored-by: Alec Solder <alecs@fb.com>
```
  c828d1bf
- [CI] Fix Pre-commit Issue (#25497) · 8b8a8afc
  Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  8b8a8afc
- Enable symmetric memory all reduce by default only enabling for TP (#25070) · 8bdd8b5c
  Ilya Markov authored Sep 23, 2025
```
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  8bdd8b5c
- [Bugfix] Lower gpt-oss max cudagraph size to 992 to be compatible with FA3 (#25508) · a8ffc4f0
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a8ffc4f0
- [Speculators][Speculative Decoding] Fix gpt-oss eagle3 accuracy issue (#25406) · d5944d51
  jiahanc authored Sep 23, 2025
```
Signed-off-by: jiahanc <173873397+jiahanc@users.noreply.github.com>
```
  d5944d51
- [Perf] Change default CUDAGraphMode from PIECEWISE to FULL_AND_PIECEWISE (#25444) · 24fab45d
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  24fab45d
- [Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) · 63400259
  ElizaWszola authored Sep 23, 2025
```
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
```
  63400259
- [core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3
  Amir Samani authored Sep 23, 2025
```
Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  8c1c81a3
- [ROCm] Add skinny gemm bias support for dtypes fp16,bf16,fp8 (#24988) · a3a78280
  Hashem Hashemi authored Sep 23, 2025
```
Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com>
Signed-off-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com>
```
  a3a78280
- [Core] Ensure LoRA linear respect the base_layer's tp_size and tp_rank (#25487) · 5abb1179
  Jee Jee Li authored Sep 24, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  5abb1179
- [Spec Decode][CI] Add e2e test for `examples/spec_decode.py` and prevent... · 867ecdd1
  Ekagra Ranjan authored Sep 23, 2025
```
[Spec Decode][CI] Add e2e test for `examples/spec_decode.py` and prevent breaking Acceptance Length (#24531)
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
```
  867ecdd1
- [Misc] Reduce initialization time of auto_tune (#23682) · 24e82227
  Weida Hong authored Sep 24, 2025
```
Signed-off-by: Weida Hong <wdhongtw@google.com>
```
  24e82227
- [V1][Kernel] Add triton implementation for `reshape_and_cache_flash` (#24503) · 100b630a
  Burkhard Ringlein authored Sep 23, 2025
```
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  100b630a
- Use macro guard CUDA functions for back compatibility in grouped_topk_kernel.cu (#25346) · 527821d1
  Ming Yang authored Sep 23, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Rahul Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  527821d1
- [Log] Optimize kv cache memory log from Bytes to GiB (#25204) · 846197f5
  Wentao Ye authored Sep 23, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  846197f5
- [BugFix] Fix UB in per_token_group_quant.cu (#24913) · 2357480b
  rivos-shreeasish authored Sep 23, 2025
```
Signed-off-by: Shreeasish Kumar <shreeasish@rivosinc.com>
```
  2357480b
- [Kernels] Support blocked fp8 quantization for compressed tensors MoE (#25219) · f11e3c51
  bnellnm authored Sep 23, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  f11e3c51
- Add backward compatibility for `GuidedDecodingParams` (#25422) · 875d6def
  Harry Mellor authored Sep 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  875d6def
- [Core/DBO][2/N] Dual-Batch Overlap add DeepEP High Throughput support and Prefill support (#24845) · cc1dc7ed
  Lucas Wilkinson authored Sep 23, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc1dc7ed
- [V1] Remove V0 code paths for Hybrid models (#25400) · a903669e
  Thomas Parnell authored Sep 23, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  a903669e
- [UX] Change kv-cache-memory log level to debug (#25479) · 2c58742d
  Michael Goin authored Sep 23, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  2c58742d
- [XPU] Fix MOE DP accuracy issue on XPU (#25465) · 4c966e44
  Fanli Lin authored Sep 23, 2025
  
  4c966e44
- [Docs] NixlConnector quickstart guide (#24249) · da5e7e43
  Peter Pan authored Sep 23, 2025
```
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
Signed-off-by: Peter Pan <peter.pan@daocloud.io>
Signed-off-by: Nicolò Lucchesi<nicolo.lucchesi@gmail.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
```
  da5e7e43
- [P/D] Support NIXL connector to disconnect during a clean shutdown (#24423) · f05a4f0e
  Chauncey authored Sep 23, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
```
  f05a4f0e
- [BugFix] Register expert_map as named buffer for wake_up and sleep (#25458) · 61d1b355
  Joel authored Sep 23, 2025
```
Signed-off-by: wuxibin <wuxibin@bytedance.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  61d1b355
- [CI/Build] Fix disabled v1 attention backend selection test (#25471) · b6a136b5
  Isotr0py authored Sep 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  b6a136b5
- [docs] Benchmark Serving Incorrect Arg (#25474) · 0d9fe260
  vllmellm authored Sep 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  0d9fe260
- [Core] Optimize LoRA weight loading (#25403) · 273690a5
  Jee Jee Li authored Sep 23, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  273690a5