Commits · fe56180c7f9088361db86c1ef40dbc54fa931e1f · OpenDAS / vllm_cscc · GitLab

24 Jul, 2025 40 commits
- [MoE] More balanced expert sharding (#21497) · fe56180c
  Woosuk Kwon authored Jul 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  fe56180c
- [TPU][TEST] HF_HUB_DISABLE_XET=1 the test 3. (#21539) · 07d80d7b
  QiliangCui authored Jul 24, 2025
```
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
```
  07d80d7b
- update flashinfer to v0.2.9rc1 (#21485) · 2dd72d23
  weiliang authored Jul 25, 2025
```
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
```
  2dd72d23
- [Docs] Add Expert Parallelism Initial Documentation (#21373) · a6c7fb8c
  Simon Mo authored Jul 24, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6c7fb8c
- [Docs][minor] Fix broken gh-file link in distributed serving docs (#21543) · a7272c23
  Ricardo Decal authored Jul 24, 2025
```
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
```
  a7272c23
- [P/D] Support CPU Transfer in NixlConnector (#18293) · 60662849
  Juncheng Gu authored Jul 24, 2025
```
Signed-off-by: Juncheng Gu <juncgu@gmail.com>
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Richard Liu <39319471+richardsliu@users.noreply.github.com>
Co-authored-by: Richard Liu <ricliu@google.com>
```
  60662849
- [P/D] Move FakeNixlWrapper to test dir (#21328) · 1e9ea8e6
  Rui Qiao authored Jul 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  1e9ea8e6
- [XPU] Conditionally import CUDA-specific passes to avoid import errors on xpu platform (#21036) · d9f9a3fd
  Chaojun Zhang authored Jul 24, 2025
```
Signed-off-by: chzhang <chaojun.zhang@intel.com>
```
  d9f9a3fd
- Update flashinfer CUTLASS MoE Kernel (#21408) · 1b25f1fe
  Shu Wang authored Jul 24, 2025
```
Signed-off-by: Shu Wang. <shuw@nvidia.com>
```
  1b25f1fe
- [Bug] Fix Compressed Tensor NVFP4 `cutlass_fp4_group_mm` illegal memory access (#21465) · e8cb0d04
  Wentao Ye authored Jul 24, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  e8cb0d04
- [Docs] Rewrite Distributed Inference and Serving guide (#20593) · 68417411
  Ricardo Decal authored Jul 24, 2025
```
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  68417411
- [Docs] Update Tensorizer usage documentation (#21190) · cdb79ee6
  Sanger Steel authored Jul 24, 2025
```
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
Signed-off-by: William Goldby <willgoldby@gmail.com>
Co-authored-by: William Goldby <willgoldby@gmail.com>
```
  cdb79ee6
- [Fix] Update mamba_ssm to 2.2.5 (#21421) · 5a19a6c6
  elvischenv authored Jul 24, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  5a19a6c6
- [Bugfix] Fix CUDA arch flags for MoE permute (#21426) · 2ded067f
  Ming Yang authored Jul 24, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  2ded067f
- [Model] Officially support Emu3 with Transformers backend (#21319) · 13abd0ea
  Harry Mellor authored Jul 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  13abd0ea
- [Attention] Optimize FlashInfer MetadataBuilder Build call (#21137) · 61b8cea3
  Lucas Wilkinson authored Jul 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  61b8cea3
- bump `flashinfer` to `v0.2.8` (#21385) · 526078a9
  cjackal authored Jul 24, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  526078a9
- [Feat] Allow custom naming of vLLM processes (#21445) · 6da00785
  Chauncey authored Jul 24, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  6da00785
- [Misc] Improve comment for DPEngineCoreActor._set_cuda_visible_devices() (#21501) · 73e3949d
  Rui Qiao authored Jul 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  73e3949d
- Replace `--expand-tools-even-if-tool-choice-none` with... · 6eca337c
  Shintarou Okada authored Jul 24, 2025
```
Replace `--expand-tools-even-if-tool-choice-none` with `--exclude-tools-when-tool-choice-none` for v0.10.0 (#20544)
Signed-off-by: okada <kokuzen@gmail.com>
Signed-off-by: okada shintarou <okada@preferred.jp>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6eca337c
- remove GLM-4.5 quantization wrong Code (#21435) · 85bda9e7
  Yuxuan Zhang authored Jul 24, 2025
  
  85bda9e7
- [Core] Support model loader plugins (#21067) · 610852a4
  22quinn authored Jul 24, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  610852a4
- [Misc] Fix duplicate FusedMoEConfig debug messages (#21455) · f0f4de8f
  Nick Hill authored Jul 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  f0f4de8f
- [v1][Core] Clean up usages of `SpecializedManager` (#21407) · fc5f756d
  Zhou Fang authored Jul 24, 2025
```
Signed-off-by: Zhou Fang <fang.github@gmail.com>
```
  fc5f756d
- [TPU][Bugfix] fix moe layer (#21340) · e74bfc70
  Chengji Yao authored Jul 24, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  e74bfc70
- [Bugfix][ROCm] Fix for warp_size uses on host (#21205) · 90eeea8f
  Gregory Shtrasberg authored Jul 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  90eeea8f
- Deduplicate Transformers backend code using inheritance (#21461) · dde295a9
  Harry Mellor authored Jul 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  dde295a9
- Add think chunk (#21333) · 6d8d0a24
  Julien Denize authored Jul 24, 2025
```
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
```
  6d8d0a24
- [BugFix] Set CUDA_VISIBLE_DEVICES before spawning the subprocesses (#21211) · 11ef7a61
  Yinghai Lu authored Jul 23, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
```
  11ef7a61
- Dump input metadata on crash for async scheduling (#21258) · dc2f159f
  Woosuk Kwon authored Jul 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  dc2f159f
- [DP] Internal Load Balancing Per Node [`one-pod-per-node`] (#21238) · d5b981f8
  Robert Shaw authored Jul 23, 2025
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  d5b981f8
- [BugFix] Fix KVConnector TP worker aggregation (#21473) · eec69420
  Nick Hill authored Jul 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  eec69420
- [BugFix]: Batch generation from prompt_embeds fails for long prompts (#21390) · fd48d99f
  KazusatoOoko authored Jul 23, 2025
```
Signed-off-by: KazusatoOko <kazusto.oko@sakana.ai>
Co-authored-by: KazusatoOko <kazusto.oko@sakana.ai>
```
  fd48d99f
- [Bugfix] Fix example disagg_example_p2p_nccl_xpyd.sh zombie process (#21437) · f8c15c4e
  WeiQing Chen authored Jul 24, 2025
```
Signed-off-by: David Chen <530634352@qq.com>
```
  f8c15c4e
- [Bugfix] Fix casing warning (#21468) · aa08a954
  Matthew Bonanni authored Jul 23, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  aa08a954
- [XPU][UT] increase intel xpu CI test scope (#21492) · 13e4ee1d
  Liangliang Ma authored Jul 24, 2025
```
Signed-off-by: Ma, Liangliang <liangliang.ma@intel.com>
```
  13e4ee1d
- [Misc] Add dummy maverick test to CI (#21324) · 772ce5af
  Ming Yang authored Jul 23, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  772ce5af
- [Frontend] Set MAX_AUDIO_CLIP_FILESIZE_MB via env var instead of hardcoding (#21374) · 63d92abb
  deven-labovitch authored Jul 23, 2025
```
Signed-off-by: Deven Labovitch <deven@videa.ai>
```
  63d92abb
- feat(gguf_loader): accept HF repo paths & URLs for GGUF (#20793) · 11599b0e
  Hardik Gupta authored Jul 23, 2025
```
Signed-off-by: Hardik <hardikgupta1999@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  11599b0e
- [Core] Freeze gc during cuda graph capture to speed up init (#21146) · f3137cdd
  Michael Goin authored Jul 23, 2025
```
Signed-off-by: Codex <codex@openai.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f3137cdd