Commits · e0be2c4d0905d8ffd08f014421d21678a358b01b · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "bac201c91ff706b041453addfd5e913dbf864644"

25 Jul, 2025 8 commits
- [TPU][Test] Temporarily suspend this MoE model in test_basic.py. (#21560) · e0be2c4d
  QiliangCui authored Jul 24, 2025
```
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
```
  e0be2c4d
- [DP] Support api-server-count > 0 in hybrid DP LB mode (#21510) · 9c8b2c2a
  Nick Hill authored Jul 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  9c8b2c2a
- [Bugfix] DeepGemm utils : Fix hardcoded type-cast (#21517) · 2212cd6c
  Varun Sundar Rabindranath authored Jul 25, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  2212cd6c
- [Kernel] adding fused_moe configs for upcoming granite4 (#21332) · ce3a9b13
  Burkhard Ringlein authored Jul 25, 2025
```
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  ce3a9b13
- Fix GLM-4 PP Missing Layer When using with PP. (#21531) · 2ce90e5b
  Yuxuan Zhang authored Jul 25, 2025
```
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
```
  2ce90e5b
- [Bug] Fix DeepGemm Init Error (#21554) · 633f6e80
  Wentao Ye authored Jul 24, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  633f6e80
- [Docs] Fix `site_url` for RunLLM (#21564) · b57296bb
  Harry Mellor authored Jul 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  b57296bb
- [Frontend] `run-batch` supports V1 (#21541) · 34ddcf9f
  Cyrus Leung authored Jul 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  34ddcf9f
24 Jul, 2025 32 commits
- [MoE] More balanced expert sharding (#21497) · fe56180c
  Woosuk Kwon authored Jul 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  fe56180c
- [TPU][TEST] HF_HUB_DISABLE_XET=1 the test 3. (#21539) · 07d80d7b
  QiliangCui authored Jul 24, 2025
```
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
```
  07d80d7b
- update flashinfer to v0.2.9rc1 (#21485) · 2dd72d23
  weiliang authored Jul 25, 2025
```
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
```
  2dd72d23
- [Docs] Add Expert Parallelism Initial Documentation (#21373) · a6c7fb8c
  Simon Mo authored Jul 24, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6c7fb8c
- [Docs][minor] Fix broken gh-file link in distributed serving docs (#21543) · a7272c23
  Ricardo Decal authored Jul 24, 2025
```
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
```
  a7272c23
- [P/D] Support CPU Transfer in NixlConnector (#18293) · 60662849
  Juncheng Gu authored Jul 24, 2025
```
Signed-off-by: Juncheng Gu <juncgu@gmail.com>
Signed-off-by: Richard Liu <ricliu@google.com>
Co-authored-by: Richard Liu <39319471+richardsliu@users.noreply.github.com>
Co-authored-by: Richard Liu <ricliu@google.com>
```
  60662849
- [P/D] Move FakeNixlWrapper to test dir (#21328) · 1e9ea8e6
  Rui Qiao authored Jul 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  1e9ea8e6
- [XPU] Conditionally import CUDA-specific passes to avoid import errors on xpu platform (#21036) · d9f9a3fd
  Chaojun Zhang authored Jul 24, 2025
```
Signed-off-by: chzhang <chaojun.zhang@intel.com>
```
  d9f9a3fd
- Update flashinfer CUTLASS MoE Kernel (#21408) · 1b25f1fe
  Shu Wang authored Jul 24, 2025
```
Signed-off-by: Shu Wang. <shuw@nvidia.com>
```
  1b25f1fe
- [Bug] Fix Compressed Tensor NVFP4 `cutlass_fp4_group_mm` illegal memory access (#21465) · e8cb0d04
  Wentao Ye authored Jul 24, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  e8cb0d04
- [Docs] Rewrite Distributed Inference and Serving guide (#20593) · 68417411
  Ricardo Decal authored Jul 24, 2025
```
Signed-off-by: Ricardo Decal <rdecal@anyscale.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  68417411
- [Docs] Update Tensorizer usage documentation (#21190) · cdb79ee6
  Sanger Steel authored Jul 24, 2025
```
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
Signed-off-by: William Goldby <willgoldby@gmail.com>
Co-authored-by: William Goldby <willgoldby@gmail.com>
```
  cdb79ee6
- [Fix] Update mamba_ssm to 2.2.5 (#21421) · 5a19a6c6
  elvischenv authored Jul 24, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  5a19a6c6
- [Bugfix] Fix CUDA arch flags for MoE permute (#21426) · 2ded067f
  Ming Yang authored Jul 24, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  2ded067f
- [Model] Officially support Emu3 with Transformers backend (#21319) · 13abd0ea
  Harry Mellor authored Jul 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  13abd0ea
- [Attention] Optimize FlashInfer MetadataBuilder Build call (#21137) · 61b8cea3
  Lucas Wilkinson authored Jul 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  61b8cea3
- bump `flashinfer` to `v0.2.8` (#21385) · 526078a9
  cjackal authored Jul 24, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  526078a9
- [Feat] Allow custom naming of vLLM processes (#21445) · 6da00785
  Chauncey authored Jul 24, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  6da00785
- [Misc] Improve comment for DPEngineCoreActor._set_cuda_visible_devices() (#21501) · 73e3949d
  Rui Qiao authored Jul 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  73e3949d
- Replace `--expand-tools-even-if-tool-choice-none` with... · 6eca337c
  Shintarou Okada authored Jul 24, 2025
```
Replace `--expand-tools-even-if-tool-choice-none` with `--exclude-tools-when-tool-choice-none` for v0.10.0 (#20544)
Signed-off-by: okada <kokuzen@gmail.com>
Signed-off-by: okada shintarou <okada@preferred.jp>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6eca337c
- remove GLM-4.5 quantization wrong Code (#21435) · 85bda9e7
  Yuxuan Zhang authored Jul 24, 2025
  
  85bda9e7
- [Core] Support model loader plugins (#21067) · 610852a4
  22quinn authored Jul 24, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  610852a4
- [Misc] Fix duplicate FusedMoEConfig debug messages (#21455) · f0f4de8f
  Nick Hill authored Jul 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  f0f4de8f
- [v1][Core] Clean up usages of `SpecializedManager` (#21407) · fc5f756d
  Zhou Fang authored Jul 24, 2025
```
Signed-off-by: Zhou Fang <fang.github@gmail.com>
```
  fc5f756d
- [TPU][Bugfix] fix moe layer (#21340) · e74bfc70
  Chengji Yao authored Jul 24, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  e74bfc70
- [Bugfix][ROCm] Fix for warp_size uses on host (#21205) · 90eeea8f
  Gregory Shtrasberg authored Jul 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  90eeea8f
- Deduplicate Transformers backend code using inheritance (#21461) · dde295a9
  Harry Mellor authored Jul 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  dde295a9
- Add think chunk (#21333) · 6d8d0a24
  Julien Denize authored Jul 24, 2025
```
Signed-off-by: Julien Denize <julien.denize@mistral.ai>
```
  6d8d0a24
- [BugFix] Set CUDA_VISIBLE_DEVICES before spawning the subprocesses (#21211) · 11ef7a61
  Yinghai Lu authored Jul 23, 2025
```
Signed-off-by: Yinghai Lu <yinghai@thinkingmachines.ai>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
```
  11ef7a61
- Dump input metadata on crash for async scheduling (#21258) · dc2f159f
  Woosuk Kwon authored Jul 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  dc2f159f
- [DP] Internal Load Balancing Per Node [`one-pod-per-node`] (#21238) · d5b981f8
  Robert Shaw authored Jul 23, 2025
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  d5b981f8
- [BugFix] Fix KVConnector TP worker aggregation (#21473) · eec69420
  Nick Hill authored Jul 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  eec69420