Commits · 2d912fb66fedf0c07bae06fb905d458b1d9a907a · OpenDAS / vllm_cscc · GitLab

14 May, 2025 11 commits
- [FEAT] [ROCm] [V1]: Add AITER biased group topk for DeepSeekV3 (#17955) · 2d912fb6
  vllmellm authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  2d912fb6
- [Bugfix][V1] Fix FlashInfer V1 backend using the wrong VllmConfig (#18086) · 12e6c0b4
  Michael Goin authored May 13, 2025
  
  12e6c0b4
- [Bugfix] Fix FP8 Marlin MoE and enable for compressed-tensors models (#18026) · 9a2a6357
  Michael Goin authored May 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  9a2a6357
- [core][distributed] add ep group and all2all interface (#18077) · 6266c57b
  youkaichao authored May 14, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6266c57b
- [Bug]: Fix S3 model/tokenizer path resolution (#18083) · 754b699c
  Jon Gill authored May 13, 2025
```
Signed-off-by: Jon Gill <jon@yurts.ai>
```
  754b699c
- [Misc] Remove unused numpy tensor (#18084) · 6e27c6d8
  Roger Wang authored May 13, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
```
  6e27c6d8
- [P/D] Add some more debug logs to `NixlConnector` (#18102) · d5af47a1
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d5af47a1
- [Hardware/NVIDIA/Modelopt] Fix modelopt forward method for v1 torch.compile (#18101) · 65f0f74b
  Pavani Majety authored May 13, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  65f0f74b
- [Fix] Support CUDAGraph capture for encoder-decoder on ROCm (#18104) · 176a95c6
  Luka Govedič authored May 13, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
```
  176a95c6
- [v1][KVCacheManager] pass num_new_computed_tokens to kv cache manager (#18001) · f2ae883b
  Chen Zhang authored May 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f2ae883b
- [FEAT] [ROCm]: Add AITER Block-Scaled GEMM Feature (#14968) · 40de1ef4
  vllmellm authored May 14, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  40de1ef4
13 May, 2025 29 commits
- [Docs] Expand security doc with firewall info (#18081) · 0189a65a
  Russell Bryant authored May 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  0189a65a
- [V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) · 55aa7af9
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  55aa7af9
- Update deprecated type hinting in `vllm/adapter_commons` (#18073) · 0b217da6
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0b217da6
- Update deprecated type hinting in `vllm/compilation` (#18072) · 19324d66
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  19324d66
- Give auto-merge label workflow permission to add labels to issues (#18078) · fc407a14
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  fc407a14
- Convert `benchmarks` to `ruff format` (#18068) · 009d9e75
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  009d9e75
- [Bugfix] Fix entrypoints metrics tests (#18063) · b922c2eb
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b922c2eb
- [CI] set token permissions for pre-commit CI job (#17729) · 00b14e0f
  Russell Bryant authored May 13, 2025
```
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
```
  00b14e0f
- [CI] Add token permissions for add-ready-label CI job (#17730) · 54e467e6
  Russell Bryant authored May 13, 2025
```
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
```
  54e467e6
- [CI] Add workflow permissions for helm CI job (#17727) · 79a1d25b
  Russell Bryant authored May 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
```
  79a1d25b
- [CI] Set token permissions for reminder comment CI job (#17728) · 9944011b
  Russell Bryant authored May 13, 2025
```
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
```
  9944011b
- Update deprecated type hinting in `vllm/transformers_utils` (#18058) · 8c946cec
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8c946cec
- Update deprecated type hinting in `vllm/profiler` (#18057) · ff334ca1
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  ff334ca1
- Update deprecated type hinting in `model_executor/layers` (#18056) · 6223dd81
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6223dd81
- [doc] add download/list/delete HF model CLI usage (#17940) · 906f0598
  Reid authored May 13, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  906f0598
- [Fix] check to make sure processor has chat templates (#18047) · cb528d05
  Aaron Pham authored May 13, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  cb528d05
- Convert `.buildkite` to `ruff format` (#17656) · 98fcba15
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  98fcba15
- [Benchmarks] Refactor run_structured_output_benchmarks.sh (#17722) · 23b3134e
  Russell Bryant authored May 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  23b3134e
- [Bugfix] Fix marlin moe fallback logic for llama4 (#18042) · ea6ae8cb
  Michael Goin authored May 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  ea6ae8cb
- [BugFix] Set default random seed to 0 for V1 (#17929) · 2ff297dc
  Woosuk Kwon authored May 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2ff297dc
- [Bugfix][V1] Only get input embeddings w/ multi-modal models if first PP (#17916) · 8dd0671b
  Jin Huang authored May 13, 2025
```
Signed-off-by: Jin Huang <jinhun@amazon.com>
Co-authored-by: Jin Huang <jinhun@amazon.com>
```
  8dd0671b
- [v1][KVCacheManager] Avoid full cache hit by controlling max_length (#17999) · f0d610a8
  Chen Zhang authored May 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f0d610a8
- Fix Broken macro for cutlass moe (#18049) · e57e4d6e
  Driss Guessous authored May 12, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  e57e4d6e
- [BugFix] Fix 4-GPU RLHF tests (#18007) · ee5be834
  Nick Hill authored May 12, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ee5be834
- cleanup invalid prints (#18050) · 48545728
  Calvin Chen authored May 13, 2025
```
Signed-off-by: calvin chen <120380290@qq.com>
```
  48545728
- [Feature][V1] Support `tool_choice: required` when using Xgrammar as the... · dc1a8217
  Chauncey authored May 13, 2025
```
[Feature][V1]  Support `tool_choice: required` when using Xgrammar as the `StructuredOutputBackend`. (#17845)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  dc1a8217
- [Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) · 61e0a506
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61e0a506
- [Bugfix] Fixes for new marlin moe usage (#18017) · 1df491c5
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  1df491c5
- [ROCm]: Fix build from source failure with gcc14 and ROCm 6.3 (#13779) · d8487ef5
  Arjun Kathuria authored May 13, 2025
```
Signed-off-by: Arjun Kathuria <arjun.kathuria8@gmail.com>
```
  d8487ef5