Commits · 612c2edb4f940b26550ca64576995957e4118a12 · OpenDAS / vllm_cscc · GitLab

14 May, 2025 19 commits
- [FEAT] [ROCm]: Add AITER CK 2 Stages MoE support (#17110) · 612c2edb
  TJian authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  612c2edb
- [Bugfix] Fix QKVCrossParallelLinear::sync_weight_attrs for PyTorch compile (#17844) · 38fe728d
  Andrzej Kotłowski authored May 14, 2025
```
Signed-off-by: Andrzej Kotłowski <akotlowski@habana.ai>
```
  38fe728d
- [Misc] replace does not exist model (#18119) · 82e7f9bb
  rongfu.leng authored May 14, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  82e7f9bb
- [Model] Add packed_modules_mapping for Qwen3-MOE (#18118) · 63dc3426
  Jee Jee Li authored May 14, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  63dc3426
- [New Model]: support GTE NewModel (#17986) · 63ad6222
  wang.yuqi authored May 14, 2025
  
  63ad6222
- [Fix] Move "model_config" as keyword args in chat_utils.py (#18098) · 6685890d
  lkchen authored May 13, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  6685890d
- [BugFix][AMD] Compatible patch for AITER lib after 04/20 (#17912) · 4f8b3732
  qli88 authored May 14, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  4f8b3732
- [AMD][torch.compile] Enable silu+fp8_quant fusion for rocm (#18082) · 7b2f28de
  Charlie Fu authored May 14, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  7b2f28de
- [FEAT] [ROCm] [V1]: Add AITER biased group topk for DeepSeekV3 (#17955) · 2d912fb6
  vllmellm authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  2d912fb6
- [Bugfix][V1] Fix FlashInfer V1 backend using the wrong VllmConfig (#18086) · 12e6c0b4
  Michael Goin authored May 13, 2025
  
  12e6c0b4
- [Bugfix] Fix FP8 Marlin MoE and enable for compressed-tensors models (#18026) · 9a2a6357
  Michael Goin authored May 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  9a2a6357
- [core][distributed] add ep group and all2all interface (#18077) · 6266c57b
  youkaichao authored May 14, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6266c57b
- [Bug]: Fix S3 model/tokenizer path resolution (#18083) · 754b699c
  Jon Gill authored May 13, 2025
```
Signed-off-by: Jon Gill <jon@yurts.ai>
```
  754b699c
- [Misc] Remove unused numpy tensor (#18084) · 6e27c6d8
  Roger Wang authored May 13, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
```
  6e27c6d8
- [P/D] Add some more debug logs to `NixlConnector` (#18102) · d5af47a1
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  d5af47a1
- [Hardware/NVIDIA/Modelopt] Fix modelopt forward method for v1 torch.compile (#18101) · 65f0f74b
  Pavani Majety authored May 13, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  65f0f74b
- [Fix] Support CUDAGraph capture for encoder-decoder on ROCm (#18104) · 176a95c6
  Luka Govedič authored May 13, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
```
  176a95c6
- [v1][KVCacheManager] pass num_new_computed_tokens to kv cache manager (#18001) · f2ae883b
  Chen Zhang authored May 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f2ae883b
- [FEAT] [ROCm]: Add AITER Block-Scaled GEMM Feature (#14968) · 40de1ef4
  vllmellm authored May 14, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  40de1ef4
13 May, 2025 20 commits
- [V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) · 55aa7af9
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  55aa7af9
- Update deprecated type hinting in `vllm/adapter_commons` (#18073) · 0b217da6
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0b217da6
- Update deprecated type hinting in `vllm/compilation` (#18072) · 19324d66
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  19324d66
- [Bugfix] Fix entrypoints metrics tests (#18063) · b922c2eb
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b922c2eb
- Update deprecated type hinting in `vllm/transformers_utils` (#18058) · 8c946cec
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8c946cec
- Update deprecated type hinting in `vllm/profiler` (#18057) · ff334ca1
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  ff334ca1
- Update deprecated type hinting in `model_executor/layers` (#18056) · 6223dd81
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6223dd81
- [Fix] check to make sure processor has chat templates (#18047) · cb528d05
  Aaron Pham authored May 13, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  cb528d05
- [Bugfix] Fix marlin moe fallback logic for llama4 (#18042) · ea6ae8cb
  Michael Goin authored May 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  ea6ae8cb
- [BugFix] Set default random seed to 0 for V1 (#17929) · 2ff297dc
  Woosuk Kwon authored May 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2ff297dc
- [Bugfix][V1] Only get input embeddings w/ multi-modal models if first PP (#17916) · 8dd0671b
  Jin Huang authored May 13, 2025
```
Signed-off-by: Jin Huang <jinhun@amazon.com>
Co-authored-by: Jin Huang <jinhun@amazon.com>
```
  8dd0671b
- [v1][KVCacheManager] Avoid full cache hit by controlling max_length (#17999) · f0d610a8
  Chen Zhang authored May 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f0d610a8
- cleanup invalid prints (#18050) · 48545728
  Calvin Chen authored May 13, 2025
```
Signed-off-by: calvin chen <120380290@qq.com>
```
  48545728
- [Feature][V1] Support `tool_choice: required` when using Xgrammar as the... · dc1a8217
  Chauncey authored May 13, 2025
```
[Feature][V1]  Support `tool_choice: required` when using Xgrammar as the `StructuredOutputBackend`. (#17845)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  dc1a8217
- [Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) · 61e0a506
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61e0a506
- [Bugfix] Fixes for new marlin moe usage (#18017) · 1df491c5
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  1df491c5
- [Misc] Slight spelling modification (#18039) · c06af9a9
  Jee Jee Li authored May 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c06af9a9
- Implements dual-chunk-flash-attn backend for dual chunk attention with sparse... · 60f76243
  Tao He authored May 13, 2025
```
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844)
```
  60f76243
- Remove noisy warnings from `SchedulerConfig` (#17995) · d67085c2
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d67085c2
- Use NVFP4 Marlin for CompressedTensorsW4A16Fp4 (#18000) · 307939f2
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  307939f2
12 May, 2025 1 commit
- Update some more deprecated type hinting (#17998) · 9d7ea9db
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  9d7ea9db