Commits · 54e467e6f8099a0d417c6880b43d188c8a49ee96 · OpenDAS / vllm_cscc · GitLab

13 May, 2025 26 commits
- [CI] Add token permissions for add-ready-label CI job (#17730) · 54e467e6
  Russell Bryant authored May 13, 2025
```
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
```
  54e467e6
- [CI] Add workflow permissions for helm CI job (#17727) · 79a1d25b
  Russell Bryant authored May 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
```
  79a1d25b
- [CI] Set token permissions for reminder comment CI job (#17728) · 9944011b
  Russell Bryant authored May 13, 2025
```
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
```
  9944011b
- Update deprecated type hinting in `vllm/transformers_utils` (#18058) · 8c946cec
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8c946cec
- Update deprecated type hinting in `vllm/profiler` (#18057) · ff334ca1
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  ff334ca1
- Update deprecated type hinting in `model_executor/layers` (#18056) · 6223dd81
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6223dd81
- [doc] add download/list/delete HF model CLI usage (#17940) · 906f0598
  Reid authored May 13, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  906f0598
- [Fix] check to make sure processor has chat templates (#18047) · cb528d05
  Aaron Pham authored May 13, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  cb528d05
- Convert `.buildkite` to `ruff format` (#17656) · 98fcba15
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  98fcba15
- [Benchmarks] Refactor run_structured_output_benchmarks.sh (#17722) · 23b3134e
  Russell Bryant authored May 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  23b3134e
- [Bugfix] Fix marlin moe fallback logic for llama4 (#18042) · ea6ae8cb
  Michael Goin authored May 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  ea6ae8cb
- [BugFix] Set default random seed to 0 for V1 (#17929) · 2ff297dc
  Woosuk Kwon authored May 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2ff297dc
- [Bugfix][V1] Only get input embeddings w/ multi-modal models if first PP (#17916) · 8dd0671b
  Jin Huang authored May 13, 2025
```
Signed-off-by: Jin Huang <jinhun@amazon.com>
Co-authored-by: Jin Huang <jinhun@amazon.com>
```
  8dd0671b
- [v1][KVCacheManager] Avoid full cache hit by controlling max_length (#17999) · f0d610a8
  Chen Zhang authored May 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f0d610a8
- Fix Broken macro for cutlass moe (#18049) · e57e4d6e
  Driss Guessous authored May 12, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  e57e4d6e
- [BugFix] Fix 4-GPU RLHF tests (#18007) · ee5be834
  Nick Hill authored May 12, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ee5be834
- cleanup invalid prints (#18050) · 48545728
  Calvin Chen authored May 13, 2025
```
Signed-off-by: calvin chen <120380290@qq.com>
```
  48545728
- [Feature][V1] Support `tool_choice: required` when using Xgrammar as the... · dc1a8217
  Chauncey authored May 13, 2025
```
[Feature][V1]  Support `tool_choice: required` when using Xgrammar as the `StructuredOutputBackend`. (#17845)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  dc1a8217
- [Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) · 61e0a506
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61e0a506
- [Bugfix] Fixes for new marlin moe usage (#18017) · 1df491c5
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  1df491c5
- [ROCm]: Fix build from source failure with gcc14 and ROCm 6.3 (#13779) · d8487ef5
  Arjun Kathuria authored May 13, 2025
```
Signed-off-by: Arjun Kathuria <arjun.kathuria8@gmail.com>
```
  d8487ef5
- [Misc] Slight spelling modification (#18039) · c06af9a9
  Jee Jee Li authored May 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c06af9a9
- Implements dual-chunk-flash-attn backend for dual chunk attention with sparse... · 60f76243
  Tao He authored May 13, 2025
```
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844)
```
  60f76243
- [ROCm] Skip tests for quantizations incompatible with ROCm (#17905) · f6518b2b
  hissu-hyvarinen authored May 13, 2025
```
Signed-off-by: Hissu Hyvarinen <hissu.hyvarinen@amd.com>
```
  f6518b2b
- Remove noisy warnings from `SchedulerConfig` (#17995) · d67085c2
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d67085c2
- Use NVFP4 Marlin for CompressedTensorsW4A16Fp4 (#18000) · 307939f2
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  307939f2
12 May, 2025 14 commits
- Update some more deprecated type hinting (#17998) · 9d7ea9db
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  9d7ea9db
- [Model] Support MiMo-7B inference with MTP (#17433) · acee8f48
  bwshen-mi authored May 13, 2025
```
Signed-off-by: wp-alpha <wangpeng66@xiaomi.com>
Co-authored-by: wangpeng66 <wangpeng66@xiaomi.com>
```
  acee8f48
- Fix FBGEMM integration (#18002) · f065de4e
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f065de4e
- [V1][Spec Decode] Eagle unit tests (#17350) · dc990536
  wwl2755 authored May 12, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  dc990536
- [CI] Make JSON output tests less likely to fail (#17859) · ebab1ac3
  Russell Bryant authored May 12, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ebab1ac3
- Enable standard language model for torhc nightly (#18004) · 2b0db9b0
  Yang Wang authored May 12, 2025
```
Signed-off-by: Yang Wang <elainewy@meta.com>
```
  2b0db9b0
- [Chore] Remove unused method (#18024) · 195adb47
  Robert Shaw authored May 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  195adb47
- [v1][KVCacheManager] Change prefix caching metric from counting blocks to counting tokens (#18003) · 302f3aca
  Chen Zhang authored May 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  302f3aca
- Enabling "Weight Loading Multiple GPU Test - Large Models" (#18020) · e9c730c9
  Alexei-V-Ivanov-AMD authored May 12, 2025
  
  e9c730c9
- [Core] Use platform-agnostic device control for DP engine core (#17245) · 289199fe
  Jade Zheng authored May 13, 2025
```
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
```
  289199fe
- [CI/Build] Fix TPU V1 Test mixed use of & and && across tests (#17968) · b9fd0d7a
  Carol Zheng authored May 12, 2025
  
  b9fd0d7a
- Construct `KVTransferConfig` properly from Python instead of using JSON blobs without CLI (#17994) · 72a3f6b8
  Harry Mellor authored May 12, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  72a3f6b8
- [Lora][Frontend]Add default local directory LoRA resolver plugin. (#16855) · 98ea3560
  Jonathan Berkhahn authored May 12, 2025
```
Signed-off-by: jberkhahn <jaberkha@us.ibm.com>
```
  98ea3560
- [P/D] NIXL Integration (#17751) · d1911020
  Robert Shaw authored May 12, 2025
```
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>
```
  d1911020