Commits · 55aa7af9947244bdc427bfae01e750b1c157bd8f · OpenDAS / vllm_cscc

"docs/vscode:/vscode.git/clone" did not exist on "6a50eaa0d3858dd19a142746fce7181813c5d632"

13 May, 2025 20 commits
- [V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) · 55aa7af9
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  55aa7af9
- Update deprecated type hinting in `vllm/adapter_commons` (#18073) · 0b217da6
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0b217da6
- Update deprecated type hinting in `vllm/compilation` (#18072) · 19324d66
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  19324d66
- [Bugfix] Fix entrypoints metrics tests (#18063) · b922c2eb
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b922c2eb
- Update deprecated type hinting in `vllm/transformers_utils` (#18058) · 8c946cec
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8c946cec
- Update deprecated type hinting in `vllm/profiler` (#18057) · ff334ca1
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  ff334ca1
- Update deprecated type hinting in `model_executor/layers` (#18056) · 6223dd81
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6223dd81
- [Fix] check to make sure processor has chat templates (#18047) · cb528d05
  Aaron Pham authored May 13, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  cb528d05
- [Bugfix] Fix marlin moe fallback logic for llama4 (#18042) · ea6ae8cb
  Michael Goin authored May 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  ea6ae8cb
- [BugFix] Set default random seed to 0 for V1 (#17929) · 2ff297dc
  Woosuk Kwon authored May 13, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2ff297dc
- [Bugfix][V1] Only get input embeddings w/ multi-modal models if first PP (#17916) · 8dd0671b
  Jin Huang authored May 13, 2025
```
Signed-off-by: Jin Huang <jinhun@amazon.com>
Co-authored-by: Jin Huang <jinhun@amazon.com>
```
  8dd0671b
- [v1][KVCacheManager] Avoid full cache hit by controlling max_length (#17999) · f0d610a8
  Chen Zhang authored May 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f0d610a8
- cleanup invalid prints (#18050) · 48545728
  Calvin Chen authored May 13, 2025
```
Signed-off-by: calvin chen <120380290@qq.com>
```
  48545728
- [Feature][V1] Support `tool_choice: required` when using Xgrammar as the... · dc1a8217
  Chauncey authored May 13, 2025
```
[Feature][V1]  Support `tool_choice: required` when using Xgrammar as the `StructuredOutputBackend`. (#17845)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  dc1a8217
- [Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) · 61e0a506
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61e0a506
- [Bugfix] Fixes for new marlin moe usage (#18017) · 1df491c5
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  1df491c5
- [Misc] Slight spelling modification (#18039) · c06af9a9
  Jee Jee Li authored May 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c06af9a9
- Implements dual-chunk-flash-attn backend for dual chunk attention with sparse... · 60f76243
  Tao He authored May 13, 2025
```
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844)
```
  60f76243
- Remove noisy warnings from `SchedulerConfig` (#17995) · d67085c2
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d67085c2
- Use NVFP4 Marlin for CompressedTensorsW4A16Fp4 (#18000) · 307939f2
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  307939f2
12 May, 2025 18 commits

Update some more deprecated type hinting (#17998) · 9d7ea9db
Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
9d7ea9db

[Model] Support MiMo-7B inference with MTP (#17433) · acee8f48

bwshen-mi authored May 13, 2025


Signed-off-by: wp-alpha <wangpeng66@xiaomi.com>
Co-authored-by: wangpeng66 <wangpeng66@xiaomi.com>

acee8f48

Fix FBGEMM integration (#18002) · f065de4e
Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
f065de4e
[V1][Spec Decode] Eagle unit tests (#17350) · dc990536
wwl2755 authored May 12, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
dc990536
[Chore] Remove unused method (#18024) · 195adb47
Robert Shaw authored May 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
195adb47
[v1][KVCacheManager] Change prefix caching metric from counting blocks to counting tokens (#18003) · 302f3aca
Chen Zhang authored May 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
302f3aca
[Core] Use platform-agnostic device control for DP engine core (#17245) · 289199fe
Jade Zheng authored May 13, 2025
```
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
```
289199fe
[Lora][Frontend]Add default local directory LoRA resolver plugin. (#16855) · 98ea3560
Jonathan Berkhahn authored May 12, 2025
```
Signed-off-by: jberkhahn <jaberkha@us.ibm.com>
```
98ea3560

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

Initialize the delta tool call fields explicitly (#17340) · 05a4324f
Maximilien de Bayser authored May 12, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: igmainc <igmainc@icloud.com>
```
05a4324f
[Misc] Improve modelscope import error (#17983) · 7ea6cb28
Jee Jee Li authored May 12, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
7ea6cb28

[Feature] Support DeepSeekV3 Function Call (#17784) · 3a5ea751

Xu Wenqing authored May 12, 2025


Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
Signed-off-by: Xu Wenqing <xuwq1993@qq.com>

3a5ea751

[Fix] Benchmark `"EngineClient" has no attribute "model_config"` (#17976) · 891b9d33
Brayden Zhong authored May 12, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
891b9d33
[Bugfix][TPU] Use np array when updating cache slot_mapping (#17971) · 43078301
Siyuan Liu authored May 11, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
43078301
[Bugfix] Fix pydantic.errors.PydanticUserError (#17962) · 19a3c78d
Li Wang authored May 12, 2025
```
Signed-off-by: wangli <wangli858794774@gmail.com>
```
19a3c78d

[bugfix] fix the wrong parser (#17958) · ada50aa2

Reid authored May 12, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

ada50aa2

[Bugfix] validate grammar and throw 400 error instead of crashing the engine... · 08bf7840

Cheng Kuan Yong Jason authored May 12, 2025


[Bugfix] validate grammar and throw 400 error instead of crashing the engine when xgrammar validation fails (#17623)
Signed-off-by: Jason Cheng <jasoncky96@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

08bf7840

[Model] Broadcast Ovis2 implementation to fit Ovis1.6 (#17861) · 021c16c7
Isotr0py authored May 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
021c16c7

11 May, 2025 2 commits
- [BUG] [ROCm] [MLA] Fix variable name bug due to change in variable name in PR #17483 (#17961) · 7de18d54
  TJian authored May 12, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  7de18d54
- [BugFix] [ROCm]: Bugfix and handle addition case of input for `rocm_aiter_rms_norm` (#17857) · a810b5b0
  TJian authored May 11, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  a810b5b0