Commits · 2fc9075b82e05007f460992b3f9d42d2746c41cb · OpenDAS / vllm_cscc

"tests/kernels/attention/test_attention.py" did not exist on "96b6f475dda40a0c7d557f73c36fe09c07be2e9c"

14 May, 2025 8 commits
- [V1] Structured Outputs + Thinking compatibility (#16577) · 2fc9075b
  Aaron Pham authored May 14, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  2fc9075b
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
- [V1][Spec Decode] Share input embedding of target model with EAGLE draft model... · 418d2f8b
  Ekagra Ranjan authored May 14, 2025
```
[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  418d2f8b
- [KVConnector] Keep KVTransferParams as a dict (#18033) · 59dd311c
  Nick Hill authored May 14, 2025
  
  59dd311c
- [Bugfix][V1] Fix FlashInfer V1 backend using the wrong VllmConfig (#18086) · 12e6c0b4
  Michael Goin authored May 13, 2025
  
  12e6c0b4
- [core][distributed] add ep group and all2all interface (#18077) · 6266c57b
  youkaichao authored May 14, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6266c57b
- [Misc] Remove unused numpy tensor (#18084) · 6e27c6d8
  Roger Wang authored May 13, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.me>
```
  6e27c6d8
- [v1][KVCacheManager] pass num_new_computed_tokens to kv cache manager (#18001) · f2ae883b
  Chen Zhang authored May 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  f2ae883b
13 May, 2025 6 commits
- [V1] DP scale-out (2/N): Decouple engine process management and comms (#15977) · 55aa7af9
  Nick Hill authored May 13, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  55aa7af9
- [Bugfix] Fix entrypoints metrics tests (#18063) · b922c2eb
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b922c2eb
- [Bugfix][V1] Only get input embeddings w/ multi-modal models if first PP (#17916) · 8dd0671b
  Jin Huang authored May 13, 2025
```
Signed-off-by: Jin Huang <jinhun@amazon.com>
Co-authored-by: Jin Huang <jinhun@amazon.com>
```
  8dd0671b
- [v1][KVCacheManager] Avoid full cache hit by controlling max_length (#17999) · f0d610a8
  Chen Zhang authored May 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f0d610a8
- [Feature][V1] Support `tool_choice: required` when using Xgrammar as the... · dc1a8217
  Chauncey authored May 13, 2025
```
[Feature][V1]  Support `tool_choice: required` when using Xgrammar as the `StructuredOutputBackend`. (#17845)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  dc1a8217
- [Bugfix] Avoid repeatedly creating dummy data during engine startup (#17935) · 61e0a506
  Cyrus Leung authored May 13, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61e0a506
12 May, 2025 7 commits

[V1][Spec Decode] Eagle unit tests (#17350) · dc990536
wwl2755 authored May 12, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
dc990536
[Chore] Remove unused method (#18024) · 195adb47
Robert Shaw authored May 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
195adb47
[v1][KVCacheManager] Change prefix caching metric from counting blocks to counting tokens (#18003) · 302f3aca
Chen Zhang authored May 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
302f3aca
[Core] Use platform-agnostic device control for DP engine core (#17245) · 289199fe
Jade Zheng authored May 13, 2025
```
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
```
289199fe

[P/D] NIXL Integration (#17751) · d1911020

Robert Shaw authored May 12, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Brent Salisbury <bsalisbu@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Brent Salisbury <bsalisbu@redhat.com>

d1911020

[Bugfix][TPU] Use np array when updating cache slot_mapping (#17971) · 43078301
Siyuan Liu authored May 11, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
43078301

[Bugfix] validate grammar and throw 400 error instead of crashing the engine... · 08bf7840

Cheng Kuan Yong Jason authored May 12, 2025


[Bugfix] validate grammar and throw 400 error instead of crashing the engine when xgrammar validation fails (#17623)
Signed-off-by: Jason Cheng <jasoncky96@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

08bf7840

11 May, 2025 3 commits
- [BUG] [ROCm] [MLA] Fix variable name bug due to change in variable name in PR #17483 (#17961) · 7de18d54
  TJian authored May 12, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  7de18d54
- [FP8][ROCm][Attention] Enable FP8 KV cache on ROCm for V1 (#17870) · 06c0922a
  Gregory Shtrasberg authored May 11, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  06c0922a
- [Bugfix]: v1 engine - consider lora adapters in allowed_token_ids (#17855) · 8132365b
  Ben Browning authored May 11, 2025
```
Signed-off-by: Ben Browning <bbrownin@redhat.com>
```
  8132365b
10 May, 2025 3 commits
- [v1] Rename specialized_manager.py to single_type_kv_cache_manager.py (#17946) · ca66a167
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  ca66a167
- [v1] Pass BlockTable and KVCacheSpec to AttentionMetadataBuilders (#17483) · 950751a9
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  950751a9
- [V1][Spec Decoding] Log accumulated metrics after system goes idle (#17913) · 7042cc96
  Mark McLoughlin authored May 10, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7042cc96
09 May, 2025 6 commits
- [V1][Spec Decoding] Include bonus tokens in mean acceptance length (#17908) · 7e357113
  Mark McLoughlin authored May 09, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7e357113
- [v1] Move block management logic from KVCacheManager to SpecializedManager (#17474) · 200da9a5
  Chen Zhang authored May 09, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  200da9a5
- [Bugfix][ROCm] Fix AITER MLA V1 (#17880) · 217db4ba
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  217db4ba
- [BUGFIX]: return fast when request requires prompt logprobs (#17251) · d310e6de
  Ning Xie authored May 09, 2025
  
  d310e6de
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394
- [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) · 3c9396a6
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  3c9396a6
08 May, 2025 5 commits
- [V1] Improve VLLM_ALLOW_INSECURE_SERIALIZATION logging (#17860) · 226a4272
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  226a4272
- [TPU] Fix the test_sampler (#17820) · a463555d
  Jevin Jiang authored May 08, 2025
  
  a463555d
- [V1] Add VLLM_ALLOW_INSECURE_SERIALIZATION env var (#17490) · 6930a411
  Russell Bryant authored May 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  6930a411
- [Core] Support full cuda graph in v1 (#16072) · 7ea2adb8
  Chanh Nguyen authored May 07, 2025
```
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
```
  7ea2adb8
- [BugFix] Fix `--disable-log-stats` in V1 server mode (#17600) · 3d13ca0e
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  3d13ca0e
07 May, 2025 2 commits
- [Core][Feature] Input metadata dump on crash (#13407) · d43f914d
  Wallas Henrique authored May 07, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  d43f914d
- [BugFix] Avoid secondary missing `MultiprocExecutor.workers` error (#17811) · ed5272cf
  Nick Hill authored May 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ed5272cf