Commits · 6bf27affb6d451a02e158aa5728a7d8c45507303 · OpenDAS / vllm_cscc

"tools/install_gdrcopy.sh" did not exist on "cef32104b4b411e5093581ad77d2e09a50c2837c"

13 Apr, 2025 1 commit
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 4 commits
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
- [Perf] Optimize Preparing Inputs for GPU Model Runner (#16484) · 93e5f3c5
  SnowCharm authored Apr 12, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  93e5f3c5
- [Feature][V1] Add xgrammar to support minLength, maxLength with test (#16516) · e92d7085
  leon-seidel authored Apr 12, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
  e92d7085
- [BugFix] Handle non-contiguous tensors properly when serializing (#16492) · 41cc883c
  Nick Hill authored Apr 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  41cc883c
11 Apr, 2025 2 commits
- [Kernel] support merge_attn_states CUDA kernel, 3x speedup (#16173) · e9528f6d
  DefTruth authored Apr 11, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  e9528f6d
- Enforce valid max_num_batched_tokens when disable_chunked_mm_input=True (#16447) · aa3b3d76
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  aa3b3d76
10 Apr, 2025 6 commits
- [TPU][V1] Disable per-request seed/Generator (#16172) · 3cc9af88
  Nicolò Lucchesi authored Apr 10, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  3cc9af88
- [V1] Zero-copy tensor/ndarray serialization/transmission (#13790) · dd143ef5
  Nick Hill authored Apr 10, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  dd143ef5
- [V1][Spec Decode] Eagle Model loading (#16035) · e8224f3d
  Lily Liu authored Apr 10, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  e8224f3d
- [TPU][V1] Use `language_model` interface for getting text backbone in MM (#16410) · c1b57855
  Nicolò Lucchesi authored Apr 10, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  c1b57855
- [Bugfix] Fix validation error for text-only Mllama 3.2 (#16377) · a5d11a54
  Cyrus Leung authored Apr 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a5d11a54
- [TPU][V1] Refine tpu_model_runner to mitigate future recompilation issues (#16275) · a4547485
  Chengji Yao authored Apr 09, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  a4547485
09 Apr, 2025 5 commits
- [Hardware] add platform-specific request validation api (#16291) · cb391d85
  Joe Runde authored Apr 09, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  cb391d85
- [BugFix] fix some typos found by typos. (#16314) · 04149cce
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  04149cce
- [Bugfix] Avoid transferring cached multi-modal items from P0 to P1 (#16273) · e484e028
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e484e028
- [Core] Upgrade to xgrammar 0.1.18, add cache size limit (#16283) · cb84e45a
  Russell Bryant authored Apr 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cb84e45a
- [Feature] Estimate max-model-len use available KV cache memory (#16168) · 4716377f
  rongfu.leng authored Apr 09, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  4716377f
08 Apr, 2025 7 commits
- [BugFix] Fix Llama4 - Index Error When Single Request Near Max Context (#16209) · e1a2c699
  Lucas Wilkinson authored Apr 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  e1a2c699
- [Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156) · 4ebc0b96
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4ebc0b96
- [V1] Add `disable_chunked_mm_input` arg to disable partial mm input prefill (#15837) · 8e5314a4
  Michael Goin authored Apr 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  8e5314a4
- [torch.compile][TPU] Make @support_torch_compile work for XLA backend (#15782) · 87918e40
  Siyuan Liu authored Apr 07, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  87918e40
- Add warning for Attention backends that do not support irope yet (#16212) · 05a015d6
  Yong Hoon Shin authored Apr 07, 2025
  
  05a015d6
- [V1] Scatter and gather placeholders in the model runner (#16076) · f2ebb6f5
  Roger Wang authored Apr 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  f2ebb6f5
- [core] do not send error across process (#16174) · a865bc1c
  youkaichao authored Apr 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a865bc1c
07 Apr, 2025 6 commits

[V1][BugFix] Exit properly if engine core fails during startup (#16137) · 7f6d47c1
Nick Hill authored Apr 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
7f6d47c1
[Bugfix] Fix guidance backend for Qwen models (#16210) · 3147586e
Benjamin Chislett authored Apr 07, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
3147586e

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

55dcce91

[V1][Structured Output] Add `supports_structured_output()` method to Platform (#16148) · e9ba99f2
Shanshan Shen authored Apr 07, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
e9ba99f2
[V1][Minor] Minor simplification for get_computed_blocks (#16139) · 3749e287
Woosuk Kwon authored Apr 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
3749e287

[Metrics] Add bucket for `request_latency`, `time_to_first_token` and... · 86fc2321

Kay Yan authored Apr 07, 2025


[Metrics] Add bucket for `request_latency`, `time_to_first_token` and `time_per_output_token` (#15202)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>

86fc2321

06 Apr, 2025 2 commits
- [V1][Minor] Optimize get_cached_block (#16135) · b10e5198
  Woosuk Kwon authored Apr 06, 2025
  
  b10e5198
- [CI][V1] Fix passing `tokenizer` as kwarg to `validate_guidance_grammar` (#16117) · 9ca710e5
  Roger Wang authored Apr 06, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  9ca710e5
04 Apr, 2025 7 commits
- [V1][Spec Decode] Update N-gram Proposer Interface (#15750) · 63375f0c
  Woosuk Kwon authored Apr 04, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  63375f0c
- [Bugfix][TPU] Fix V1 TPU worker for sliding window (#16059) · 70ad3f9e
  Michael Goin authored Apr 04, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  70ad3f9e
- Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) · af51d80f
  Roger Wang authored Apr 04, 2025
  
  af51d80f
- [V1] Scatter and gather placeholders in the model runner (#15712) · f5722a50
  Cyrus Leung authored Apr 05, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f5722a50
- [V1] DP scale-out (1/N): Use zmq ROUTER/DEALER sockets for input queue (#15906) · 651cf0fe
  Nick Hill authored Apr 04, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  651cf0fe
- [V1][Spec Decode] Avoid logging useless nan metrics (#16023) · a35a8a83
  Mark McLoughlin authored Apr 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  a35a8a83
- [TPU][V1] Remove ragged attention kernel parameter hard coding (#16041) · fadc59c0
  Chengji Yao authored Apr 04, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  fadc59c0