Commits · 70363bccfac1a6a0818ea577ad9cf8123a0ec3ae · OpenDAS / vllm_cscc

10 Apr, 2025 1 commit
- [V1][Spec Decode] Eagle Model loading (#16035) · e8224f3d
  Lily Liu authored Apr 10, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  e8224f3d
09 Apr, 2025 1 commit
- [BugFix] fix some typos found by typos. (#16314) · 04149cce
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  04149cce
08 Apr, 2025 1 commit

[V1] Scatter and gather placeholders in the model runner (#16076) · f2ebb6f5

Roger Wang authored Apr 07, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>

f2ebb6f5

07 Apr, 2025 1 commit

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

55dcce91

04 Apr, 2025 3 commits
- [V1][Spec Decode] Update N-gram Proposer Interface (#15750) · 63375f0c
  Woosuk Kwon authored Apr 04, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  63375f0c
- Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) · af51d80f
  Roger Wang authored Apr 04, 2025
  
  af51d80f
- [V1] Scatter and gather placeholders in the model runner (#15712) · f5722a50
  Cyrus Leung authored Apr 05, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f5722a50
01 Apr, 2025 2 commits
- [V1][Spec Decode] Implement Eagle Proposer [1/N] (#15729) · e75a6301
  Woosuk Kwon authored Apr 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e75a6301
- [V1] Implement sliding window attention in kv_cache_manager (#14097) · 3a5f0afc
  Chen Zhang authored Apr 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  3a5f0afc
31 Mar, 2025 4 commits
- fix: change GB to GiB in logging close #14979 (#15807) · 2de41182
  yihong authored Apr 01, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  2de41182
- [Bugfix] Check dimensions of multimodal embeddings in V1 (#15816) · 09e974d4
  Cyrus Leung authored Apr 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  09e974d4
- [V1] Fully Transparent Implementation of CPU Offloading (#15354) · 555aa219
  youkaichao authored Mar 31, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  555aa219
- [Misc] update the comments (#15780) · 18ed3132
  Chengyang LIU authored Mar 30, 2025
```
Signed-off-by: chengyang liu <lcy4869@gmail.com>
Co-authored-by: chengyang liu <lcy4869@gmail.com>
```
  18ed3132
30 Mar, 2025 1 commit
- [V1] Override `mm_counts` for dummy data creation (#15703) · 803d5c35
  Cyrus Leung authored Mar 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  803d5c35
29 Mar, 2025 1 commit
- [V1][Minor] Simplify rejection sampler's parse_output (#15741) · 2bc4be4e
  Woosuk Kwon authored Mar 29, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2bc4be4e
28 Mar, 2025 1 commit
- [V1] Remove legacy input registry (#15673) · 355f6634
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  355f6634
27 Mar, 2025 3 commits
- [Misc] Avoid direct access of global `mm_registry` in `compute_encoder_budget` (#15621) · 13ac9cab
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  13ac9cab
- [Bugfix] Fix use_cascade_attention handling for Alibi-based models on vllm/v1 (#15211) · 8958217a
  Hiroaki Sugiyama authored Mar 27, 2025
```
Signed-off-by: h-sugi <h.sugi@ieee.org>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8958217a
- [V1] Refactor num_computed_tokens logic (#15307) · 54aa6194
  Cody Yu authored Mar 26, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  54aa6194
25 Mar, 2025 1 commit
- [V1][Spec Decode] Update target_logits in place for rejection sampling (#15427) · 25f560a6
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25f560a6
24 Mar, 2025 1 commit
- [V1] Aggregate chunked prompt logprobs in model runner (#14875) · 3aee6573
  Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  3aee6573
23 Mar, 2025 2 commits
- [V1][Spec Decode] Respect prompt_lookup_max (#15348) · b9bd76ca
  Woosuk Kwon authored Mar 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b9bd76ca
- [V1][Usage] Refactor speculative decoding configuration and tests (#14434) · 50c9636d
  shangmingc authored Mar 23, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  50c9636d
21 Mar, 2025 2 commits

[v1] Refactor KVCacheConfig (#14079) · 93a00d7d
Chen Zhang authored Mar 21, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
93a00d7d

[V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) · 0c6f5023

Woosuk Kwon authored Mar 20, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

0c6f5023

20 Mar, 2025 2 commits
- [V1] Add flag to disable cascade attention (#15243) · 2b22290c
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2b22290c
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
18 Mar, 2025 1 commit
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
17 Mar, 2025 3 commits
- [V1] Remove input cache client (#14864) · b539222d
  Cyrus Leung authored Mar 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  b539222d
- [V1] [Spec Decode] Support random sampling for spec decode (#13933) · 8d6cf895
  Lily Liu authored Mar 16, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d6cf895
- [V1] Optimize the overhead of rewinding (#14905) · faa02757
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  faa02757
16 Mar, 2025 1 commit
- [V1][BugFix] Detect interleaved sliding window attention (#14896) · 31060b27
  Woosuk Kwon authored Mar 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  31060b27
14 Mar, 2025 1 commit

[V1] Move OOM check into sampler run (#14728) · ad19c8a0

Roger Wang authored Mar 13, 2025


Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ad19c8a0

12 Mar, 2025 1 commit

[V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due... · 5c538c37

Benjamin Chislett authored Mar 12, 2025


[V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due to batch indexing (#14645)
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>

5c538c37

11 Mar, 2025 1 commit

[V1][Core] Fix memory issue with logits & sampling (#14508) · 1fc973c0

Roger Wang authored Mar 10, 2025


Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Varun Sundar Rabindranath <3337719+varun-sundar-rabindranath@users.noreply.github.com>

1fc973c0

09 Mar, 2025 1 commit
- Revert "[V1][Core] Fix memory issue with logits & sampling" (#14504) · 5f0b53c6
  Robert Shaw authored Mar 08, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  5f0b53c6
08 Mar, 2025 3 commits
- [V1] Support bad_words in sampler (#13376) · eb8b5eb1
  22quinn authored Mar 08, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  eb8b5eb1
- [V1][Core] Fix memory issue with logits & sampling (#13776) · 8d5aa466
  Roger Wang authored Mar 08, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  8d5aa466
- [Bugfix][V1] Handle MLA in kv_cache_interface (#14462) · 33368140
  Tyler Michael Smith authored Mar 08, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  33368140
07 Mar, 2025 1 commit
- [V1] Eagerly remove finished requests from the batch (#14388) · 8ed5421a
  Nick Hill authored Mar 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  8ed5421a