Commits · a04720bc36401d831cb048c3917b9e58173d9c1d · OpenDAS / vllm_cscc · GitLab

22 May, 2025 1 commit
- [V1][Spec Decode][Bugfix] Load quantize weights for EAGLE (#18290) · a04720bc
  Ekagra Ranjan authored May 22, 2025
  
  a04720bc
16 May, 2025 2 commits

[Misc] Add Ray Prometheus logger to V1 (#17925) · 54181767
Seiji Eicher authored May 16, 2025
```
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
```
54181767

[Model] vLLM v1 supports Medusa (#17956) · f4937a51

Sky Lee authored May 16, 2025


Signed-off-by: lisiqi23 <lisiqi23@xiaomi.com>
Signed-off-by: skylee-01 <497627264@qq.com>
Co-authored-by: lisiqi23 <lisiqi23@xiaomi.com>

f4937a51

14 May, 2025 1 commit

[V1][Spec Decode] Share input embedding of target model with EAGLE draft model... · 418d2f8b

Ekagra Ranjan authored May 14, 2025


[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

418d2f8b

12 May, 2025 1 commit
- [V1][Spec Decode] Eagle unit tests (#17350) · dc990536
  wwl2755 authored May 12, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  dc990536
10 May, 2025 1 commit
- [V1][Spec Decoding] Log accumulated metrics after system goes idle (#17913) · 7042cc96
  Mark McLoughlin authored May 10, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7042cc96
09 May, 2025 1 commit
- [V1][Spec Decoding] Include bonus tokens in mean acceptance length (#17908) · 7e357113
  Mark McLoughlin authored May 09, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7e357113
07 May, 2025 2 commits
- [Misc] Split model loader (#17712) · 822de7fb
  Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  822de7fb
- [BugFix][Spec Decode] Fix hidden size mismatch between target and eagle head (#17740) · 8d84d836
  Woosuk Kwon authored May 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d84d836
06 May, 2025 2 commits
- [v1] AttentionMetadata for each layer (#17394) · cba31c47
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cba31c47
- [Bugfix] Fix triton import with local TritonPlaceholder (#17446) · f9bc5a06
  Mengqing Cao authored May 06, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  f9bc5a06
01 May, 2025 1 commit
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE3 (#17504) · 39c0813a
  qizixi authored May 01, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  39c0813a
29 Apr, 2025 2 commits
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE (#17211) · 70788bdb
  Bryan Lu authored Apr 29, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  70788bdb
- [V1][Spec Decode] Make Eagle model arch config driven (#17323) · e1360005
  Ekagra Ranjan authored Apr 28, 2025
  
  e1360005
25 Apr, 2025 1 commit

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

24 Apr, 2025 1 commit
- [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) · 340d7b1b
  Mark McLoughlin authored Apr 24, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  340d7b1b
23 Apr, 2025 1 commit
- [V1][Spec Decode] Always use argmax for sampling draft tokens (#16899) · 41fb013d
  Woosuk Kwon authored Apr 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  41fb013d
22 Apr, 2025 1 commit
- [BugFix][Spec Decode] No in-place update to draft probs (#16952) · 1311913f
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1311913f
21 Apr, 2025 1 commit
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
17 Apr, 2025 1 commit
- [V1] Remove log noise when idle (#16735) · 9dbf7a2d
  Russell Bryant authored Apr 17, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  9dbf7a2d
10 Apr, 2025 1 commit
- [V1][Spec Decode] Eagle Model loading (#16035) · e8224f3d
  Lily Liu authored Apr 10, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  e8224f3d
04 Apr, 2025 1 commit
- [V1][Spec Decode] Update N-gram Proposer Interface (#15750) · 63375f0c
  Woosuk Kwon authored Apr 04, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  63375f0c
02 Apr, 2025 2 commits
- [V1][Minor] Enhance SpecDecoding Metrics Log in V1 (#15902) · 274d8e88
  Woosuk Kwon authored Apr 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  274d8e88
- [Spec Decode] Fix input triton kernel for eagle (#15909) · 24b7fb45
  Ekagra Ranjan authored Apr 01, 2025
  
  24b7fb45
01 Apr, 2025 2 commits
- [V1][Spec Decode] Implement Eagle Proposer [1/N] (#15729) · e75a6301
  Woosuk Kwon authored Apr 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e75a6301
- [V1][Metrics] Initial speculative decoding metrics (#15151) · a79cc68b
  Mark McLoughlin authored Apr 01, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  a79cc68b
25 Mar, 2025 1 commit
- [V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) · ebcebeeb
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ebcebeeb
23 Mar, 2025 1 commit
- [V1][Spec Decode] Respect prompt_lookup_max (#15348) · b9bd76ca
  Woosuk Kwon authored Mar 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b9bd76ca
18 Mar, 2025 1 commit
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
17 Mar, 2025 1 commit
- [V1] [Spec Decode] Support random sampling for spec decode (#13933) · 8d6cf895
  Lily Liu authored Mar 16, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d6cf895
18 Feb, 2025 1 commit
- [V1][Spec Decode] Optimize N-gram matching with Numba (#13365) · 4c822298
  Woosuk Kwon authored Feb 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4c822298
17 Feb, 2025 1 commit
- [V1][Spec decode] Move drafter to model runner (#13363) · cd4a72a2
  Woosuk Kwon authored Feb 17, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  cd4a72a2
16 Feb, 2025 2 commits
- [V1][BugFix] Add __init__.py to v1/spec_decode/ (#13359) · 7b893865
  Woosuk Kwon authored Feb 16, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7b893865
- [V1][Spec Decode] Ngram Spec Decode (#12193) · 80f63a39
  Lily Liu authored Feb 15, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  80f63a39