Commits · fe393be80b94f07bb7e1e141aa4136138a1f7795 · OpenDAS / vllm_cscc

31 Jul, 2025 1 commit
- [feat]支持v1 engine mtp cudagraph · fe393be8
  王敏 authored Jul 31, 2025
  
  fe393be8
12 Jul, 2025 1 commit
- [fix]1.优化mtp代码，解决prefix-caching不兼容问题; 2.修复v1 engine从tokenizer config中读取max_model_len导致长文本输入报错 · b16f183c
  王敏 authored Jul 12, 2025
  
  b16f183c
07 Jul, 2025 1 commit
- [feat]支持v1 engine flashmla和mtp同时开启 · ac4dedb1
  王敏 authored Jul 07, 2025
  
  ac4dedb1
06 Jul, 2025 1 commit
- [V1] Support any head size for FlexAttention backend (#20467) · 9fb52e52
  Cyrus Leung authored Jul 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9fb52e52
30 Jun, 2025 1 commit
- [fix]解决v1 engine mtp只能设为1的问题 · 504c262e
  王敏 authored Jun 30, 2025
  
  504c262e
25 Jun, 2025 1 commit
- [V1][Speculative Decoding] Fix DeepSeek MTP (#20022) · 8359f4c8
  cjackal authored Jun 26, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  8359f4c8
13 Jun, 2025 3 commits
- [CUDA] Enable full cudagraph for FlashMLA (#18581) · 3597b06a
  Luka Govedič authored Jun 13, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  3597b06a
- [torch.compile] reorganize the cache directory to support compiling multiple models (#19064) · d70bc7c0
  youkaichao authored Jun 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d70bc7c0
- [Bugfix] Fix EAGLE vocab embedding for multimodal target model (#19570) · c68698b3
  qizixi authored Jun 12, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c68698b3
06 Jun, 2025 1 commit
- [Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033) · 3465b87e
  Benjamin Chislett authored Jun 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  3465b87e
03 Jun, 2025 1 commit
- [Misc] Add SPDX-FileCopyrightText (#19100) · 02f0c7b2
  Simon Mo authored Jun 03, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  02f0c7b2
28 May, 2025 1 commit
- [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) (#18781) · 5e13c07d
  RonaldBXu authored May 27, 2025
```
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
```
  5e13c07d
24 May, 2025 2 commits
- [V1][Spec Decode] Support multi-layer eagle draft model (#18030) · c1e4a405
  qizixi authored May 24, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c1e4a405
- [V1][Spec Decode] Small refactors to improve eagle bookkeeping performance (#18424) · d55e446d
  qizixi authored May 23, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  d55e446d
23 May, 2025 4 commits
- [V1] Support Deepseek MTP (#18435) · 2628a69e
  Jiayi Yao authored May 23, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>
```
  2628a69e
- Revert "[V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (#18034)" (#18600) · fbb13a2c
  Cyrus Leung authored May 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  fbb13a2c
- [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (#18034) · 4c611348
  RonaldBXu authored May 23, 2025
```
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
```
  4c611348
- [V1][Spec Decoding] Use model_loader.get_model() to load models (#18273) · c6b636f9
  Mark McLoughlin authored May 23, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c6b636f9
22 May, 2025 1 commit
- [V1][Spec Decode][Bugfix] Load quantize weights for EAGLE (#18290) · a04720bc
  Ekagra Ranjan authored May 22, 2025
  
  a04720bc
14 May, 2025 1 commit

[V1][Spec Decode] Share input embedding of target model with EAGLE draft model... · 418d2f8b

Ekagra Ranjan authored May 14, 2025


[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

418d2f8b

12 May, 2025 1 commit
- [V1][Spec Decode] Eagle unit tests (#17350) · dc990536
  wwl2755 authored May 12, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  dc990536
07 May, 2025 2 commits
- [Misc] Split model loader (#17712) · 822de7fb
  Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  822de7fb
- [BugFix][Spec Decode] Fix hidden size mismatch between target and eagle head (#17740) · 8d84d836
  Woosuk Kwon authored May 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d84d836
06 May, 2025 2 commits
- [v1] AttentionMetadata for each layer (#17394) · cba31c47
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cba31c47
- [Bugfix] Fix triton import with local TritonPlaceholder (#17446) · f9bc5a06
  Mengqing Cao authored May 06, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  f9bc5a06
01 May, 2025 1 commit
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE3 (#17504) · 39c0813a
  qizixi authored May 01, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  39c0813a
29 Apr, 2025 2 commits
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE (#17211) · 70788bdb
  Bryan Lu authored Apr 29, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  70788bdb
- [V1][Spec Decode] Make Eagle model arch config driven (#17323) · e1360005
  Ekagra Ranjan authored Apr 28, 2025
  
  e1360005
25 Apr, 2025 1 commit

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

23 Apr, 2025 1 commit
- [V1][Spec Decode] Always use argmax for sampling draft tokens (#16899) · 41fb013d
  Woosuk Kwon authored Apr 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  41fb013d
22 Apr, 2025 1 commit
- [BugFix][Spec Decode] No in-place update to draft probs (#16952) · 1311913f
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1311913f
21 Apr, 2025 1 commit
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
10 Apr, 2025 1 commit
- [V1][Spec Decode] Eagle Model loading (#16035) · e8224f3d
  Lily Liu authored Apr 10, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  e8224f3d
02 Apr, 2025 1 commit
- [Spec Decode] Fix input triton kernel for eagle (#15909) · 24b7fb45
  Ekagra Ranjan authored Apr 01, 2025
  
  24b7fb45
01 Apr, 2025 1 commit
- [V1][Spec Decode] Implement Eagle Proposer [1/N] (#15729) · e75a6301
  Woosuk Kwon authored Apr 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e75a6301