Commits · 96ae75ad7f96c70c436f9297b06d059ad85140f9 · OpenDAS / vllm_cscc · GitLab

23 Dec, 2024 1 commit
- [Docs] Convert rST to MyST (Markdown) (#11145) · 32aa2059
  Rafael Vasquez authored Dec 23, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  32aa2059
11 Dec, 2024 1 commit
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68
07 Dec, 2024 1 commit
- [Doc] Explicitly state that PP isn't compatible with speculative decoding yet (#10975) · c889d588
  Cyrus Leung authored Dec 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c889d588
06 Dec, 2024 1 commit
- [feat]并行解码支持多卡推理 · 4a4e3601
  王敏 authored Dec 06, 2024
  
  4a4e3601
05 Dec, 2024 2 commits
- [fix]修复单测test_mlp_correctness失败问题 · d623e721
  王敏 authored Dec 05, 2024
  
  d623e721
- [Doc] Create a new "Usage" section (#10827) · aa39a8e1
  Cyrus Leung authored Dec 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa39a8e1
04 Dec, 2024 1 commit
- 修复新request kv cache移动错误 · f0021a4d
  王敏 authored Dec 04, 2024
  
  f0021a4d
03 Dec, 2024 1 commit
- [Speculative Decoding] Move indices to device before filtering output (#10850) · f6084f63
  Yang Zheng authored Dec 03, 2024
```
Co-authored-by: Yang Zheng(SW)(Alex) <you@example.com>
```
  f6084f63
27 Nov, 2024 3 commits

1.优化medusa推理，节省cpu耗时 · 28375803
王敏 authored Nov 27, 2024
```
2.更新medusa readme
3.解决benchmark_moe报错问题
```
28375803

[Bugfix][SpecDecode] apply sampling parameters to target probabilities for... · 1bf905dd

jeongin601 authored Nov 27, 2024


[Bugfix][SpecDecode] apply sampling parameters to target probabilities for consistency in rejection sampling. (#10198)
Signed-off-by: jeongin601 <0200angela@gmail.com>
Signed-off-by: jeong_in.bae <jeong_in.bae@navercorp.com>

1bf905dd

Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
0a71900b

26 Nov, 2024 1 commit

[Bugfix] Fix for Spec model TP + Chunked Prefill (#10232) · db66e018

Murali Andoorveedu authored Nov 26, 2024


Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
Signed-off-by: Sourashis Roy <sroy@roblox.com>
Co-authored-by: Sourashis Roy <sroy@roblox.com>

db66e018

22 Nov, 2024 1 commit
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
20 Nov, 2024 1 commit
- [fix]修复单测test_medusa_correctness.py中的错误 · 215f33b0
  王敏 authored Nov 20, 2024
  
  215f33b0
18 Nov, 2024 1 commit
- [fix]修复test_spec_decode_worker中的错误 · dc2ce261
  王敏 authored Nov 18, 2024
  
  dc2ce261
15 Nov, 2024 1 commit
- [Bugfix] Qwen-vl output is inconsistent in speculative decoding (#10350) · 2ec88272
  Sky Lee authored Nov 15, 2024
  
  2ec88272
09 Nov, 2024 1 commit
- [0/N] Rename `MultiModalInputs` to `MultiModalKwargs` (#10040) · e0191a95
  Cyrus Leung authored Nov 09, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e0191a95
07 Nov, 2024 1 commit
- [Feature] [Spec decode]: Combine chunked prefill with speculative decoding (#9291) · 9d43afcc
  Nicolò Lucchesi authored Nov 07, 2024
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  9d43afcc
06 Nov, 2024 2 commits
- 1.解决medusa模型多卡推理结果异常问题 · f9060e6b
  王敏 authored Nov 06, 2024
```
2.examples中添加medusa readme
3.修复model_runner中input_positions配置错误的笔误，解决多个模型运行失败问题
```
  f9060e6b
- [Bugfix][SpecDecode] kv corruption with bonus tokens in spec decode (#9730) · 0c63c34f
  Sungjae Lee authored Nov 06, 2024
```
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  0c63c34f
04 Nov, 2024 1 commit
- [4.5/N] bugfix for quant config in speculative decode (#10007) · 2094062b
  youkaichao authored Nov 04, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2094062b
02 Nov, 2024 1 commit
- [2/N] executor pass the complete config to worker/modelrunner (#9938) · e8937954
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e8937954
27 Oct, 2024 1 commit
- [Misc] SpecDecodeWorker supports profiling (#9719) · 67a6882d
  科英 authored Oct 27, 2024
```
Signed-off-by: Abatom <abatom@163.com>
```
  67a6882d
24 Oct, 2024 1 commit
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
21 Oct, 2024 1 commit
- [Doc] Consistent naming of attention backends (#9498) · 496e991d
  Thomas Parnell authored Oct 21, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  496e991d
16 Oct, 2024 1 commit
- [Performance][Spec Decode] Optimize ngram lookup performance (#9333) · 83450458
  Lily Liu authored Oct 16, 2024
  
  83450458
12 Oct, 2024 1 commit
- [SpecDec] Remove Batch Expansion (2/3) (#9298) · 89feb4c8
  Lily Liu authored Oct 11, 2024
  
  89feb4c8
11 Oct, 2024 1 commit
- [Doc] Compatibility matrix for mutual exclusive features (#8512) · 8baf85e4
  Wallas Henrique authored Oct 11, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  8baf85e4
06 Oct, 2024 1 commit
- [Bugfix] Fix try-catch conditions to import correct Flash Attention Backend in Draft Model (#9101) · 23fea871
  TJian authored Oct 05, 2024
  
  23fea871
03 Oct, 2024 1 commit
- [misc] add forward context for attention (#9029) · 9aaf14c6
  youkaichao authored Oct 03, 2024
  
  9aaf14c6
01 Oct, 2024 1 commit
- [Spec Decode] (1/2) Remove batch expansion (#8839) · 15702038
  Lily Liu authored Oct 01, 2024
  
  15702038
25 Sep, 2024 1 commit
- [Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047) · 01b6f9e1
  Travis Johnson authored Sep 24, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  01b6f9e1
22 Sep, 2024 1 commit
- [SpecDec][Misc] Cleanup, remove bonus token logic. (#8701) · c6bd70d7
  Lily Liu authored Sep 22, 2024
  
  c6bd70d7
18 Sep, 2024 1 commit

[CI/Build] Update Ruff version (#8469) · 9d104b5b

Aaron Pham authored Sep 18, 2024


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9d104b5b

10 Sep, 2024 1 commit
- [Spec Decode] Move ops.advance_step to flash attn advance_step (#8224) · 5faedf1b
  Kevin Lin authored Sep 10, 2024
  
  5faedf1b
02 Sep, 2024 1 commit
- [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) · e6a26ed0
  Lily Liu authored Sep 01, 2024
  
  e6a26ed0
30 Aug, 2024 1 commit
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
29 Aug, 2024 1 commit
- [Bugfix] Unify rank computation across regular decoding and speculative decoding (#7899) · f205c098
  Jonas M. Kübler authored Aug 29, 2024
  
  f205c098
25 Aug, 2024 1 commit
- [Spec Decoding] Streamline batch expansion tensor manipulation (#7851) · 1856aff4
  Nick Hill authored Aug 25, 2024
  
  1856aff4
22 Aug, 2024 1 commit
- [Bugfix] spec decode handle None entries in topk args in create_sequence_group_output (#7232) · cc0eaf12
  Travis Johnson authored Aug 22, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  cc0eaf12