Commits · ad385667103b7a98fee5a4fc46d2a7745407333e · OpenDAS / vllm_cscc

30 Aug, 2024 1 commit
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
25 Aug, 2024 1 commit
- [Spec Decoding] Streamline batch expansion tensor manipulation (#7851) · 1856aff4
  Nick Hill authored Aug 25, 2024
  
  1856aff4
20 Aug, 2024 1 commit
- [Speculative Decoding] Fixing hidden states handling in batch expansion (#7508) · 312f7612
  Abhinav Goyal authored Aug 20, 2024
  
  312f7612
19 Jul, 2024 1 commit
- [BUGFIX] Raise an error for no draft token case when draft_tp>1 (#6369) · a921e863
  Woo-Yeon Lee authored Jul 19, 2024
  
  a921e863
17 Jul, 2024 1 commit
- [Misc][Speculative decoding] Typos and typing fixes (#6467) · a19e8d37
  shangmingc authored Jul 17, 2024
```
Co-authored-by: caishangming.csm <caishangming.csm@alibaba-inc.com>
```
  a19e8d37
10 Jul, 2024 1 commit
- [Speculative Decoding] Enabling bonus token in speculative decoding for KV... · ae151d73
  sroy745 authored Jul 10, 2024
```
[Speculative Decoding] Enabling bonus token in speculative decoding for KV cache based models (#5765)
```
  ae151d73
21 Jun, 2024 1 commit

[Model] MLPSpeculator speculative decoding support (#4947) · b12518d3

Joshua Rosenkranz authored Jun 20, 2024


Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>

b12518d3

11 Jun, 2024 1 commit
- [Misc] Various simplifications and typing fixes (#5368) · a0086298
  Nick Hill authored Jun 10, 2024
  
  a0086298
05 Jun, 2024 1 commit
- [Speculative Decoding] Add `ProposerWorkerBase` abstract class (#5252) · faf71bcd
  Nick Hill authored Jun 05, 2024
  
  faf71bcd
13 May, 2024 1 commit
- [Speculative decoding] Improve n-gram efficiency (#4724) · ce532ff4
  Cody Yu authored May 13, 2024
  
  ce532ff4
08 May, 2024 1 commit
- [Dynamic Spec Decoding] Auto-disable by the running queue size (#4592) · f942efb5
  Cody Yu authored May 08, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  f942efb5
04 May, 2024 1 commit
- [Misc][Refactor] Introduce ExecuteModelData (#4540) · bc8ad684
  Cody Yu authored May 03, 2024
  
  bc8ad684
03 May, 2024 1 commit
- [Speculative decoding] Support target-model logprobs (#4378) · ab502751
  Cade Daniel authored May 03, 2024
  
  ab502751
01 May, 2024 1 commit
- [Speculative decoding] Add ngram prompt lookup decoding (#4237) · b38e42fb
  leiwen83 authored May 02, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  b38e42fb