Commits · f205c09854853172a446c92aa81eb7199da324ab · OpenDAS / vllm_cscc

29 Aug, 2024 1 commit
- [Bugfix] Unify rank computation across regular decoding and speculative decoding (#7899) · f205c098
  Jonas M. Kübler authored Aug 29, 2024
  
  f205c098
25 Aug, 2024 1 commit
- [Spec Decoding] Streamline batch expansion tensor manipulation (#7851) · 1856aff4
  Nick Hill authored Aug 25, 2024
  
  1856aff4
22 Aug, 2024 1 commit
- [Bugfix] spec decode handle None entries in topk args in create_sequence_group_output (#7232) · cc0eaf12
  Travis Johnson authored Aug 22, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  cc0eaf12
20 Aug, 2024 1 commit
- [Speculative Decoding] Fixing hidden states handling in batch expansion (#7508) · 312f7612
  Abhinav Goyal authored Aug 20, 2024
  
  312f7612
05 Aug, 2024 1 commit
- [Speculative decoding] Add periodic log with time spent in proposal/scoring/verification (#6963) · 82a1b1a8
  Cade Daniel authored Aug 05, 2024
  
  82a1b1a8
21 Jul, 2024 1 commit
- [Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both... · 14f91fe6
  sroy745 authored Jul 20, 2024
```
[Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both draft and target models. (#6485)
```
  14f91fe6
21 Jun, 2024 1 commit

[Model] MLPSpeculator speculative decoding support (#4947) · b12518d3

Joshua Rosenkranz authored Jun 20, 2024


Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Davis Wertheimer <Davis.Wertheimer@ibm.com>

b12518d3

15 Jun, 2024 1 commit
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
11 Jun, 2024 1 commit
- [Misc] Various simplifications and typing fixes (#5368) · a0086298
  Nick Hill authored Jun 10, 2024
  
  a0086298
11 May, 2024 1 commit
- [Model][Misc] Add e5-mistral-7b-instruct and Embedding API (#3734) · e254497b
  Chang Su authored May 11, 2024
  
  e254497b
03 May, 2024 1 commit
- [Speculative decoding] Support target-model logprobs (#4378) · ab502751
  Cade Daniel authored May 03, 2024
  
  ab502751
01 May, 2024 1 commit
- [Speculative decoding] Add ngram prompt lookup decoding (#4237) · b38e42fb
  leiwen83 authored May 02, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  b38e42fb
16 Apr, 2024 1 commit
- [Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894) · e95cd879
  Cade Daniel authored Apr 16, 2024
  
  e95cd879
25 Mar, 2024 1 commit
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
09 Mar, 2024 1 commit
- [Speculative decoding 3/9] Worker which speculates, scores, and applies rejection sampling (#3103) · 8437bae6
  Cade Daniel authored Mar 08, 2024
  
  8437bae6