Commits · 702bee461f448b0186eb9d673baad29fd923c884 · OpenDAS / vllm_cscc · GitLab

10 May, 2024 1 commit
- [Speculative decoding] CUDA graph support (#4295) · 2e7796f2
  heeju-kim2 authored May 11, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  2e7796f2
08 May, 2024 1 commit
- [Dynamic Spec Decoding] Auto-disable by the running queue size (#4592) · f942efb5
  Cody Yu authored May 08, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  f942efb5
07 May, 2024 1 commit

[Bug fix][Core] fixup ngram not setup correctly (#4551) · 8344f774

leiwen83 authored May 08, 2024


Co-authored-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Cade Daniel <edacih@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

8344f774

03 May, 2024 2 commits
- [Speculative decoding] Support target-model logprobs (#4378) · ab502751
  Cade Daniel authored May 03, 2024
  
  ab502751
- [Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518) · 3521ba4f
  SangBin Cho authored May 04, 2024
  
  3521ba4f
01 May, 2024 1 commit
- [Speculative decoding] Add ngram prompt lookup decoding (#4237) · b38e42fb
  leiwen83 authored May 02, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  b38e42fb
30 Apr, 2024 1 commit
- [BugFix] fix num_lookahead_slots missing in async executor (#4165) · 4bb53e2d
  leiwen83 authored May 01, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  4bb53e2d
23 Apr, 2024 1 commit
- [Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. (#3951) · 62b8aebc
  Cade Daniel authored Apr 23, 2024
  
  62b8aebc
16 Apr, 2024 1 commit
- [Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894) · e95cd879
  Cade Daniel authored Apr 16, 2024
  
  e95cd879
03 Apr, 2024 1 commit
- [Speculative decoding] Adding configuration object for speculative decoding (#3706) · 5757d90e
  Cade Daniel authored Apr 02, 2024
```
Co-authored-by: Lily Liu <lilyliupku@gmail.com>
```
  5757d90e