Commits · cb3b2b9ba4a95c413a879e30e2b8674187519a93 · OpenDAS / vllm_cscc

06 Oct, 2024 1 commit
- [Bugfix] Fix incorrect updates to num_computed_tokens in multi-step scheduling (#9038) · cb3b2b9b
  Varun Sundar Rabindranath authored Oct 06, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  cb3b2b9b
27 Aug, 2024 1 commit
- [Core] Asynchronous Output Processor (#7049) · 2eedede8
  Megha Agarwal authored Aug 26, 2024
```
Co-authored-by: Alexander Matveev <alexm@neuralmagic.com>
```
  2eedede8
08 Aug, 2024 1 commit
- [Bugfix][fast] Fix the get_num_blocks_touched logic (#6849) · 782e53ab
  Zach Zheng authored Aug 08, 2024
  
  782e53ab
06 Aug, 2024 1 commit

[Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026

afeldman-nm authored Aug 06, 2024


[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

fd95e026

15 Jun, 2024 1 commit
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
29 May, 2024 2 commits
- [Core] Avoid the need to pass `None` values to `Sequence.inputs` (#5099) · b1c25563
  Cyrus Leung authored May 30, 2024
  
  b1c25563
- [Core] Cross-attention KV caching and memory-management (towards eventual... · 4238bc82
  afeldman-nm authored May 29, 2024
```
[Core] Cross-attention KV caching and memory-management (towards eventual encoder/decoder model support) (#4837)
```
  4238bc82
28 May, 2024 1 commit
- [Core] Consolidate prompt arguments to LLM engines (#4328) · 5ae5ed1e
  Cyrus Leung authored May 29, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  5ae5ed1e
10 May, 2024 1 commit
- [CI] Nits for bad initialization of SeqGroup in testing (#4748) · fcc2994b
  Robert Shaw authored May 10, 2024
  
  fcc2994b
16 Apr, 2024 1 commit
- [Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894) · e95cd879
  Cade Daniel authored Apr 16, 2024
  
  e95cd879
03 Apr, 2024 1 commit
- [3/N] Refactor scheduler for chunked prefill scheduling (#3550) · 3dcb3e8b
  SangBin Cho authored Apr 04, 2024
  
  3dcb3e8b
01 Apr, 2024 1 commit
- [Speculative decoding 4/9] Lookahead scheduling for speculative decoding (#3250) · 93deb0b3
  Cade Daniel authored Apr 01, 2024
  
  93deb0b3
28 Mar, 2024 1 commit
- [Core][Bugfix]Refactor block manager for better testability (#3492) · 14ccd94c
  Cade Daniel authored Mar 27, 2024
  
  14ccd94c
06 Mar, 2024 2 commits
- [Testing] Fix core tests (#3224) · a33ce60c
  Cade Daniel authored Mar 06, 2024
  
  a33ce60c
- [Tests] Add block manager and scheduler tests (#3108) · 24aecf42
  SangBin Cho authored Mar 06, 2024
  
  24aecf42