Commits · 0640f227c9a5fb1e7193792eb36ff48d152af099 · OpenDAS / vllm_cscc · GitLab

28 Aug, 2024 1 commit
- [Performance] Enable chunked prefill and prefix caching together (#7753) · e3580537
  Cody Yu authored Aug 28, 2024
  
  e3580537
19 Aug, 2024 1 commit
- [MISC] Add prefix cache hit rate to metrics (#7606) · 3ac50b47
  Cody Yu authored Aug 19, 2024
  
  3ac50b47
03 Jun, 2024 1 commit
- [Misc]: Implement CPU/GPU swapping in BlockManagerV2 (#3834) · 10c38e3e
  Kaiyang Chen authored Jun 04, 2024
  
  10c38e3e
11 May, 2024 1 commit
- [Model][Misc] Add e5-mistral-7b-instruct and Embedding API (#3734) · e254497b
  Chang Su authored May 11, 2024
  
  e254497b
08 May, 2024 1 commit
- [Core][Optimization] change python dict to pytorch tensor for blocks to swap (#4659) · 20cfcdec
  youkaichao authored May 08, 2024
  
  20cfcdec
07 May, 2024 1 commit
- [Core][Optimization] change copy-on-write from dict[int, list] to list (#4648) · 469f85c7
  youkaichao authored May 07, 2024
  
  469f85c7
02 May, 2024 1 commit
- [Core] Ignore infeasible swap requests. (#4557) · 0f8a9140
  SangBin Cho authored May 03, 2024
  
  0f8a9140
15 Apr, 2024 1 commit
- [Typing] Fix Sequence type GenericAlias only available after Python 3.9. (#4092) · 37e84a40
  SangBin Cho authored Apr 16, 2024
  
  37e84a40
12 Apr, 2024 1 commit
- [mypy] Add mypy type annotation part 1 (#4006) · 09473ee4
  SangBin Cho authored Apr 13, 2024
  
  09473ee4
01 Apr, 2024 1 commit
- [Speculative decoding 4/9] Lookahead scheduling for speculative decoding (#3250) · 93deb0b3
  Cade Daniel authored Apr 01, 2024
  
  93deb0b3
28 Mar, 2024 1 commit
- [Core][Bugfix]Refactor block manager for better testability (#3492) · 14ccd94c
  Cade Daniel authored Mar 27, 2024
  
  14ccd94c