Commits · 37e84a403d6d11b670a42e84153204cd8b76b849 · OpenDAS / vllm_cscc

15 Apr, 2024 1 commit
- [Typing] Fix Sequence type GenericAlias only available after Python 3.9. (#4092) · 37e84a40
  SangBin Cho authored Apr 16, 2024
  
  37e84a40
12 Apr, 2024 3 commits
- [mypy] Add mypy type annotation part 1 (#4006) · 09473ee4
  SangBin Cho authored Apr 13, 2024
  
  09473ee4
- [Misc] Fix typo in scheduler.py (#4022) · d4ec9ffb
  Zhuohan Li authored Apr 12, 2024
  
  d4ec9ffb
- [Doc] Add typing hints / mypy types cleanup (#3816) · c2b4a1bc
  Michael Feil authored Apr 11, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  c2b4a1bc
11 Apr, 2024 1 commit
- [Core][5/N] Fully working chunked prefill e2e (#3884) · 67b4221a
  SangBin Cho authored Apr 11, 2024
  
  67b4221a
07 Apr, 2024 1 commit
- [Core] latency optimization (#3890) · 2f192835
  youkaichao authored Apr 06, 2024
  
  2f192835
05 Apr, 2024 1 commit
- [Chunked Prefill][4/n] Chunked prefill scheduler. (#3853) · 18de8834
  SangBin Cho authored Apr 06, 2024
  
  18de8834
03 Apr, 2024 1 commit
- [3/N] Refactor scheduler for chunked prefill scheduling (#3550) · 3dcb3e8b
  SangBin Cho authored Apr 04, 2024
  
  3dcb3e8b
02 Apr, 2024 1 commit
- [Bugfix] Add `__init__.py` files for `vllm/core/block/` and `vllm/spec_decode/` (#3798) · b321d488
  Michael Goin authored Apr 02, 2024
  
  b321d488
01 Apr, 2024 1 commit
- [Speculative decoding 4/9] Lookahead scheduling for speculative decoding (#3250) · 93deb0b3
  Cade Daniel authored Apr 01, 2024
  
  93deb0b3
28 Mar, 2024 3 commits
- fix logging msg for block manager (#3701) · 4716a32d
  Simon Mo authored Mar 28, 2024
  
  4716a32d
- [2/N] Chunked prefill data update (#3538) · b51c1cc9
  SangBin Cho authored Mar 29, 2024
  
  b51c1cc9
- [Core][Bugfix]Refactor block manager for better testability (#3492) · 14ccd94c
  Cade Daniel authored Mar 27, 2024
  
  14ccd94c
25 Mar, 2024 3 commits
- [Feature] Add vision language model support. (#3042) · 64172a97
  xwjiang2010 authored Mar 25, 2024
  
  64172a97
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
- [Bugfix] fix automatic prefix args and add log info (#3608) · e67c295b
  TianYu GUO authored Mar 25, 2024
  
  e67c295b
22 Mar, 2024 1 commit
- Dynamic scheduler delay to improve ITL performance (#3279) · cf2f084d
  Thomas Parnell authored Mar 22, 2024
```
Co-authored-by: Jan van Lunteren <jvl@zurich.ibm.com>
```
  cf2f084d
21 Mar, 2024 1 commit
- [PREFIX CACHING FOLLOW UP] OrderedDict-based evictor (#3431) · 6ebd02bd
  ElizaWszola authored Mar 21, 2024
```
Co-authored-by: rsnm2 <rshaw@neuralmagic.com>
Co-authored-by: Luka <luka@paperspace>
```
  6ebd02bd
20 Mar, 2024 2 commits
- [1/n][Chunked Prefill] Refactor input query shapes (#3236) · 6e435de7
  SangBin Cho authored Mar 21, 2024
  
  6e435de7
- [PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance... · 9474e89b
  ElizaWszola authored Mar 20, 2024
```
[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9474e89b
15 Mar, 2024 1 commit
- Fixes the misuse/mixuse of time.time()/time.monotonic() (#3220) · 14b8ae02
  Tao He authored Mar 16, 2024
```
Signed-off-by: Tao He <sighingnow@gmail.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  14b8ae02
13 Mar, 2024 1 commit
- Fixes #1556 double free (#3347) · 49a3c866
  Breno Faria authored Mar 13, 2024
  
  49a3c866
11 Mar, 2024 1 commit
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba
08 Mar, 2024 1 commit
- Fix auto prefix bug (#3239) · b35cc934
  ElizaWszola authored Mar 08, 2024
  
  b35cc934
05 Mar, 2024 1 commit
- Store `eos_token_id` in `Sequence` for easy access (#3166) · 8999ec3c
  Nick Hill authored Mar 05, 2024
  
  8999ec3c
03 Mar, 2024 1 commit
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark (#3158) · 996d095c
  Zhuohan Li authored Mar 03, 2024
  
  996d095c
02 Mar, 2024 1 commit

Add Automatic Prefix Caching (#2762) · ce4f5a29

Sage Moore authored Mar 02, 2024


Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

ce4f5a29

22 Feb, 2024 1 commit
- chore(vllm): codespell for spell checking (#2820) · 93dc5a28
  Massimiliano Pronesti authored Feb 22, 2024
  
  93dc5a28
21 Feb, 2024 2 commits
- Support per-request seed (#2514) · 7d2dcce1
  Nick Hill authored Feb 21, 2024
  
  7d2dcce1
- Add metrics to RequestOutput (#2876) · 017d9f15
  Antoni Baum authored Feb 20, 2024
  
  017d9f15
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

21 Jan, 2024 1 commit
- [Fix] Keep `scheduler.running` as deque (#2523) · d75c4073
  Nick Hill authored Jan 20, 2024
  
  d75c4073
18 Jan, 2024 2 commits
- [BugFix] Fix abort_seq_group (#2463) · d2a68364
  ljss authored Jan 19, 2024
  
  d2a68364
- [Experimental] Prefix Caching Support (#1669) · d10f8e1d
  shiyi.c_98 authored Jan 17, 2024
```
Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  d10f8e1d
12 Jan, 2024 2 commits
- fix: deque mutated during iteration in abort_seq_group (#2371) · 48cf1e41
  陈序 authored Jan 13, 2024
  
  48cf1e41
- [DOC] Add additional comments for LLMEngine and AsyncLLMEngine (#1011) · 6549aef2
  Jiaxiang authored Jan 12, 2024
  
  6549aef2
07 Jan, 2024 1 commit
- Changed scheduler to use deques instead of lists (#2290) · 05921a9a
  Nadav Shmayovits authored Jan 07, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05921a9a
21 Dec, 2023 1 commit
- [BugFix] Fix recovery logic for sequence group (#2186) · a1b9cb2a
  Woosuk Kwon authored Dec 20, 2023
  
  a1b9cb2a
29 Nov, 2023 2 commits
- [FIX] Fix formatting error · 1cb4ad8d
  Zhuohan Li authored Nov 29, 2023
  
  1cb4ad8d
- Use the type BlockTable (#1791) · 6ed068a7
  explainerauthors authored Nov 28, 2023
  
  6ed068a7