Commits · 18de88348954b7e535a62c0b7e55004f253e9f21 · OpenDAS / vllm_cscc

05 Apr, 2024 1 commit
- [Chunked Prefill][4/n] Chunked prefill scheduler. (#3853) · 18de8834
  SangBin Cho authored Apr 06, 2024
  
  18de8834
03 Apr, 2024 1 commit
- [3/N] Refactor scheduler for chunked prefill scheduling (#3550) · 3dcb3e8b
  SangBin Cho authored Apr 04, 2024
  
  3dcb3e8b
02 Apr, 2024 1 commit
- [Bugfix] Add `__init__.py` files for `vllm/core/block/` and `vllm/spec_decode/` (#3798) · b321d488
  Michael Goin authored Apr 02, 2024
  
  b321d488
01 Apr, 2024 1 commit
- [Speculative decoding 4/9] Lookahead scheduling for speculative decoding (#3250) · 93deb0b3
  Cade Daniel authored Apr 01, 2024
  
  93deb0b3
28 Mar, 2024 3 commits
- fix logging msg for block manager (#3701) · 4716a32d
  Simon Mo authored Mar 28, 2024
  
  4716a32d
- [2/N] Chunked prefill data update (#3538) · b51c1cc9
  SangBin Cho authored Mar 29, 2024
  
  b51c1cc9
- [Core][Bugfix]Refactor block manager for better testability (#3492) · 14ccd94c
  Cade Daniel authored Mar 27, 2024
  
  14ccd94c
25 Mar, 2024 3 commits
- [Feature] Add vision language model support. (#3042) · 64172a97
  xwjiang2010 authored Mar 25, 2024
  
  64172a97
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
- [Bugfix] fix automatic prefix args and add log info (#3608) · e67c295b
  TianYu GUO authored Mar 25, 2024
  
  e67c295b
22 Mar, 2024 1 commit
- Dynamic scheduler delay to improve ITL performance (#3279) · cf2f084d
  Thomas Parnell authored Mar 22, 2024
```
Co-authored-by: Jan van Lunteren <jvl@zurich.ibm.com>
```
  cf2f084d
21 Mar, 2024 1 commit
- [PREFIX CACHING FOLLOW UP] OrderedDict-based evictor (#3431) · 6ebd02bd
  ElizaWszola authored Mar 21, 2024
```
Co-authored-by: rsnm2 <rshaw@neuralmagic.com>
Co-authored-by: Luka <luka@paperspace>
```
  6ebd02bd
20 Mar, 2024 2 commits
- [1/n][Chunked Prefill] Refactor input query shapes (#3236) · 6e435de7
  SangBin Cho authored Mar 21, 2024
  
  6e435de7
- [PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance... · 9474e89b
  ElizaWszola authored Mar 20, 2024
```
[PREFIX CACHING FOLLOW UP] A bunch of fixes to block allocator performance when automatic prefix caching is disabled (#3357)
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9474e89b
15 Mar, 2024 1 commit
- Fixes the misuse/mixuse of time.time()/time.monotonic() (#3220) · 14b8ae02
  Tao He authored Mar 16, 2024
```
Signed-off-by: Tao He <sighingnow@gmail.com>
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  14b8ae02
13 Mar, 2024 1 commit
- Fixes #1556 double free (#3347) · 49a3c866
  Breno Faria authored Mar 13, 2024
  
  49a3c866
11 Mar, 2024 1 commit
- Re-enable the 80 char line width limit (#3305) · 2f8844ba
  Zhuohan Li authored Mar 10, 2024
  
  2f8844ba
08 Mar, 2024 1 commit
- Fix auto prefix bug (#3239) · b35cc934
  ElizaWszola authored Mar 08, 2024
  
  b35cc934
05 Mar, 2024 1 commit
- Store `eos_token_id` in `Sequence` for easy access (#3166) · 8999ec3c
  Nick Hill authored Mar 05, 2024
  
  8999ec3c
03 Mar, 2024 1 commit
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark (#3158) · 996d095c
  Zhuohan Li authored Mar 03, 2024
  
  996d095c
02 Mar, 2024 1 commit

Add Automatic Prefix Caching (#2762) · ce4f5a29

Sage Moore authored Mar 02, 2024


Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

ce4f5a29

22 Feb, 2024 1 commit
- chore(vllm): codespell for spell checking (#2820) · 93dc5a28
  Massimiliano Pronesti authored Feb 22, 2024
  
  93dc5a28
21 Feb, 2024 2 commits
- Support per-request seed (#2514) · 7d2dcce1
  Nick Hill authored Feb 21, 2024
  
  7d2dcce1
- Add metrics to RequestOutput (#2876) · 017d9f15
  Antoni Baum authored Feb 20, 2024
  
  017d9f15
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

21 Jan, 2024 1 commit
- [Fix] Keep `scheduler.running` as deque (#2523) · d75c4073
  Nick Hill authored Jan 20, 2024
  
  d75c4073
18 Jan, 2024 2 commits
- [BugFix] Fix abort_seq_group (#2463) · d2a68364
  ljss authored Jan 19, 2024
  
  d2a68364
- [Experimental] Prefix Caching Support (#1669) · d10f8e1d
  shiyi.c_98 authored Jan 17, 2024
```
Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  d10f8e1d
12 Jan, 2024 2 commits
- fix: deque mutated during iteration in abort_seq_group (#2371) · 48cf1e41
  陈序 authored Jan 13, 2024
  
  48cf1e41
- [DOC] Add additional comments for LLMEngine and AsyncLLMEngine (#1011) · 6549aef2
  Jiaxiang authored Jan 12, 2024
  
  6549aef2
07 Jan, 2024 1 commit
- Changed scheduler to use deques instead of lists (#2290) · 05921a9a
  Nadav Shmayovits authored Jan 07, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  05921a9a
21 Dec, 2023 1 commit
- [BugFix] Fix recovery logic for sequence group (#2186) · a1b9cb2a
  Woosuk Kwon authored Dec 20, 2023
  
  a1b9cb2a
29 Nov, 2023 2 commits
- [FIX] Fix formatting error · 1cb4ad8d
  Zhuohan Li authored Nov 29, 2023
  
  1cb4ad8d
- Use the type BlockTable (#1791) · 6ed068a7
  explainerauthors authored Nov 28, 2023
  
  6ed068a7
21 Nov, 2023 2 commits
- [Fix] Fix bugs in scheduler (#1727) · 19849db5
  Zhuofan authored Nov 21, 2023
  
  19849db5
- Fix hanging in the scheduler caused by long prompts (#1534) · 3d4ceb29
  陈序 authored Nov 21, 2023
  
  3d4ceb29
20 Nov, 2023 1 commit
- Migrate linter from `pylint` to `ruff` (#1665) · 5ffc0d13
  Simon Mo authored Nov 20, 2023
  
  5ffc0d13
20 Oct, 2023 1 commit
- Fix type hints (#1427) · f61dc807
  Light Lin authored Oct 20, 2023
  
  f61dc807
17 Oct, 2023 1 commit
- Change scheduler & input tensor shape (#1381) · c1376e0f
  Woosuk Kwon authored Oct 16, 2023
  
  c1376e0f
03 Oct, 2023 1 commit
- Use monotonic time where appropriate (#1249) · acbed3ef
  Antoni Baum authored Oct 02, 2023
  
  acbed3ef