Commits · 996d095c541e1cd67f0a7ec2579bc3bb0a435494 · OpenDAS / vllm_cscc

03 Mar, 2024 1 commit
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark (#3158) · 996d095c
  Zhuohan Li authored Mar 03, 2024
  
  996d095c
02 Mar, 2024 1 commit

Add Automatic Prefix Caching (#2762) · ce4f5a29

Sage Moore authored Mar 02, 2024


Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

ce4f5a29

21 Feb, 2024 2 commits
- Support per-request seed (#2514) · 7d2dcce1
  Nick Hill authored Feb 21, 2024
  
  7d2dcce1
- Add metrics to RequestOutput (#2876) · 017d9f15
  Antoni Baum authored Feb 20, 2024
  
  017d9f15
01 Feb, 2024 1 commit
- Fix default length_penalty to 1.0 (#2667) · 0e163fce
  zspo authored Feb 02, 2024
  
  0e163fce
31 Jan, 2024 1 commit
- Refactor Prometheus and Add Request Level Metrics (#2316) · 93b38bea
  Robert Shaw authored Jan 31, 2024
  
  93b38bea
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

18 Jan, 2024 2 commits
- fix: fix some args desc (#2487) · 4df417d0
  zspo authored Jan 19, 2024
  
  4df417d0
- [Experimental] Prefix Caching Support (#1669) · d10f8e1d
  shiyi.c_98 authored Jan 17, 2024
```
Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  d10f8e1d
28 Nov, 2023 1 commit
- [FIX] Fix class naming (#1803) · 708e6c18
  Zhuohan Li authored Nov 28, 2023
  
  708e6c18
17 Oct, 2023 1 commit
- [BugFix] Define `__eq__` in SequenceGroupOutputs (#1389) · f8a1e39f
  Woosuk Kwon authored Oct 17, 2023
  
  f8a1e39f
16 Oct, 2023 1 commit
- Implement prompt logprobs & Batched topk for computing logprobs (#1328) · 9d9072a0
  Zhuohan Li authored Oct 16, 2023
```
Co-authored-by: Yunmo Chen <16273544+wanmok@users.noreply.github.com>
```
  9d9072a0
10 Oct, 2023 1 commit
- Fix `__repr__` of `SequenceOutputs` (#1311) · ac5cf86a
  Wang Ran (汪然) authored Oct 11, 2023
  
  ac5cf86a
08 Oct, 2023 1 commit
- [FIX] Explain why the finished_reason of ignored sequences are length (#1289) · 6b5296aa
  Zhuohan Li authored Oct 08, 2023
  
  6b5296aa
18 Sep, 2023 1 commit
- Fix get_max_num_running_seqs for waiting and swapped seq groups (#1068) · f029ef94
  Zhuohan Li authored Sep 18, 2023
  
  f029ef94
13 Sep, 2023 2 commits
- [FIX] Minor bug fixes (#1035) · f04908ca
  Zhuohan Li authored Sep 13, 2023
```
* [FIX] Minor bug fixes

* Address review comments
```
  f04908ca
- Use TGI-like incremental detokenization (#984) · 9841d48a
  Antoni Baum authored Sep 13, 2023
  
  9841d48a
05 Sep, 2023 1 commit
- Align vLLM's beam search implementation with HF generate (#857) · 002800f0
  Zhuohan Li authored Sep 04, 2023
  
  002800f0
08 Jul, 2023 1 commit
- avoid python list copy in sequence initialization (#401) · 2179e4f4
  Lily Liu authored Jul 08, 2023
  
  2179e4f4
03 Jul, 2023 1 commit
- [Quality] Add code formatter and linter (#326) · d6fa1be3
  Zhuohan Li authored Jul 03, 2023
  
  d6fa1be3
01 Jul, 2023 1 commit
- Raise error for long prompt (#273) · dafd924c
  Lily Liu authored Jun 30, 2023
  
  dafd924c
17 Jun, 2023 1 commit
- Change the name to vLLM (#150) · 0b98ba15
  Woosuk Kwon authored Jun 17, 2023
  
  0b98ba15
05 Jun, 2023 1 commit
- Fix various issues of async servers (#135) · 1a956e13
  Zhuohan Li authored Jun 05, 2023
  
  1a956e13
24 May, 2023 3 commits
- OpenAI Compatible Frontend (#116) · 057daef7
  Zhuohan Li authored May 23, 2023
  
  057daef7
- Incrementally decode output tokens (#121) · e8671783
  Woosuk Kwon authored May 23, 2023
  
  e8671783
- Add contributing guideline and mypy config (#122) · a283ec2e
  Woosuk Kwon authored May 23, 2023
  
  a283ec2e
21 May, 2023 1 commit
- Implement stop strings and best_of (#114) · f746ced0
  Woosuk Kwon authored May 21, 2023
  
  f746ced0
20 May, 2023 1 commit
- Refactor system architecture (#109) · c3442c1f
  Woosuk Kwon authored May 20, 2023
  
  c3442c1f
11 May, 2023 1 commit
- Implement presence and frequency penalties (#95) · 55f8b0a5
  Woosuk Kwon authored May 10, 2023
  
  55f8b0a5
10 May, 2023 1 commit
- Rename variables and methods (#91) · 8d66a7b6
  Woosuk Kwon authored May 10, 2023
  
  8d66a7b6
12 Apr, 2023 1 commit
- Collect system stats in scheduler & Add scripts for experiments (#30) · 84eee24e
  Woosuk Kwon authored Apr 12, 2023
  
  84eee24e
30 Mar, 2023 1 commit
- Implement preemption via recomputation & Refactor scheduling logic (#12) · 7a7929ab
  Woosuk Kwon authored Mar 30, 2023
  
  7a7929ab
26 Mar, 2023 1 commit
- Minor · d359cda5
  Woosuk Kwon authored Mar 26, 2023
  
  d359cda5
21 Mar, 2023 1 commit
- Support tensor parallel (#2) · 2f49f155
  Zhuohan Li authored Mar 22, 2023
  
  2f49f155
10 Mar, 2023 1 commit
- Support beam search & parallel generation (#7) · 1a7eb7da
  Woosuk Kwon authored Mar 10, 2023
  
  1a7eb7da
24 Feb, 2023 1 commit
- Add is_finished · b39f149a
  Woosuk Kwon authored Feb 24, 2023
  
  b39f149a
23 Feb, 2023 2 commits
- Add get_len · af16c050
  Woosuk Kwon authored Feb 23, 2023
  
  af16c050
- Move max_context_len · d0945122
  Woosuk Kwon authored Feb 23, 2023
  
  d0945122
14 Feb, 2023 1 commit
- Add __repr__ · 3363c27d
  Woosuk Kwon authored Feb 14, 2023
  
  3363c27d
13 Feb, 2023 1 commit
- Add find method to sequence group · 0961f5a4
  Woosuk Kwon authored Feb 13, 2023
  
  0961f5a4