Commits · beeee69bc92980c0cf70f9a733522641da0d08ec · norm / vllm

06 Dec, 2023 1 commit
- Save pytorch profiler output for latency benchmark (#1871) · 05ff90b6
  Antoni Baum authored Dec 05, 2023
```
* Save profiler output

* Apply feedback from code review
```
  05ff90b6
30 Nov, 2023 2 commits
- Remove max_num_seqs in latency benchmark script (#1855) · 51d3cb95
  Woosuk Kwon authored Nov 30, 2023
  
  51d3cb95
- Add profile option to latency benchmark script (#1839) · e74b1736
  Woosuk Kwon authored Nov 29, 2023
  
  e74b1736
22 Oct, 2023 1 commit

chooper1 authored Oct 22, 2023


Co-authored-by: squeeze-ai-lab <squeezeailab.bair@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

1f24755b

03 Oct, 2023 1 commit
- Use monotonic time where appropriate (#1249) · acbed3ef
  Antoni Baum authored Oct 02, 2023
  
  acbed3ef
01 Oct, 2023 1 commit
- Added `dtype` arg to benchmarks (#1228) · b5a10eb0
  kg6-sleipnir authored Oct 01, 2023
  
  b5a10eb0
16 Sep, 2023 1 commit

Implement AWQ quantization support for LLaMA (#1032) · e3e79e9e

Woosuk Kwon authored Sep 16, 2023


Co-authored-by: Robert Irvine <robert@seamlessml.com>
Co-authored-by: root <rirv938@gmail.com>
Co-authored-by: Casper <casperbh.96@gmail.com>
Co-authored-by: julian-q <julianhquevedo@gmail.com>

e3e79e9e

20 Jul, 2023 1 commit
- fix: enable trust-remote-code in api server & benchmark. (#509) · 8c4b2592
  Ricardo Lu authored Jul 20, 2023
  
  8c4b2592
28 Jun, 2023 1 commit
- [Tokenizer] Add an option to specify tokenizer (#284) · 4338cc47
  Woosuk Kwon authored Jun 28, 2023
  
  4338cc47
17 Jun, 2023 2 commits
- Change the name to vLLM (#150) · 0b98ba15
  Woosuk Kwon authored Jun 17, 2023
  
  0b98ba15
- Rename servers to engines (#152) · e5464ee4
  Zhuohan Li authored Jun 17, 2023
  
  e5464ee4
15 Jun, 2023 1 commit
- Add script for benchmarking serving throughput (#145) · 311490a7
  Woosuk Kwon authored Jun 14, 2023
  
  311490a7
04 Jun, 2023 1 commit
- Add docstrings for LLM (#137) · 8274ca23
  Woosuk Kwon authored Jun 04, 2023
  
  8274ca23
28 May, 2023 1 commit
- Add throughput benchmarking script (#133) · 211318d4
  Woosuk Kwon authored May 28, 2023
  
  211318d4
23 May, 2023 1 commit
- Fix latency benchmark script (#118) · 3f942acf
  Woosuk Kwon authored May 22, 2023
  
  3f942acf
11 May, 2023 1 commit
- Enhance SamplingParams (#96) · 42f1042e
  Woosuk Kwon authored May 11, 2023
  
  42f1042e
03 May, 2023 1 commit
- New weight loader without np copy (#52) · 27f1410d
  Zhuohan Li authored May 03, 2023
  
  27f1410d
30 Apr, 2023 1 commit
- Add an option to launch cacheflow without ray (#51) · 4858f3bb
  Zhuohan Li authored Apr 30, 2023
  
  4858f3bb
12 Apr, 2023 1 commit
- Collect system stats in scheduler & Add scripts for experiments (#30) · 84eee24e
  Woosuk Kwon authored Apr 12, 2023
  
  84eee24e
09 Apr, 2023 1 commit
- Add an option to use dummy model weights (#33) · ee88a7e5
  Woosuk Kwon authored Apr 08, 2023
  
  ee88a7e5
08 Apr, 2023 1 commit
- Implement block copy kernel to optimize beam search (#32) · 0f40557a
  Woosuk Kwon authored Apr 07, 2023
  
  0f40557a
05 Apr, 2023 1 commit
- Add CUDA graph-based all reduce launcher (#26) · 12659a0b
  Woosuk Kwon authored Apr 05, 2023
  
  12659a0b
31 Mar, 2023 1 commit
- Optimize tensor parallel execution speed (#17) · c45f3c3a
  Zhuohan Li authored Apr 01, 2023
  
  c45f3c3a
29 Mar, 2023 1 commit
- FastAPI-based working frontend (#10) · 721fa3df
  Zhuohan Li authored Mar 29, 2023
  
  721fa3df