Commits · b7955ef17b8d899327b25564f20665ec3ffa71cb · norm / vllm

19 May, 2023 2 commits
- Fix timeout error in the FastAPI frontend (#34) · b7955ef1
  Zhuohan Li authored May 19, 2023
  
  b7955ef1
- Use runtime profiling to replace manual memory analyzers (#81) · f756799b
  Zhuohan Li authored May 19, 2023
  
  f756799b
15 May, 2023 3 commits
- Add docstrings to some modules and classes (#100) · b322fd16
  Woosuk Kwon authored May 14, 2023
  
  b322fd16
- Add copyright headers to source files adapted from FT (#104) · 667ba399
  Woosuk Kwon authored May 14, 2023
  
  667ba399
- Add copyright headers for HF models (#103) · 707ec647
  Woosuk Kwon authored May 14, 2023
  
  707ec647
13 May, 2023 1 commit
- Minor code cleaning for SamplingParams (#99) · 6208d622
  Woosuk Kwon authored May 12, 2023
  
  6208d622
11 May, 2023 2 commits
- Enhance SamplingParams (#96) · 42f1042e
  Woosuk Kwon authored May 11, 2023
  
  42f1042e
- Implement presence and frequency penalties (#95) · 55f8b0a5
  Woosuk Kwon authored May 10, 2023
  
  55f8b0a5
10 May, 2023 4 commits
- Support top-k sampling (#94) · 9f88db35
  Woosuk Kwon authored May 10, 2023
  
  9f88db35
- Avoid sorting waiting queue & Minor code cleaning (#93) · ae356774
  Woosuk Kwon authored May 10, 2023
  
  ae356774
- Log system stats (#90) · e3319577
  Woosuk Kwon authored May 10, 2023
  
  e3319577
- Rename variables and methods (#91) · 8d66a7b6
  Woosuk Kwon authored May 10, 2023
  
  8d66a7b6
09 May, 2023 4 commits
- Use slow tokenizer for LLaMA (#84) · 85eb6318
  Woosuk Kwon authored May 09, 2023
  
  85eb6318
- Enhance model loader (#83) · add055e1
  Woosuk Kwon authored May 09, 2023
  
  add055e1
- Refactor system architecture (#82) · 7c041ab5
  Woosuk Kwon authored May 09, 2023
  
  7c041ab5
- Add a system logger (#85) · 8917782a
  Woosuk Kwon authored May 08, 2023
  
  8917782a
06 May, 2023 1 commit
- [Minor] Fix a dtype bug (#79) · c84e9242
  Woosuk Kwon authored May 06, 2023
  
  c84e9242
05 May, 2023 1 commit
- Replace FlashAttention with xformers (#70) · c9d5b6d4
  Woosuk Kwon authored May 05, 2023
  
  c9d5b6d4
04 May, 2023 2 commits
- Use dtype from model config & Add Dolly V2 (#63) · 189ae231
  Woosuk Kwon authored May 04, 2023
  
  189ae231
- Add support for GPT-2 (#60) · e548c148
  Woosuk Kwon authored May 04, 2023
  
  e548c148
03 May, 2023 2 commits
- Support bfloat16 data type (#54) · e070829a
  Woosuk Kwon authored May 03, 2023
  
  e070829a
- New weight loader without np copy (#52) · 27f1410d
  Zhuohan Li authored May 03, 2023
  
  27f1410d
30 Apr, 2023 1 commit
- Add an option to launch cacheflow without ray (#51) · 4858f3bb
  Zhuohan Li authored Apr 30, 2023
  
  4858f3bb
28 Apr, 2023 1 commit
- Add support for GPT-NeoX (Pythia) (#50) · a96d63c2
  Woosuk Kwon authored Apr 28, 2023
  
  a96d63c2
15 Apr, 2023 1 commit
- Support various block sizes & Change default block size to 16 (#38) · 0f4b3219
  Woosuk Kwon authored Apr 15, 2023
  
  0f4b3219
12 Apr, 2023 1 commit
- Collect system stats in scheduler & Add scripts for experiments (#30) · 84eee24e
  Woosuk Kwon authored Apr 12, 2023
  
  84eee24e
10 Apr, 2023 1 commit
- Support block size 32 (#35) · b9926f7f
  Woosuk Kwon authored Apr 09, 2023
  
  b9926f7f
09 Apr, 2023 1 commit
- Add an option to use dummy model weights (#33) · ee88a7e5
  Woosuk Kwon authored Apr 08, 2023
  
  ee88a7e5
08 Apr, 2023 1 commit
- Implement block copy kernel to optimize beam search (#32) · 0f40557a
  Woosuk Kwon authored Apr 07, 2023
  
  0f40557a
06 Apr, 2023 1 commit
- Fix potential bugs in FastAPI frontend and add comments (#28) · a490aafa
  Zhuohan Li authored Apr 06, 2023
  
  a490aafa
05 Apr, 2023 1 commit
- Add CUDA graph-based all reduce launcher (#26) · 12659a0b
  Woosuk Kwon authored Apr 05, 2023
  
  12659a0b
02 Apr, 2023 2 commits
- Optimize data movement (#20) · 897cb2ae
  Woosuk Kwon authored Apr 02, 2023
  
  897cb2ae
- Merge QKV into one linear layer (#15) · 1f01a18d
  Zhuohan Li authored Apr 02, 2023
  
  1f01a18d
01 Apr, 2023 1 commit
- Use FP32 for log probabilities (#19) · a90c97d7
  Woosuk Kwon authored Mar 31, 2023
  
  a90c97d7
31 Mar, 2023 2 commits
- Add custom kernel for RMS normalization (#16) · 09e92454
  Woosuk Kwon authored Mar 31, 2023
  
  09e92454
- Optimize tensor parallel execution speed (#17) · c45f3c3a
  Zhuohan Li authored Apr 01, 2023
  
  c45f3c3a
30 Mar, 2023 3 commits
- Implement preemption via recomputation & Refactor scheduling logic (#12) · 7a7929ab
  Woosuk Kwon authored Mar 30, 2023
  
  7a7929ab
- Implement custom kernel for LLaMA rotary embedding (#14) · 88c0268a
  Woosuk Kwon authored Mar 30, 2023
  
  88c0268a
- Implement LLaMA (#9) · 80a2f812
  Woosuk Kwon authored Mar 29, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  80a2f812
29 Mar, 2023 1 commit
- Add cache watermark to avoid frequent cache eviction (#11) · 64e0e383
  Woosuk Kwon authored Mar 29, 2023
  
  64e0e383