Commits · 55f8b0a5def22ed6b85d3b91b726a7573d54313b · norm / vllm

11 May, 2023 1 commit
- Implement presence and frequency penalties (#95) · 55f8b0a5
  Woosuk Kwon authored May 10, 2023
  
  55f8b0a5
10 May, 2023 4 commits
- Support top-k sampling (#94) · 9f88db35
  Woosuk Kwon authored May 10, 2023
  
  9f88db35
- Avoid sorting waiting queue & Minor code cleaning (#93) · ae356774
  Woosuk Kwon authored May 10, 2023
  
  ae356774
- Log system stats (#90) · e3319577
  Woosuk Kwon authored May 10, 2023
  
  e3319577
- Rename variables and methods (#91) · 8d66a7b6
  Woosuk Kwon authored May 10, 2023
  
  8d66a7b6
09 May, 2023 5 commits
- Update sample prompts in `simple_server.py` (#89) · ce26e57f
  Woosuk Kwon authored May 09, 2023
  
  ce26e57f
- Use slow tokenizer for LLaMA (#84) · 85eb6318
  Woosuk Kwon authored May 09, 2023
  
  85eb6318
- Enhance model loader (#83) · add055e1
  Woosuk Kwon authored May 09, 2023
  
  add055e1
- Refactor system architecture (#82) · 7c041ab5
  Woosuk Kwon authored May 09, 2023
  
  7c041ab5
- Add a system logger (#85) · 8917782a
  Woosuk Kwon authored May 08, 2023
  
  8917782a
07 May, 2023 1 commit
- Specify python package dependencies in requirements.txt (#78) · 7addca59
  Woosuk Kwon authored May 07, 2023
  
  7addca59
06 May, 2023 1 commit
- [Minor] Fix a dtype bug (#79) · c84e9242
  Woosuk Kwon authored May 06, 2023
  
  c84e9242
05 May, 2023 1 commit
- Replace FlashAttention with xformers (#70) · c9d5b6d4
  Woosuk Kwon authored May 05, 2023
  
  c9d5b6d4
04 May, 2023 3 commits
- Use dtype from model config & Add Dolly V2 (#63) · 189ae231
  Woosuk Kwon authored May 04, 2023
  
  189ae231
- Add support for GPT-2 (#60) · e548c148
  Woosuk Kwon authored May 04, 2023
  
  e548c148
- Fix a bug in attention kernel (#68) · 130d5fd8
  Woosuk Kwon authored May 04, 2023
  
  130d5fd8
03 May, 2023 3 commits
- Support bfloat16 data type (#54) · e070829a
  Woosuk Kwon authored May 03, 2023
  
  e070829a
- Refactor attention kernels (#53) · 436e523b
  Woosuk Kwon authored May 03, 2023
  
  436e523b
- New weight loader without np copy (#52) · 27f1410d
  Zhuohan Li authored May 03, 2023
  
  27f1410d
30 Apr, 2023 1 commit
- Add an option to launch cacheflow without ray (#51) · 4858f3bb
  Zhuohan Li authored Apr 30, 2023
  
  4858f3bb
28 Apr, 2023 1 commit
- Add support for GPT-NeoX (Pythia) (#50) · a96d63c2
  Woosuk Kwon authored Apr 28, 2023
  
  a96d63c2
17 Apr, 2023 1 commit
- Change plotting script · aa50b17c
  Woosuk Kwon authored Apr 17, 2023
  
  aa50b17c
15 Apr, 2023 1 commit
- Support various block sizes & Change default block size to 16 (#38) · 0f4b3219
  Woosuk Kwon authored Apr 15, 2023
  
  0f4b3219
12 Apr, 2023 1 commit
- Collect system stats in scheduler & Add scripts for experiments (#30) · 84eee24e
  Woosuk Kwon authored Apr 12, 2023
  
  84eee24e
11 Apr, 2023 1 commit
- Memcpy kernel for flash attention (#29) · e3cec88a
  Siyuan (Ryans) Zhuang authored Apr 10, 2023
```
* optimize

* add benchmark

* add assert

* add test
```
  e3cec88a
10 Apr, 2023 1 commit
- Support block size 32 (#35) · b9926f7f
  Woosuk Kwon authored Apr 09, 2023
  
  b9926f7f
09 Apr, 2023 1 commit
- Add an option to use dummy model weights (#33) · ee88a7e5
  Woosuk Kwon authored Apr 08, 2023
  
  ee88a7e5
08 Apr, 2023 2 commits
- Add query stride to multi_query_cached_kv_attention & Add kernel benchmark script (#27) · c267b1a0
  Woosuk Kwon authored Apr 08, 2023
```
* Add query stride to multi_query_cached_kv_attention

* Add kernel benchmark script
```
  c267b1a0
- Implement block copy kernel to optimize beam search (#32) · 0f40557a
  Woosuk Kwon authored Apr 07, 2023
  
  0f40557a
06 Apr, 2023 1 commit
- Fix potential bugs in FastAPI frontend and add comments (#28) · a490aafa
  Zhuohan Li authored Apr 06, 2023
  
  a490aafa
05 Apr, 2023 2 commits
- Add CUDA graph-based all reduce launcher (#26) · 12659a0b
  Woosuk Kwon authored Apr 05, 2023
  
  12659a0b
- Basic attention kernel that supports cached KV + (multi-)prompts (#24) · 21b3671b
  Siyuan (Ryans) Zhuang authored Apr 04, 2023
  
  21b3671b
02 Apr, 2023 3 commits
- Optimize data movement (#20) · 897cb2ae
  Woosuk Kwon authored Apr 02, 2023
  
  897cb2ae
- Merge QKV into one linear layer (#15) · 1f01a18d
  Zhuohan Li authored Apr 02, 2023
  
  1f01a18d
- Add ninja to dependency (#21) · 2c5cd0de
  Woosuk Kwon authored Apr 01, 2023
  
  2c5cd0de
01 Apr, 2023 1 commit
- Use FP32 for log probabilities (#19) · a90c97d7
  Woosuk Kwon authored Mar 31, 2023
  
  a90c97d7
31 Mar, 2023 3 commits
- Modify README to include info on loading LLaMA (#18) · e3f00d19
  Zhuohan Li authored Apr 01, 2023
  
  e3f00d19
- Add custom kernel for RMS normalization (#16) · 09e92454
  Woosuk Kwon authored Mar 31, 2023
  
  09e92454
- Optimize tensor parallel execution speed (#17) · c45f3c3a
  Zhuohan Li authored Apr 01, 2023
  
  c45f3c3a
30 Mar, 2023 1 commit
- Implement preemption via recomputation & Refactor scheduling logic (#12) · 7a7929ab
  Woosuk Kwon authored Mar 30, 2023
  
  7a7929ab