Commits · 3f942acfe15de367931a63aa96c1931eb74799c0 · norm / vllm

18 May, 2023 1 commit
- Use pytest format for unit tests (#107) · 825d8892
  Woosuk Kwon authored May 17, 2023
  
  825d8892
05 May, 2023 1 commit
- Replace FlashAttention with xformers (#70) · c9d5b6d4
  Woosuk Kwon authored May 05, 2023
  
  c9d5b6d4
03 May, 2023 1 commit
- Refactor attention kernels (#53) · 436e523b
  Woosuk Kwon authored May 03, 2023
  
  436e523b
28 Apr, 2023 1 commit
- Add support for GPT-NeoX (Pythia) (#50) · a96d63c2
  Woosuk Kwon authored Apr 28, 2023
  
  a96d63c2
11 Apr, 2023 1 commit
- Memcpy kernel for flash attention (#29) · e3cec88a
  Siyuan (Ryans) Zhuang authored Apr 10, 2023
```
* optimize

* add benchmark

* add assert

* add test
```
  e3cec88a
10 Apr, 2023 1 commit
- Support block size 32 (#35) · b9926f7f
  Woosuk Kwon authored Apr 09, 2023
  
  b9926f7f
08 Apr, 2023 2 commits
- Add query stride to multi_query_cached_kv_attention & Add kernel benchmark script (#27) · c267b1a0
  Woosuk Kwon authored Apr 08, 2023
```
* Add query stride to multi_query_cached_kv_attention

* Add kernel benchmark script
```
  c267b1a0
- Implement block copy kernel to optimize beam search (#32) · 0f40557a
  Woosuk Kwon authored Apr 07, 2023
  
  0f40557a
05 Apr, 2023 1 commit
- Basic attention kernel that supports cached KV + (multi-)prompts (#24) · 21b3671b
  Siyuan (Ryans) Zhuang authored Apr 04, 2023
  
  21b3671b
02 Apr, 2023 1 commit
- Optimize data movement (#20) · 897cb2ae
  Woosuk Kwon authored Apr 02, 2023
  
  897cb2ae
31 Mar, 2023 1 commit
- Add custom kernel for RMS normalization (#16) · 09e92454
  Woosuk Kwon authored Mar 31, 2023
  
  09e92454
30 Mar, 2023 2 commits
- Implement custom kernel for LLaMA rotary embedding (#14) · 88c0268a
  Woosuk Kwon authored Mar 30, 2023
  
  88c0268a
- Refactor the test code for attention kernels (#13) · a1b3de86
  Woosuk Kwon authored Mar 29, 2023
  
  a1b3de86
02 Mar, 2023 1 commit
- Use FlashAttention for `multi_query_kv_attention` (#4) · 3e9f991d
  Woosuk Kwon authored Mar 01, 2023
  
  3e9f991d
01 Mar, 2023 1 commit
- Implement `single_query_cached_kv_attention` kernel (#3) · 0deacbce
  Woosuk Kwon authored Mar 01, 2023
  
  0deacbce
18 Feb, 2023 1 commit
- Add tests for kernels · af68ec1c
  Woosuk Kwon authored Feb 18, 2023
  
  af68ec1c