Commits · db09d4ad833b1d8911f14852a484790c53818a5d · norm / vllm

07 Sep, 2023 1 commit

[FIX] Fix Alibi implementation in PagedAttention kernel (#945) · db09d4ad

Zhuohan Li authored Sep 07, 2023



* [FIX] Fix Alibi implementation in PagedAttention kernel

* Fix test_attention

* Fix

---------
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Oliver-ss <yuansongwx@outlook.com>

db09d4ad

06 Sep, 2023 1 commit
- [BugFix] Implement RoPE for GPT-J (#941) · 320a622e
  Woosuk Kwon authored Sep 06, 2023
  
  320a622e
04 Sep, 2023 1 commit
- [BugFix] Fix NaN errors in paged attention kernel (#936) · bf87484e
  Woosuk Kwon authored Sep 04, 2023
  
  bf87484e
02 Sep, 2023 1 commit
- Avoid compiling kernels for double data type (#933) · 8ce9c50d
  Woosuk Kwon authored Sep 02, 2023
  
  8ce9c50d
22 Aug, 2023 1 commit
- Implement approximate GELU kernels (#828) · d64bf164
  Woosuk Kwon authored Aug 23, 2023
  
  d64bf164
04 Aug, 2023 1 commit
- [OPTIMIZATION] Optimizes the single_query_cached_kv_attention kernel (#420) · 79af7e96
  Dean Leitersdorf authored Aug 04, 2023
  
  79af7e96
02 Aug, 2023 1 commit
- Add Falcon support (new) (#592) · 1b0bd0fe
  Zhuohan Li authored Aug 02, 2023
  
  1b0bd0fe
20 Jul, 2023 1 commit
- Add support for LLaMA-2 (#505) · 6fc2a38b
  Zhuohan Li authored Jul 20, 2023
  
  6fc2a38b
15 Jul, 2023 1 commit
- Optimize MQA Kernel (#452) · 96853af5
  Zhuohan Li authored Jul 14, 2023
  
  96853af5
09 Jul, 2023 1 commit
- [Model] Add support for GPT-J (#226) · c8948361
  Andre Slavescu authored Jul 08, 2023
```
Co-authored-by: woWoosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c8948361
03 Jul, 2023 2 commits
- [Model] Add support for MPT (#334) · 404422f4
  Woosuk Kwon authored Jul 03, 2023
  
  404422f4
- Add support for BLOOM (#331) · e41f0670
  Woosuk Kwon authored Jul 03, 2023
  
  e41f0670
17 Jun, 2023 1 commit
- Change the name to vLLM (#150) · 0b98ba15
  Woosuk Kwon authored Jun 17, 2023
  
  0b98ba15
07 Jun, 2023 1 commit
- Support FP32 (#141) · e38074b1
  Woosuk Kwon authored Jun 07, 2023
  
  e38074b1
27 May, 2023 1 commit
- Improve setup script & Add a guard for bfloat16 kernels (#130) · d7211684
  Woosuk Kwon authored May 27, 2023
  
  d7211684
15 May, 2023 1 commit
- Add copyright headers to source files adapted from FT (#104) · 667ba399
  Woosuk Kwon authored May 14, 2023
  
  667ba399
04 May, 2023 1 commit
- Fix a bug in attention kernel (#68) · 130d5fd8
  Woosuk Kwon authored May 04, 2023
  
  130d5fd8
03 May, 2023 2 commits
- Support bfloat16 data type (#54) · e070829a
  Woosuk Kwon authored May 03, 2023
  
  e070829a
- Refactor attention kernels (#53) · 436e523b
  Woosuk Kwon authored May 03, 2023
  
  436e523b
28 Apr, 2023 1 commit
- Add support for GPT-NeoX (Pythia) (#50) · a96d63c2
  Woosuk Kwon authored Apr 28, 2023
  
  a96d63c2
15 Apr, 2023 1 commit
- Support various block sizes & Change default block size to 16 (#38) · 0f4b3219
  Woosuk Kwon authored Apr 15, 2023
  
  0f4b3219
11 Apr, 2023 1 commit
- Memcpy kernel for flash attention (#29) · e3cec88a
  Siyuan (Ryans) Zhuang authored Apr 10, 2023
```
* optimize

* add benchmark

* add assert

* add test
```
  e3cec88a
10 Apr, 2023 1 commit
- Support block size 32 (#35) · b9926f7f
  Woosuk Kwon authored Apr 09, 2023
  
  b9926f7f
08 Apr, 2023 2 commits
- Add query stride to multi_query_cached_kv_attention & Add kernel benchmark script (#27) · c267b1a0
  Woosuk Kwon authored Apr 08, 2023
```
* Add query stride to multi_query_cached_kv_attention

* Add kernel benchmark script
```
  c267b1a0
- Implement block copy kernel to optimize beam search (#32) · 0f40557a
  Woosuk Kwon authored Apr 07, 2023
  
  0f40557a
05 Apr, 2023 1 commit
- Basic attention kernel that supports cached KV + (multi-)prompts (#24) · 21b3671b
  Siyuan (Ryans) Zhuang authored Apr 04, 2023
  
  21b3671b
02 Apr, 2023 1 commit
- Optimize data movement (#20) · 897cb2ae
  Woosuk Kwon authored Apr 02, 2023
  
  897cb2ae
31 Mar, 2023 1 commit
- Add custom kernel for RMS normalization (#16) · 09e92454
  Woosuk Kwon authored Mar 31, 2023
  
  09e92454
30 Mar, 2023 1 commit
- Implement custom kernel for LLaMA rotary embedding (#14) · 88c0268a
  Woosuk Kwon authored Mar 30, 2023
  
  88c0268a
13 Mar, 2023 1 commit
- Add miscellaneous updates (#8) · cfae35b8
  Woosuk Kwon authored Mar 13, 2023
  
  cfae35b8
10 Mar, 2023 1 commit
- Support beam search & parallel generation (#7) · 1a7eb7da
  Woosuk Kwon authored Mar 10, 2023
  
  1a7eb7da
01 Mar, 2023 1 commit
- Implement `single_query_cached_kv_attention` kernel (#3) · 0deacbce
  Woosuk Kwon authored Mar 01, 2023
  
  0deacbce
18 Feb, 2023 1 commit
- Add reshape_and_cache op · c413c41c
  Woosuk Kwon authored Feb 18, 2023
  
  c413c41c
16 Feb, 2023 3 commits
- cache_kernel -> cache_kernels · ffad4e1e
  Woosuk Kwon authored Feb 16, 2023
  
  ffad4e1e
- Remove redundant fn · 6d2f74ef
  Woosuk Kwon authored Feb 16, 2023
  
  6d2f74ef
- Implement cache ops · 6f058c7b
  Woosuk Kwon authored Feb 16, 2023
  
  6f058c7b