Commits · d10f8e1d43bfb0656b6848ad0c681ecbdec812d6 · norm / vllm

18 Jan, 2024 1 commit

[Experimental] Prefix Caching Support (#1669) · d10f8e1d

shiyi.c_98 authored Jan 17, 2024


Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

d10f8e1d

14 Jan, 2024 1 commit
- [CI] Add Buildkite (#2355) · 6e01e8c1
  Simon Mo authored Jan 14, 2024
  
  6e01e8c1
04 Jan, 2024 1 commit
- Revert the changes in test_cache (#2335) · 94176712
  Woosuk Kwon authored Jan 03, 2024
  
  94176712
03 Jan, 2024 2 commits
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
- [FIX] Support non-zero CUDA devices in custom kernels (#1959) · 77af974b
  Jee Li authored Jan 03, 2024
  
  77af974b
10 Dec, 2023 1 commit
- Replace head_mapping params with num_kv_heads to attention kernel. (#1997) · dacaf5a4
  wbn authored Dec 11, 2023
```
Co-authored-by: wangguoya <wangguoya@baidu.com>
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com>
```
  dacaf5a4
03 Dec, 2023 1 commit
- Add PyTorch-native implementation of custom layers (#1898) · 9b294976
  Woosuk Kwon authored Dec 02, 2023
  
  9b294976
24 Nov, 2023 1 commit
- [Build] Avoid building too many extensions (#1624) · e0c6f556
  Yanming W authored Nov 23, 2023
  
  e0c6f556
20 Nov, 2023 1 commit
- Migrate linter from `pylint` to `ruff` (#1665) · 5ffc0d13
  Simon Mo authored Nov 20, 2023
  
  5ffc0d13
31 Oct, 2023 1 commit
- Fix integer overflows in attention & cache ops (#1514) · 0ce8647d
  Woosuk Kwon authored Oct 31, 2023
  
  0ce8647d
16 Oct, 2023 1 commit
- Implement PagedAttention V2 (#1348) · 928de468
  Woosuk Kwon authored Oct 16, 2023
  
  928de468
02 Oct, 2023 1 commit
- TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic (#1181) · ba0bfd40
  Zhuohan Li authored Oct 02, 2023
  
  ba0bfd40
28 Sep, 2023 1 commit
- Fix OOM in attention kernel test (#1223) · 6f88f762
  Woosuk Kwon authored Sep 28, 2023
  
  6f88f762
27 Sep, 2023 1 commit
- Allocate more shared memory to attention kernel (#1154) · cf5cb1e3
  Antoni Baum authored Sep 26, 2023
  
  cf5cb1e3
11 Sep, 2023 1 commit
- Use FP32 in RoPE initialization (#1004) · e67b4f2c
  Woosuk Kwon authored Sep 11, 2023
```
Co-authored-by: One <imone@tuta.io>
```
  e67b4f2c
07 Sep, 2023 1 commit

[FIX] Fix Alibi implementation in PagedAttention kernel (#945) · db09d4ad

Zhuohan Li authored Sep 07, 2023



* [FIX] Fix Alibi implementation in PagedAttention kernel

* Fix test_attention

* Fix

---------
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Oliver-ss <yuansongwx@outlook.com>

db09d4ad

06 Sep, 2023 1 commit
- [BugFix] Implement RoPE for GPT-J (#941) · 320a622e
  Woosuk Kwon authored Sep 06, 2023
  
  320a622e
05 Sep, 2023 1 commit
- Clean up kernel unit tests (#938) · fbd80ad4
  Woosuk Kwon authored Sep 06, 2023
  
  fbd80ad4
30 Aug, 2023 1 commit
- use flash-attn via xformers (#877) · 75471386
  Aman Gupta Karmani authored Aug 30, 2023
  
  75471386
22 Aug, 2023 1 commit
- Implement approximate GELU kernels (#828) · d64bf164
  Woosuk Kwon authored Aug 23, 2023
  
  d64bf164
25 Jul, 2023 1 commit
- Fix paged attention testing. (#495) · d7a1c6d6
  Tao Peng authored Jul 25, 2023
```
Signed-off-by: Tao Peng <jiankeng.pt@alibaba-inc.com>
```
  d7a1c6d6
18 Jul, 2023 1 commit
- hotfix attn alibi wo head mapping (#496) · bda41c70
  Song authored Jul 19, 2023
```
Co-authored-by: oliveryuan <oliveryuan@basemind.com>
```
  bda41c70
09 Jul, 2023 1 commit
- [Model] Add support for GPT-J (#226) · c8948361
  Andre Slavescu authored Jul 08, 2023
```
Co-authored-by: woWoosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c8948361
03 Jul, 2023 2 commits
- Add support for BLOOM (#331) · e41f0670
  Woosuk Kwon authored Jul 03, 2023
  
  e41f0670
- [Quality] Add code formatter and linter (#326) · d6fa1be3
  Zhuohan Li authored Jul 03, 2023
  
  d6fa1be3
17 Jun, 2023 1 commit
- Change the name to vLLM (#150) · 0b98ba15
  Woosuk Kwon authored Jun 17, 2023
  
  0b98ba15
07 Jun, 2023 1 commit
- Support FP32 (#141) · e38074b1
  Woosuk Kwon authored Jun 07, 2023
  
  e38074b1
24 May, 2023 1 commit
- Add contributing guideline and mypy config (#122) · a283ec2e
  Woosuk Kwon authored May 23, 2023
  
  a283ec2e
18 May, 2023 1 commit
- Use pytest format for unit tests (#107) · 825d8892
  Woosuk Kwon authored May 17, 2023
  
  825d8892
05 May, 2023 1 commit
- Replace FlashAttention with xformers (#70) · c9d5b6d4
  Woosuk Kwon authored May 05, 2023
  
  c9d5b6d4
03 May, 2023 1 commit
- Refactor attention kernels (#53) · 436e523b
  Woosuk Kwon authored May 03, 2023
  
  436e523b
28 Apr, 2023 1 commit
- Add support for GPT-NeoX (Pythia) (#50) · a96d63c2
  Woosuk Kwon authored Apr 28, 2023
  
  a96d63c2
11 Apr, 2023 1 commit
- Memcpy kernel for flash attention (#29) · e3cec88a
  Siyuan (Ryans) Zhuang authored Apr 10, 2023
```
* optimize

* add benchmark

* add assert

* add test
```
  e3cec88a
10 Apr, 2023 1 commit
- Support block size 32 (#35) · b9926f7f
  Woosuk Kwon authored Apr 09, 2023
  
  b9926f7f
08 Apr, 2023 2 commits
- Add query stride to multi_query_cached_kv_attention & Add kernel benchmark script (#27) · c267b1a0
  Woosuk Kwon authored Apr 08, 2023
```
* Add query stride to multi_query_cached_kv_attention

* Add kernel benchmark script
```
  c267b1a0
- Implement block copy kernel to optimize beam search (#32) · 0f40557a
  Woosuk Kwon authored Apr 07, 2023
  
  0f40557a
05 Apr, 2023 1 commit
- Basic attention kernel that supports cached KV + (multi-)prompts (#24) · 21b3671b
  Siyuan (Ryans) Zhuang authored Apr 04, 2023
  
  21b3671b
02 Apr, 2023 1 commit
- Optimize data movement (#20) · 897cb2ae
  Woosuk Kwon authored Apr 02, 2023
  
  897cb2ae
31 Mar, 2023 1 commit
- Add custom kernel for RMS normalization (#16) · 09e92454
  Woosuk Kwon authored Mar 31, 2023
  
  09e92454
30 Mar, 2023 1 commit
- Implement custom kernel for LLaMA rotary embedding (#14) · 88c0268a
  Woosuk Kwon authored Mar 30, 2023
  
  88c0268a