Commits · 79d64c495463665cf80937a05226806785cfa583 · norm / vllm

09 Jan, 2024 1 commit
- [Speculative decoding 1/9] Optimized rejection sampler (#2336) · 79d64c49
  Cade Daniel authored Jan 09, 2024
  
  79d64c49
04 Jan, 2024 1 commit
- Revert the changes in test_cache (#2335) · 94176712
  Woosuk Kwon authored Jan 03, 2024
  
  94176712
03 Jan, 2024 2 commits
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
- [FIX] Support non-zero CUDA devices in custom kernels (#1959) · 77af974b
  Jee Li authored Jan 03, 2024
  
  77af974b
27 Dec, 2023 1 commit
- [BUGFIX] Fix communication test (#2285) · 358c328d
  Zhuohan Li authored Dec 28, 2023
  
  358c328d
26 Dec, 2023 2 commits
- [BUGFIX] Fix the path of test prompts (#2273) · 4aaafdd2
  Zhuohan Li authored Dec 27, 2023
  
  4aaafdd2
- [BUGFIX] Fix API server test (#2270) · 66b108d1
  Zhuohan Li authored Dec 27, 2023
  
  66b108d1
19 Dec, 2023 1 commit
- Added DeciLM-7b and DeciLM-7b-instruct (#2062) · de60a3fb
  avideci authored Dec 19, 2023
  
  de60a3fb
17 Dec, 2023 1 commit
- [Minor] Add Phi 2 to supported models (#2159) · f8c688d7
  Woosuk Kwon authored Dec 17, 2023
  
  f8c688d7
13 Dec, 2023 1 commit
- [BugFix] Fix input positions for long context with sliding window (#2088) · f1c85201
  Woosuk Kwon authored Dec 13, 2023
  
  f1c85201
10 Dec, 2023 1 commit
- Replace head_mapping params with num_kv_heads to attention kernel. (#1997) · dacaf5a4
  wbn authored Dec 11, 2023
```
Co-authored-by: wangguoya <wangguoya@baidu.com>
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com>
```
  dacaf5a4
03 Dec, 2023 3 commits
- Fix broken worker test (#1900) · cd3aa153
  Woosuk Kwon authored Dec 02, 2023
  
  cd3aa153
- Add PyTorch-native implementation of custom layers (#1898) · 9b294976
  Woosuk Kwon authored Dec 02, 2023
  
  9b294976
- Fix broken sampler tests (#1896) · 5f09cbdb
  Woosuk Kwon authored Dec 02, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  5f09cbdb
01 Dec, 2023 1 commit
- Support chat template and `echo` for chat API (#1756) · 66785cc0
  Adam Brusselback authored Nov 30, 2023
  
  66785cc0
24 Nov, 2023 1 commit
- [Build] Avoid building too many extensions (#1624) · e0c6f556
  Yanming W authored Nov 23, 2023
  
  e0c6f556
20 Nov, 2023 1 commit
- Migrate linter from `pylint` to `ruff` (#1665) · 5ffc0d13
  Simon Mo authored Nov 20, 2023
  
  5ffc0d13
17 Nov, 2023 1 commit
- [Fix] Fix comm test (#1691) · 20d0699d
  Zhuohan Li authored Nov 16, 2023
  
  20d0699d
16 Nov, 2023 2 commits
- Support Microsoft Phi 1.5 (#1664) · 521b35f7
  maximzubkov authored Nov 16, 2023
  
  521b35f7
- [Minor] Fix duplication of ignored seq group in engine step (#1666) · cb08cd0d
  Simon Mo authored Nov 16, 2023
  
  cb08cd0d
08 Nov, 2023 1 commit
- Fix input_metadata.selected_token_indices in worker prepare_inputs (#1546) · 8efe23f1
  Yanming W authored Nov 09, 2023
  
  8efe23f1
03 Nov, 2023 1 commit
- Added logits processor API to sampling params (#1469) · 555bdcc5
  Noam Gat authored Nov 03, 2023
  
  555bdcc5
31 Oct, 2023 2 commits
- [Small] Formatter only checks lints in changed files (#1528) · e575df33
  Cade Daniel authored Oct 31, 2023
  
  e575df33
- Fix integer overflows in attention & cache ops (#1514) · 0ce8647d
  Woosuk Kwon authored Oct 31, 2023
  
  0ce8647d
17 Oct, 2023 1 commit
- Add Mistral 7B to `test_models` (#1366) · 95248677
  Woosuk Kwon authored Oct 16, 2023
  
  95248677
16 Oct, 2023 3 commits
- Fix sampler test (#1379) · d3a5bd9f
  Woosuk Kwon authored Oct 16, 2023
  
  d3a5bd9f
- Implement prompt logprobs & Batched topk for computing logprobs (#1328) · 9d9072a0
  Zhuohan Li authored Oct 16, 2023
```
Co-authored-by: Yunmo Chen <16273544+wanmok@users.noreply.github.com>
```
  9d9072a0
- Implement PagedAttention V2 (#1348) · 928de468
  Woosuk Kwon authored Oct 16, 2023
  
  928de468
02 Oct, 2023 1 commit
- TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic (#1181) · ba0bfd40
  Zhuohan Li authored Oct 02, 2023
  
  ba0bfd40
28 Sep, 2023 1 commit
- Fix OOM in attention kernel test (#1223) · 6f88f762
  Woosuk Kwon authored Sep 28, 2023
  
  6f88f762
27 Sep, 2023 1 commit
- Allocate more shared memory to attention kernel (#1154) · cf5cb1e3
  Antoni Baum authored Sep 26, 2023
  
  cf5cb1e3
23 Sep, 2023 1 commit
- [Sampler] Vectorized sampling (simplified) (#1048) · 947b7941
  Zhuohan Li authored Sep 22, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  947b7941
17 Sep, 2023 1 commit
- Remove AsyncLLMEngine busy loop, shield background task (#1059) · ff36139f
  Antoni Baum authored Sep 17, 2023
  
  ff36139f
14 Sep, 2023 1 commit
- Fix detokenization leaving special tokens (#1044) · dd54a4b0
  Antoni Baum authored Sep 14, 2023
```
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
```
  dd54a4b0
13 Sep, 2023 1 commit
- Use TGI-like incremental detokenization (#984) · 9841d48a
  Antoni Baum authored Sep 13, 2023
  
  9841d48a
11 Sep, 2023 1 commit
- Use FP32 in RoPE initialization (#1004) · e67b4f2c
  Woosuk Kwon authored Sep 11, 2023
```
Co-authored-by: One <imone@tuta.io>
```
  e67b4f2c
08 Sep, 2023 1 commit
- Start background task in `AsyncLLMEngine.generate` (#988) · 08043847
  Antoni Baum authored Sep 08, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  08043847
07 Sep, 2023 2 commits

[FIX] Fix Alibi implementation in PagedAttention kernel (#945) · db09d4ad

Zhuohan Li authored Sep 07, 2023



* [FIX] Fix Alibi implementation in PagedAttention kernel

* Fix test_attention

* Fix

---------
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Oliver-ss <yuansongwx@outlook.com>

db09d4ad

Make `AsyncLLMEngine` more robust & fix batched abort (#969) · c07ece5c

Antoni Baum authored Sep 07, 2023


Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Co-authored-by: Avnish Narayan <38871737+avnishn@users.noreply.github.com>

c07ece5c

06 Sep, 2023 1 commit
- [BugFix] Implement RoPE for GPT-J (#941) · 320a622e
  Woosuk Kwon authored Sep 06, 2023
  
  320a622e