Commits · d6545ad22ee89c8b6e3eb6cfcf8ff914a06ccee1 · kecinstone / 2024pra-vllm

12 Sep, 2023 1 commit

add option to shorten prompt print in log (#991) · d6545ad2

leiwen83 authored Sep 13, 2023


Signed-off-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

d6545ad2

11 Sep, 2023 2 commits
- Bump up the version to v0.1.7 (#1013) · 90eb3f43
  Woosuk Kwon authored Sep 11, 2023
  
  90eb3f43
- Use FP32 in RoPE initialization (#1004) · e67b4f2c
  Woosuk Kwon authored Sep 11, 2023
```
Co-authored-by: One <imone@tuta.io>
```
  e67b4f2c
10 Sep, 2023 1 commit
- fix: CUDA error when inferencing with Falcon-40B base model (#992) · 898285c9
  Kyujin Cho authored Sep 10, 2023
  
  898285c9
09 Sep, 2023 2 commits
- Fix wrong dtype in PagedAttentionWithALiBi bias (#996) · a62de9ec
  Antoni Baum authored Sep 09, 2023
```
---------
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
```
  a62de9ec
- fix "tansformers_module" ModuleNotFoundError when load model with `trust_remote_code=True` (#871) · 4042d192
  Jingru authored Sep 09, 2023
  
  4042d192
08 Sep, 2023 3 commits
- Bump up the version to v0.1.6 (#989) · 1117aa14
  Zhuohan Li authored Sep 08, 2023
  
  1117aa14
- Start background task in `AsyncLLMEngine.generate` (#988) · 08043847
  Antoni Baum authored Sep 08, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  08043847
- faster startup of vLLM (#982) · 4b5bcf89
  Robert Irvine authored Sep 08, 2023
```
* update

---------
Co-authored-by: Robert Irvine <robert@seamlessml.com>
```
  4b5bcf89
07 Sep, 2023 4 commits
- Bump up the version to v0.1.5 (#944) · 852ef5b4
  Woosuk Kwon authored Sep 08, 2023
  
  852ef5b4
- Enable safetensors loading for all models (#974) · c957c741
  Zhuohan Li authored Sep 07, 2023
  
  c957c741
- Make `AsyncLLMEngine` more robust & fix batched abort (#969) · c07ece5c
  Antoni Baum authored Sep 07, 2023
```
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Co-authored-by: Avnish Narayan <38871737+avnishn@users.noreply.github.com>
```
  c07ece5c
- Set torch default dtype in a context manager (#971) · 005ba458
  Antoni Baum authored Sep 06, 2023
```
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
```
  005ba458
06 Sep, 2023 2 commits
- [BugFix] Implement RoPE for GPT-J (#941) · 320a622e
  Woosuk Kwon authored Sep 06, 2023
  
  320a622e
- Use queue for finished requests (#957) · c9927c1a
  Antoni Baum authored Sep 05, 2023
  
  c9927c1a
05 Sep, 2023 3 commits
- fix: typo (#948) · 22379d55
  Wen Sun authored Sep 05, 2023
  
  22379d55
- Initialize AsyncLLMEngine bg loop correctly (#943) · 16967258
  Antoni Baum authored Sep 04, 2023
  
  16967258
- Align vLLM's beam search implementation with HF generate (#857) · 002800f0
  Zhuohan Li authored Sep 04, 2023
  
  002800f0
04 Sep, 2023 2 commits
- Only emit warning about internal tokenizer if it isn't being used (#939) · e15932bb
  Nelson Liu authored Sep 04, 2023
  
  e15932bb
- Refactor AsyncLLMEngine (#880) · ce741ba3
  Antoni Baum authored Sep 03, 2023
  
  ce741ba3
31 Aug, 2023 2 commits
- fix: bug fix when penalties are negative (#913) · e1122233
  Dong-Yong Lee authored Sep 01, 2023
```
Co-authored-by: dongyong-lee <dongyong.lee@navercorp.com>
```
  e1122233
- Improve _prune_hidden_states micro-benchmark (#707) · 28873a27
  Aman Gupta Karmani authored Aug 31, 2023
  
  28873a27
30 Aug, 2023 3 commits
- Accelerate LLaMA model loading (#234) · 0d93f156
  JFDuan authored Aug 30, 2023
  
  0d93f156
- Enable request body OpenAPI spec for OpenAI endpoints (#865) · becd7a56
  lplcor authored Aug 29, 2023
  
  becd7a56
- use flash-attn via xformers (#877) · 75471386
  Aman Gupta Karmani authored Aug 30, 2023
  
  75471386
28 Aug, 2023 1 commit
- [Fix] Fix a condition for ignored sequences (#867) · d2b2eed6
  Zhuohan Li authored Aug 27, 2023
  
  d2b2eed6
25 Aug, 2023 3 commits
- Add support for CodeLlama (#854) · 4b6f069b
  Antoni Baum authored Aug 25, 2023
  
  4b6f069b
- Bump up the version to v0.1.4 (#846) · 791d79de
  Woosuk Kwon authored Aug 25, 2023
  
  791d79de
- Set replacement=True in torch.multinomial (#858) · 94d2f598
  Woosuk Kwon authored Aug 25, 2023
  
  94d2f598
23 Aug, 2023 2 commits
- Clean up code (#844) · 75c0ca9d
  wenjun93 authored Aug 24, 2023
  
  75c0ca9d
- Fix for breaking changes in xformers 0.0.21 (#834) · 2a4ec908
  Woosuk Kwon authored Aug 23, 2023
  
  2a4ec908
22 Aug, 2023 4 commits

Implement approximate GELU kernels (#828) · d64bf164
Woosuk Kwon authored Aug 23, 2023

d64bf164
fix: revert code to avoid no attribute problem (#827) · eedac9db
Wen Sun authored Aug 23, 2023

eedac9db

Add support for aquila (#663) · ad5f2fe3

shunxing1234 authored Aug 22, 2023



* add aquila
Signed-off-by: ftgreat <ftgreat@163.com>

* fix some bug
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete pdb
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete whitespace
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* format

* fix order

---------
Signed-off-by: ftgreat <ftgreat@163.com>
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
Co-authored-by: ftgreat <ftgreat@163.com>

ad5f2fe3

Fix mqa is false case in gpt_bigcode (#806) · 4f858475
zhaoyang-star authored Aug 22, 2023

4f858475

18 Aug, 2023 1 commit
- Fix typo in sampling_params.py (#788) · 0c04ce32
  wangcx18 authored Aug 18, 2023
  
  0c04ce32
17 Aug, 2023 1 commit
- explicitly del state (#784) · 73b3de79
  Xinyu Yang authored Aug 18, 2023
  
  73b3de79
15 Aug, 2023 2 commits
- Align with huggingface Top K sampling (#753) · d1744376
  Abraham-Xu authored Aug 16, 2023
  
  d1744376
- Fix typo in tokenizer.py (#750) · 805de738
  Ikko Eltociear Ashimine authored Aug 15, 2023
```
conjuction -> conjunction
```
  805de738
11 Aug, 2023 1 commit
- Supports tokens and arrays of tokens as inputs to the OpenAI completion API (#715) · e06f504a
  WanMok authored Aug 11, 2023
  
  e06f504a