Commits · 1ac4ccf73c91370f1fcb4f60c1117646cd7a7502 · norm / vllm

21 Sep, 2023 2 commits
- Add float16 and float32 (#1115) · 1ac4ccf7
  Woosuk Kwon authored Sep 21, 2023
  
  1ac4ccf7
- Replace DtypeTensor (#1123) · 2ac4d5e2
  Woosuk Kwon authored Sep 21, 2023
  
  2ac4d5e2
20 Sep, 2023 3 commits
- rope_theta and max_position_embeddings from config (#1096) · 3302f0ae
  Antoni Baum authored Sep 20, 2023
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: wnma3mz <wnma3mz@gmail.com>
```
  3302f0ae
- Add documentation to Triton server tutorial (#983) · 6f2dd6c3
  Tanmay Verma authored Sep 20, 2023
  
  6f2dd6c3
- Add gpu_memory_utilization and swap_space to LLM (#1090) · bc064457
  Woosuk Kwon authored Sep 19, 2023
  
  bc064457
19 Sep, 2023 1 commit
- Add pyarrow to dependencies & Print warning on Ray import error (#1094) · 400b8289
  Woosuk Kwon authored Sep 18, 2023
  
  400b8289
18 Sep, 2023 7 commits
- [Community] Add vLLM Discord server (#1086) · c1026311
  Zhuohan Li authored Sep 18, 2023
  
  c1026311
- Add minimum capability requirement for AWQ (#1064) · 2b1c116b
  Woosuk Kwon authored Sep 18, 2023
  
  2b1c116b
- Convert before transpose (#1073) · cc796b13
  Woosuk Kwon authored Sep 18, 2023
  
  cc796b13
- Fix get_max_num_running_seqs for waiting and swapped seq groups (#1068) · f029ef94
  Zhuohan Li authored Sep 18, 2023
  
  f029ef94
- align llm_engine and async_engine. (#1081) · 95592fa0
  Roy authored Sep 19, 2023
  
  95592fa0
- added support for quantize on LLM module (#1080) · fbe66e1d
  orellavie1212 authored Sep 18, 2023
  
  fbe66e1d
- [FIX] Don't initialize parameter by default (#1067) · 90979c38
  Zhuohan Li authored Sep 17, 2023
  
  90979c38
17 Sep, 2023 2 commits
- Fix hanging when prompt exceeds limit (#1029) · e21d7687
  陈序 authored Sep 17, 2023
  
  e21d7687
- Remove AsyncLLMEngine busy loop, shield background task (#1059) · ff36139f
  Antoni Baum authored Sep 17, 2023
  
  ff36139f
16 Sep, 2023 1 commit

Implement AWQ quantization support for LLaMA (#1032) · e3e79e9e

Woosuk Kwon authored Sep 16, 2023


Co-authored-by: Robert Irvine <robert@seamlessml.com>
Co-authored-by: root <rirv938@gmail.com>
Co-authored-by: Casper <casperbh.96@gmail.com>
Co-authored-by: julian-q <julianhquevedo@gmail.com>

e3e79e9e

15 Sep, 2023 4 commits
- Abort when coroutine is cancelled (#1020) · b9fe4616
  Jerry Yang authored Sep 15, 2023
  
  b9fe4616
- Fix warning message on LLaMA FastTokenizer (#1037) · 64ca424e
  Woosuk Kwon authored Sep 14, 2023
  
  64ca424e
- Only fail if logit_bias has actual values (#1045) · b5f93d06
  Lukas Kreussel authored Sep 15, 2023
  
  b5f93d06
- Add pandas to requirements.txt (#1047) · a5893696
  Woosuk Kwon authored Sep 14, 2023
```
* Add pandas to requirements.txt

* Minor
```
  a5893696
14 Sep, 2023 2 commits
- Fix detokenization leaving special tokens (#1044) · dd54a4b0
  Antoni Baum authored Sep 14, 2023
```
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
```
  dd54a4b0
- Announce paper release (#1036) · eda1a7ca
  Woosuk Kwon authored Sep 13, 2023
  
  eda1a7ca
13 Sep, 2023 4 commits
- [FIX] Minor bug fixes (#1035) · f04908ca
  Zhuohan Li authored Sep 13, 2023
```
* [FIX] Minor bug fixes

* Address review comments
```
  f04908ca
- Add Model Revision Support (#1014) · ab019eea
  Jasmond L authored Sep 14, 2023
```
Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  ab019eea
- Use TGI-like incremental detokenization (#984) · 9841d48a
  Antoni Baum authored Sep 13, 2023
  
  9841d48a
- Fix typo in README.md (#1033) · 3272d7a0
  Ikko Eltociear Ashimine authored Sep 14, 2023
  
  3272d7a0
12 Sep, 2023 2 commits
- Make `max_model_len` configurable (#972) · 0bb1e885
  Antoni Baum authored Sep 12, 2023
  
  0bb1e885
- add option to shorten prompt print in log (#991) · d6545ad2
  leiwen83 authored Sep 13, 2023
```
Signed-off-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  d6545ad2
11 Sep, 2023 3 commits
- Bump up the version to v0.1.7 (#1013) · 90eb3f43
  Woosuk Kwon authored Sep 11, 2023
  
  90eb3f43
- Use FP32 in RoPE initialization (#1004) · e67b4f2c
  Woosuk Kwon authored Sep 11, 2023
```
Co-authored-by: One <imone@tuta.io>
```
  e67b4f2c
- Update setup.py (#1006) · d6770d1f
  Woosuk Kwon authored Sep 10, 2023
  
  d6770d1f
10 Sep, 2023 2 commits
- [Docs] Update installation page (#1005) · b9cecc26
  Woosuk Kwon authored Sep 10, 2023
  
  b9cecc26
- fix: CUDA error when inferencing with Falcon-40B base model (#992) · 898285c9
  Kyujin Cho authored Sep 10, 2023
  
  898285c9
09 Sep, 2023 2 commits
- Fix wrong dtype in PagedAttentionWithALiBi bias (#996) · a62de9ec
  Antoni Baum authored Sep 09, 2023
```
---------
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
```
  a62de9ec
- fix "tansformers_module" ModuleNotFoundError when load model with `trust_remote_code=True` (#871) · 4042d192
  Jingru authored Sep 09, 2023
  
  4042d192
08 Sep, 2023 3 commits
- Bump up the version to v0.1.6 (#989) · 1117aa14
  Zhuohan Li authored Sep 08, 2023
  
  1117aa14
- Start background task in `AsyncLLMEngine.generate` (#988) · 08043847
  Antoni Baum authored Sep 08, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  08043847
- faster startup of vLLM (#982) · 4b5bcf89
  Robert Irvine authored Sep 08, 2023
```
* update

---------
Co-authored-by: Robert Irvine <robert@seamlessml.com>
```
  4b5bcf89
07 Sep, 2023 2 commits

Bump up the version to v0.1.5 (#944) · 852ef5b4
Woosuk Kwon authored Sep 08, 2023

852ef5b4

[FIX] Fix Alibi implementation in PagedAttention kernel (#945) · db09d4ad

Zhuohan Li authored Sep 07, 2023



* [FIX] Fix Alibi implementation in PagedAttention kernel

* Fix test_attention

* Fix

---------
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Oliver-ss <yuansongwx@outlook.com>

db09d4ad