Commits · 84e4e37d1427bd254bea6ff366026773d36f3982 · kecinstone / 2024pra-vllm

02 Oct, 2023 2 commits
- [Minor] Fix type annotations (#1238) · 84e4e37d
  Woosuk Kwon authored Oct 02, 2023
  
  84e4e37d
- support sharding llama2-70b on more than 8 GPUs (#1209) · a60b3530
  Zhuohan Li authored Oct 02, 2023
```
Co-authored-by: JiCheng <247153481@qq.com>
```
  a60b3530
28 Sep, 2023 7 commits
- Bump up the version to v0.2.0 (#1212) · e2fb71ec
  Woosuk Kwon authored Sep 28, 2023
  
  e2fb71ec
- Provide default max model length (#1224) · f936657e
  Woosuk Kwon authored Sep 28, 2023
  
  f936657e
- [Fix] Remove false assertion (#1222) · 2e8e49fc
  Woosuk Kwon authored Sep 28, 2023
  
  2e8e49fc
- Fix Mistral model (#1220) · a8e98aee
  Woosuk Kwon authored Sep 28, 2023
  
  a8e98aee
- [Mistral] Mistral-7B-v0.1 support (#1196) · bb1ba58f
  Chris Bamford authored Sep 28, 2023
```
Co-authored-by: timlacroix <t@mistral.ai>
```
  bb1ba58f
- Add rope_scaling to Qwen (#1210) · 7bedab57
  Qing authored Sep 28, 2023
  
  7bedab57
- Add `skip_special_tokens` sampling params (#1186) · 20f7cc4c
  Dan Lord authored Sep 27, 2023
  
  20f7cc4c
27 Sep, 2023 5 commits
- Automatically configure `max_num_batched_tokens` (#1198) · a19bc5c6
  Woosuk Kwon authored Sep 27, 2023
  
  a19bc5c6
- fix qwen-14b model (#1173) · 28e616c4
  Qing authored Sep 28, 2023
  
  28e616c4
- fix typo (#1184) · 30e77528
  Wang Ran (汪然) authored Sep 28, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  30e77528
- Support Longchat and RoPE scaling (#555) · 21877b0d
  Lily Liu authored Sep 27, 2023
```
Co-authored-by: Wing Lian <wing.lian@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  21877b0d
- Allocate more shared memory to attention kernel (#1154) · cf5cb1e3
  Antoni Baum authored Sep 26, 2023
  
  cf5cb1e3
26 Sep, 2023 1 commit
- Add comments on RoPE initialization (#1176) · 03ffd0a0
  Woosuk Kwon authored Sep 26, 2023
  
  03ffd0a0
24 Sep, 2023 3 commits
- Align `max_tokens` behavior with openai (#852) · bbbf8656
  Wen Sun authored Sep 24, 2023
  
  bbbf8656
- Fix config for Falcon (#1164) · 9f6be869
  Woosuk Kwon authored Sep 23, 2023
  
  9f6be869
- [FIX] Simplify sampler logic (#1156) · f1878779
  Zhuohan Li authored Sep 23, 2023
  
  f1878779
23 Sep, 2023 1 commit
- [Sampler] Vectorized sampling (simplified) (#1048) · 947b7941
  Zhuohan Li authored Sep 22, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  947b7941
21 Sep, 2023 4 commits
- feat: support stop_token_ids parameter. (#1097) · f98b745a
  Ricardo Lu authored Sep 22, 2023
  
  f98b745a
- clean api code, remove redundant background task. (#1102) · 2d1e86f1
  Roy authored Sep 22, 2023
  
  2d1e86f1
- Add float16 and float32 (#1115) · 1ac4ccf7
  Woosuk Kwon authored Sep 21, 2023
  
  1ac4ccf7
- Replace DtypeTensor (#1123) · 2ac4d5e2
  Woosuk Kwon authored Sep 21, 2023
  
  2ac4d5e2
20 Sep, 2023 2 commits
- rope_theta and max_position_embeddings from config (#1096) · 3302f0ae
  Antoni Baum authored Sep 20, 2023
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: wnma3mz <wnma3mz@gmail.com>
```
  3302f0ae
- Add gpu_memory_utilization and swap_space to LLM (#1090) · bc064457
  Woosuk Kwon authored Sep 19, 2023
  
  bc064457
19 Sep, 2023 1 commit
- Add pyarrow to dependencies & Print warning on Ray import error (#1094) · 400b8289
  Woosuk Kwon authored Sep 18, 2023
  
  400b8289
18 Sep, 2023 6 commits
- Add minimum capability requirement for AWQ (#1064) · 2b1c116b
  Woosuk Kwon authored Sep 18, 2023
  
  2b1c116b
- Convert before transpose (#1073) · cc796b13
  Woosuk Kwon authored Sep 18, 2023
  
  cc796b13
- Fix get_max_num_running_seqs for waiting and swapped seq groups (#1068) · f029ef94
  Zhuohan Li authored Sep 18, 2023
  
  f029ef94
- align llm_engine and async_engine. (#1081) · 95592fa0
  Roy authored Sep 19, 2023
  
  95592fa0
- added support for quantize on LLM module (#1080) · fbe66e1d
  orellavie1212 authored Sep 18, 2023
  
  fbe66e1d
- [FIX] Don't initialize parameter by default (#1067) · 90979c38
  Zhuohan Li authored Sep 17, 2023
  
  90979c38
17 Sep, 2023 2 commits
- Fix hanging when prompt exceeds limit (#1029) · e21d7687
  陈序 authored Sep 17, 2023
  
  e21d7687
- Remove AsyncLLMEngine busy loop, shield background task (#1059) · ff36139f
  Antoni Baum authored Sep 17, 2023
  
  ff36139f
16 Sep, 2023 1 commit

Implement AWQ quantization support for LLaMA (#1032) · e3e79e9e

Woosuk Kwon authored Sep 16, 2023


Co-authored-by: Robert Irvine <robert@seamlessml.com>
Co-authored-by: root <rirv938@gmail.com>
Co-authored-by: Casper <casperbh.96@gmail.com>
Co-authored-by: julian-q <julianhquevedo@gmail.com>

e3e79e9e

15 Sep, 2023 3 commits
- Abort when coroutine is cancelled (#1020) · b9fe4616
  Jerry Yang authored Sep 15, 2023
  
  b9fe4616
- Fix warning message on LLaMA FastTokenizer (#1037) · 64ca424e
  Woosuk Kwon authored Sep 14, 2023
  
  64ca424e
- Only fail if logit_bias has actual values (#1045) · b5f93d06
  Lukas Kreussel authored Sep 15, 2023
  
  b5f93d06
14 Sep, 2023 1 commit
- Fix detokenization leaving special tokens (#1044) · dd54a4b0
  Antoni Baum authored Sep 14, 2023
```
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
```
  dd54a4b0
13 Sep, 2023 1 commit
- [FIX] Minor bug fixes (#1035) · f04908ca
  Zhuohan Li authored Sep 13, 2023
```
* [FIX] Minor bug fixes

* Address review comments
```
  f04908ca