Commits · c1376e0f825e88e32b5aca85c676fe547bcb03c9 · kecinstone / 2024pra-vllm

17 Oct, 2023 1 commit
- Change scheduler & input tensor shape (#1381) · c1376e0f
  Woosuk Kwon authored Oct 16, 2023
  
  c1376e0f
16 Oct, 2023 1 commit
- Implement prompt logprobs & Batched topk for computing logprobs (#1328) · 9d9072a0
  Zhuohan Li authored Oct 16, 2023
```
Co-authored-by: Yunmo Chen <16273544+wanmok@users.noreply.github.com>
```
  9d9072a0
03 Oct, 2023 2 commits
- Use monotonic time where appropriate (#1249) · acbed3ef
  Antoni Baum authored Oct 02, 2023
  
  acbed3ef
- add support for tokenizer revision (#1163) · 66d18a7f
  Federico Cassano authored Oct 02, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  66d18a7f
28 Sep, 2023 3 commits
- Provide default max model length (#1224) · f936657e
  Woosuk Kwon authored Sep 28, 2023
  
  f936657e
- [Mistral] Mistral-7B-v0.1 support (#1196) · bb1ba58f
  Chris Bamford authored Sep 28, 2023
```
Co-authored-by: timlacroix <t@mistral.ai>
```
  bb1ba58f
- Add `skip_special_tokens` sampling params (#1186) · 20f7cc4c
  Dan Lord authored Sep 27, 2023
  
  20f7cc4c
27 Sep, 2023 2 commits
- Automatically configure `max_num_batched_tokens` (#1198) · a19bc5c6
  Woosuk Kwon authored Sep 27, 2023
  
  a19bc5c6
- fix typo (#1184) · 30e77528
  Wang Ran (汪然) authored Sep 28, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  30e77528
21 Sep, 2023 2 commits
- feat: support stop_token_ids parameter. (#1097) · f98b745a
  Ricardo Lu authored Sep 22, 2023
  
  f98b745a
- Add float16 and float32 (#1115) · 1ac4ccf7
  Woosuk Kwon authored Sep 21, 2023
  
  1ac4ccf7
19 Sep, 2023 1 commit
- Add pyarrow to dependencies & Print warning on Ray import error (#1094) · 400b8289
  Woosuk Kwon authored Sep 18, 2023
  
  400b8289
18 Sep, 2023 1 commit
- align llm_engine and async_engine. (#1081) · 95592fa0
  Roy authored Sep 19, 2023
  
  95592fa0
17 Sep, 2023 2 commits
- Fix hanging when prompt exceeds limit (#1029) · e21d7687
  陈序 authored Sep 17, 2023
  
  e21d7687
- Remove AsyncLLMEngine busy loop, shield background task (#1059) · ff36139f
  Antoni Baum authored Sep 17, 2023
  
  ff36139f
16 Sep, 2023 1 commit

Implement AWQ quantization support for LLaMA (#1032) · e3e79e9e

Woosuk Kwon authored Sep 16, 2023


Co-authored-by: Robert Irvine <robert@seamlessml.com>
Co-authored-by: root <rirv938@gmail.com>
Co-authored-by: Casper <casperbh.96@gmail.com>
Co-authored-by: julian-q <julianhquevedo@gmail.com>

e3e79e9e

15 Sep, 2023 1 commit
- Abort when coroutine is cancelled (#1020) · b9fe4616
  Jerry Yang authored Sep 15, 2023
  
  b9fe4616
13 Sep, 2023 2 commits
- Add Model Revision Support (#1014) · ab019eea
  Jasmond L authored Sep 14, 2023
```
Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  ab019eea
- Use TGI-like incremental detokenization (#984) · 9841d48a
  Antoni Baum authored Sep 13, 2023
  
  9841d48a
12 Sep, 2023 2 commits
- Make `max_model_len` configurable (#972) · 0bb1e885
  Antoni Baum authored Sep 12, 2023
  
  0bb1e885
- add option to shorten prompt print in log (#991) · d6545ad2
  leiwen83 authored Sep 13, 2023
```
Signed-off-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  d6545ad2
09 Sep, 2023 1 commit
- fix "tansformers_module" ModuleNotFoundError when load model with `trust_remote_code=True` (#871) · 4042d192
  Jingru authored Sep 09, 2023
  
  4042d192
08 Sep, 2023 1 commit
- Start background task in `AsyncLLMEngine.generate` (#988) · 08043847
  Antoni Baum authored Sep 08, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  08043847
07 Sep, 2023 2 commits
- Enable safetensors loading for all models (#974) · c957c741
  Zhuohan Li authored Sep 07, 2023
  
  c957c741
- Make `AsyncLLMEngine` more robust & fix batched abort (#969) · c07ece5c
  Antoni Baum authored Sep 07, 2023
```
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Co-authored-by: Avnish Narayan <38871737+avnishn@users.noreply.github.com>
```
  c07ece5c
06 Sep, 2023 1 commit
- Use queue for finished requests (#957) · c9927c1a
  Antoni Baum authored Sep 05, 2023
  
  c9927c1a
05 Sep, 2023 3 commits
- fix: typo (#948) · 22379d55
  Wen Sun authored Sep 05, 2023
  
  22379d55
- Initialize AsyncLLMEngine bg loop correctly (#943) · 16967258
  Antoni Baum authored Sep 04, 2023
  
  16967258
- Align vLLM's beam search implementation with HF generate (#857) · 002800f0
  Zhuohan Li authored Sep 04, 2023
  
  002800f0
04 Sep, 2023 1 commit
- Refactor AsyncLLMEngine (#880) · ce741ba3
  Antoni Baum authored Sep 03, 2023
  
  ce741ba3
02 Aug, 2023 1 commit
- Refactor scheduler (#658) · 55fe8a81
  Woosuk Kwon authored Aug 02, 2023
  
  55fe8a81
31 Jul, 2023 1 commit
- fix doc (#622) · aa39e42c
  Chaofan Lin authored Aug 01, 2023
  
  aa39e42c
30 Jul, 2023 1 commit
- fix ModuleNotFoundError (#599) · 953f28cf
  Fang li authored Jul 30, 2023
```
Co-authored-by: fangli <fangli@tencent.com>
```
  953f28cf
28 Jul, 2023 1 commit
- [Fix] fix import error of RayWorker (#604) (#605) · c0d00f5b
  Xudong Zhang authored Jul 28, 2023
  
  c0d00f5b
26 Jul, 2023 1 commit
- [Fix] Add model sequence length into model config (#575) · 58a072be
  Zhuohan Li authored Jul 25, 2023
  
  58a072be
20 Jul, 2023 3 commits
- Fix bad assert in initialize_cluster if PG already exists (#526) · c487a221
  Antoni Baum authored Jul 19, 2023
  
  c487a221
- Ray placement group support (#397) · 9925c179
  Antoni Baum authored Jul 19, 2023
  
  9925c179
- fix(ray_utils): ignore re-init error (#465) · 16c3e295
  Massimiliano Pronesti authored Jul 20, 2023
  
  16c3e295
18 Jul, 2023 1 commit
- fix max seq len (#489) · b4b195b3
  Lily Liu authored Jul 17, 2023
  
  b4b195b3
17 Jul, 2023 1 commit
- [Fix] Fix the condition of max_seq_len (#477) · 2bdea7ac
  Zhuohan Li authored Jul 17, 2023
  
  2bdea7ac