Commits · acbed3ef40f015fcf64460e629813922fab90380 · kecinstone / 2024pra-vllm

03 Oct, 2023 2 commits
- Use monotonic time where appropriate (#1249) · acbed3ef
  Antoni Baum authored Oct 02, 2023
  
  acbed3ef
- add support for tokenizer revision (#1163) · 66d18a7f
  Federico Cassano authored Oct 02, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  66d18a7f
28 Sep, 2023 2 commits
- Provide default max model length (#1224) · f936657e
  Woosuk Kwon authored Sep 28, 2023
  
  f936657e
- Add `skip_special_tokens` sampling params (#1186) · 20f7cc4c
  Dan Lord authored Sep 27, 2023
  
  20f7cc4c
24 Sep, 2023 1 commit
- Align `max_tokens` behavior with openai (#852) · bbbf8656
  Wen Sun authored Sep 24, 2023
  
  bbbf8656
21 Sep, 2023 2 commits
- feat: support stop_token_ids parameter. (#1097) · f98b745a
  Ricardo Lu authored Sep 22, 2023
  
  f98b745a
- clean api code, remove redundant background task. (#1102) · 2d1e86f1
  Roy authored Sep 22, 2023
  
  2d1e86f1
20 Sep, 2023 1 commit
- Add gpu_memory_utilization and swap_space to LLM (#1090) · bc064457
  Woosuk Kwon authored Sep 19, 2023
  
  bc064457
18 Sep, 2023 1 commit
- added support for quantize on LLM module (#1080) · fbe66e1d
  orellavie1212 authored Sep 18, 2023
  
  fbe66e1d
15 Sep, 2023 1 commit
- Only fail if logit_bias has actual values (#1045) · b5f93d06
  Lukas Kreussel authored Sep 15, 2023
  
  b5f93d06
13 Sep, 2023 1 commit

Add Model Revision Support (#1014) · ab019eea

Jasmond L authored Sep 14, 2023


Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

ab019eea

08 Sep, 2023 1 commit
- Start background task in `AsyncLLMEngine.generate` (#988) · 08043847
  Antoni Baum authored Sep 08, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  08043847
07 Sep, 2023 1 commit

Make `AsyncLLMEngine` more robust & fix batched abort (#969) · c07ece5c

Antoni Baum authored Sep 07, 2023


Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Co-authored-by: Avnish Narayan <38871737+avnishn@users.noreply.github.com>

c07ece5c

05 Sep, 2023 1 commit
- Initialize AsyncLLMEngine bg loop correctly (#943) · 16967258
  Antoni Baum authored Sep 04, 2023
  
  16967258
30 Aug, 2023 1 commit
- Enable request body OpenAPI spec for OpenAI endpoints (#865) · becd7a56
  lplcor authored Aug 29, 2023
  
  becd7a56
11 Aug, 2023 1 commit
- Supports tokens and arrays of tokens as inputs to the OpenAI completion API (#715) · e06f504a
  WanMok authored Aug 11, 2023
  
  e06f504a
09 Aug, 2023 1 commit
- Check the max prompt length for the OpenAI completions API (#472) · 66c54aa9
  Nicolas Basile authored Aug 08, 2023
  
  66c54aa9
02 Aug, 2023 1 commit
- [BUG FIX] upgrade fschat version to 0.2.23 (#650) · e8ddc08e
  YHPeter authored Aug 02, 2023
```
Co-authored-by: hao.yu <hao.yu@cn-c017.server.mila.quebec>
```
  e8ddc08e
26 Jul, 2023 2 commits
- [Fix] Add model sequence length into model config (#575) · 58a072be
  Zhuohan Li authored Jul 25, 2023
  
  58a072be
- [Fix] Add chat completion Example and simplify dependencies (#576) · 82ad323d
  Zhuohan Li authored Jul 25, 2023
  
  82ad323d
20 Jul, 2023 1 commit
- fix: enable trust-remote-code in api server & benchmark. (#509) · 8c4b2592
  Ricardo Lu authored Jul 20, 2023
  
  8c4b2592
08 Jul, 2023 1 commit
- Sort the outputs before return (#402) · b6fbb9a5
  Woosuk Kwon authored Jul 08, 2023
  
  b6fbb9a5
07 Jul, 2023 3 commits
- Add trust-remote-code flag to handle remote tokenizers (#364) · a945fcc2
  codethazine authored Jul 07, 2023
  
  a945fcc2
- [Fix] Change /generate response-type to json for non-streaming (#374) · be54f8e5
  Nicolas Frenay authored Jul 06, 2023
  
  be54f8e5
- fix: only response [DONE] once when streaming response. (#378) · b396cb49
  Ricardo Lu authored Jul 07, 2023
  
  b396cb49
05 Jul, 2023 1 commit
- [Server] use fastchat.model.model_adapter.get_conversation_template method to... · 3d64cf01
  akxxsb authored Jul 05, 2023
```
[Server] use fastchat.model.model_adapter.get_conversation_template method to get model template (#357)
```
  3d64cf01
04 Jul, 2023 1 commit
- [Server] Add option to specify chat template for chat endpoint (#345) · 98fe8cb5
  Zhuohan Li authored Jul 03, 2023
  
  98fe8cb5
03 Jul, 2023 5 commits
- [Quality] Add CI for formatting (#343) · 42e0c1df
  Zhuohan Li authored Jul 03, 2023
  
  42e0c1df
- [Quality] Add code formatter and linter (#326) · d6fa1be3
  Zhuohan Li authored Jul 03, 2023
  
  d6fa1be3
- [Fix] Better error message for batched prompts (#342) · 0ffded81
  Zhuohan Li authored Jul 03, 2023
  
  0ffded81
- Allow send list of str for the Prompt on openai demo endpoint /v1/completions (#323) · 0bd2a573
  Michele Catalano authored Jul 03, 2023
```
* allow str or List[str] for prompt

* Update vllm/entrypoints/openai/api_server.py
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

---------
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  0bd2a573
- feat: add ChatCompletion endpoint in OpenAI demo server. (#330) · 49b26e2c
  Ricardo Lu authored Jul 03, 2023
  
  49b26e2c
28 Jun, 2023 3 commits
- [Tokenizer] Add tokenizer mode (#298) · 998d9d15
  Woosuk Kwon authored Jun 28, 2023
  
  998d9d15
- [Tokenizer] Add an option to specify tokenizer (#284) · 4338cc47
  Woosuk Kwon authored Jun 28, 2023
  
  4338cc47
- Add LLM.set_tokenizer (#283) · bdd6b4c8
  Jishnu Ray Chowdhury authored Jun 28, 2023
  
  bdd6b4c8
22 Jun, 2023 1 commit
- [Bugfix] Fix a bug in RequestOutput.finished (#202) · 14f0b39c
  Woosuk Kwon authored Jun 22, 2023
  
  14f0b39c
17 Jun, 2023 1 commit
- Change the name to vLLM (#150) · 0b98ba15
  Woosuk Kwon authored Jun 17, 2023
  
  0b98ba15