Commits · 0f90effc660317070b88b860eda76bfb3b596701 · kecinstone / 2024pra-vllm

03 Dec, 2023 1 commit
- Add Production Metrics in Prometheus format (#1890) · 5313c2cb
  Simon Mo authored Dec 02, 2023
  
  5313c2cb
01 Dec, 2023 1 commit
- Support chat template and `echo` for chat API (#1756) · 66785cc0
  Adam Brusselback authored Nov 30, 2023
  
  66785cc0
30 Nov, 2023 1 commit
- Disable Logs Requests should Disable Logging of requests. (#1779) · c7821956
  Michael McCulloch authored Nov 29, 2023
```
Co-authored-by: Michael McCulloch <mjm.gitlab@fastmail.com>
```
  c7821956
27 Nov, 2023 1 commit
- Added echo function to OpenAI API server. (#1504) · 665cbcec
  Yunmo Chen authored Nov 27, 2023
  
  665cbcec
20 Nov, 2023 1 commit
- Migrate linter from `pylint` to `ruff` (#1665) · 5ffc0d13
  Simon Mo authored Nov 20, 2023
  
  5ffc0d13
18 Nov, 2023 1 commit
- Support download models from www.modelscope.cn (#1588) · edb30558
  liuyhwangyh authored Nov 18, 2023
  
  edb30558
16 Nov, 2023 1 commit
- Return usage for openai streaming requests (#1663) · 686f5e32
  Iskren Ivov Chernev authored Nov 17, 2023
  
  686f5e32
01 Nov, 2023 1 commit
- Add `/health` Endpoint for both Servers (#1540) · 7e90a2d1
  Fluder-Paradyne authored Nov 01, 2023
  
  7e90a2d1
30 Oct, 2023 1 commit
- Add support for `spaces_between_special_tokens` · 7013a801
  Dan Lord authored Oct 30, 2023
  
  7013a801
07 Oct, 2023 1 commit
- API server support ipv4 / ipv6 dualstack (#1288) · 09ff7f10
  Yunfeng Bai authored Oct 07, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  09ff7f10
03 Oct, 2023 2 commits
- Use monotonic time where appropriate (#1249) · acbed3ef
  Antoni Baum authored Oct 02, 2023
  
  acbed3ef
- add support for tokenizer revision (#1163) · 66d18a7f
  Federico Cassano authored Oct 02, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  66d18a7f
28 Sep, 2023 2 commits
- Provide default max model length (#1224) · f936657e
  Woosuk Kwon authored Sep 28, 2023
  
  f936657e
- Add `skip_special_tokens` sampling params (#1186) · 20f7cc4c
  Dan Lord authored Sep 27, 2023
  
  20f7cc4c
24 Sep, 2023 1 commit
- Align `max_tokens` behavior with openai (#852) · bbbf8656
  Wen Sun authored Sep 24, 2023
  
  bbbf8656
21 Sep, 2023 2 commits
- feat: support stop_token_ids parameter. (#1097) · f98b745a
  Ricardo Lu authored Sep 22, 2023
  
  f98b745a
- clean api code, remove redundant background task. (#1102) · 2d1e86f1
  Roy authored Sep 22, 2023
  
  2d1e86f1
20 Sep, 2023 1 commit
- Add gpu_memory_utilization and swap_space to LLM (#1090) · bc064457
  Woosuk Kwon authored Sep 19, 2023
  
  bc064457
18 Sep, 2023 1 commit
- added support for quantize on LLM module (#1080) · fbe66e1d
  orellavie1212 authored Sep 18, 2023
  
  fbe66e1d
15 Sep, 2023 1 commit
- Only fail if logit_bias has actual values (#1045) · b5f93d06
  Lukas Kreussel authored Sep 15, 2023
  
  b5f93d06
13 Sep, 2023 1 commit

Add Model Revision Support (#1014) · ab019eea

Jasmond L authored Sep 14, 2023


Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

ab019eea

08 Sep, 2023 1 commit
- Start background task in `AsyncLLMEngine.generate` (#988) · 08043847
  Antoni Baum authored Sep 08, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  08043847
07 Sep, 2023 1 commit

Make `AsyncLLMEngine` more robust & fix batched abort (#969) · c07ece5c

Antoni Baum authored Sep 07, 2023


Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
Co-authored-by: Avnish Narayan <38871737+avnishn@users.noreply.github.com>

c07ece5c

05 Sep, 2023 1 commit
- Initialize AsyncLLMEngine bg loop correctly (#943) · 16967258
  Antoni Baum authored Sep 04, 2023
  
  16967258
30 Aug, 2023 1 commit
- Enable request body OpenAPI spec for OpenAI endpoints (#865) · becd7a56
  lplcor authored Aug 29, 2023
  
  becd7a56
11 Aug, 2023 1 commit
- Supports tokens and arrays of tokens as inputs to the OpenAI completion API (#715) · e06f504a
  WanMok authored Aug 11, 2023
  
  e06f504a
09 Aug, 2023 1 commit
- Check the max prompt length for the OpenAI completions API (#472) · 66c54aa9
  Nicolas Basile authored Aug 08, 2023
  
  66c54aa9
02 Aug, 2023 1 commit
- [BUG FIX] upgrade fschat version to 0.2.23 (#650) · e8ddc08e
  YHPeter authored Aug 02, 2023
```
Co-authored-by: hao.yu <hao.yu@cn-c017.server.mila.quebec>
```
  e8ddc08e
26 Jul, 2023 2 commits
- [Fix] Add model sequence length into model config (#575) · 58a072be
  Zhuohan Li authored Jul 25, 2023
  
  58a072be
- [Fix] Add chat completion Example and simplify dependencies (#576) · 82ad323d
  Zhuohan Li authored Jul 25, 2023
  
  82ad323d
20 Jul, 2023 1 commit
- fix: enable trust-remote-code in api server & benchmark. (#509) · 8c4b2592
  Ricardo Lu authored Jul 20, 2023
  
  8c4b2592
08 Jul, 2023 1 commit
- Sort the outputs before return (#402) · b6fbb9a5
  Woosuk Kwon authored Jul 08, 2023
  
  b6fbb9a5
07 Jul, 2023 3 commits
- Add trust-remote-code flag to handle remote tokenizers (#364) · a945fcc2
  codethazine authored Jul 07, 2023
  
  a945fcc2
- [Fix] Change /generate response-type to json for non-streaming (#374) · be54f8e5
  Nicolas Frenay authored Jul 06, 2023
  
  be54f8e5
- fix: only response [DONE] once when streaming response. (#378) · b396cb49
  Ricardo Lu authored Jul 07, 2023
  
  b396cb49
05 Jul, 2023 1 commit
- [Server] use fastchat.model.model_adapter.get_conversation_template method to... · 3d64cf01
  akxxsb authored Jul 05, 2023
```
[Server] use fastchat.model.model_adapter.get_conversation_template method to get model template (#357)
```
  3d64cf01
04 Jul, 2023 1 commit
- [Server] Add option to specify chat template for chat endpoint (#345) · 98fe8cb5
  Zhuohan Li authored Jul 03, 2023
  
  98fe8cb5
03 Jul, 2023 3 commits
- [Quality] Add CI for formatting (#343) · 42e0c1df
  Zhuohan Li authored Jul 03, 2023
  
  42e0c1df
- [Quality] Add code formatter and linter (#326) · d6fa1be3
  Zhuohan Li authored Jul 03, 2023
  
  d6fa1be3
- [Fix] Better error message for batched prompts (#342) · 0ffded81
  Zhuohan Li authored Jul 03, 2023
  
  0ffded81