Commits · 7e1d5e5308fa3549dfed1821188d588260a03c8a · kecinstone / 2024pra-vllm

04 Feb, 2024 1 commit
- set&get llm internal tokenizer instead of the TokenizerGroup (#2741) · 51cd22ce
  dancingpipi authored Feb 05, 2024
```
Co-authored-by: shujunhua1 <shujunhua1@jd.com>
```
  51cd22ce
01 Feb, 2024 1 commit
- fix python 3.8 syntax (#2716) · b9e96b17
  Simon Mo authored Feb 01, 2024
  
  b9e96b17
27 Jan, 2024 1 commit
- Implement custom all reduce kernels (#2192) · 38017003
  Hanzhi Zhou authored Jan 28, 2024
  
  38017003
25 Jan, 2024 1 commit
- Support Batch Completion in Server (#2529) · 3a7dd7e3
  Simon Mo authored Jan 24, 2024
  
  3a7dd7e3
24 Jan, 2024 1 commit
- Added `include_stop_str_in_output` and `length_penalty` parameters to OpenAI API (#2562) · f1f6cc10
  Federico Galatolo authored Jan 24, 2024
  
  f1f6cc10
23 Jan, 2024 2 commits
- [Experimental] Add multi-LoRA support (#1804) · 9b945daa
  Antoni Baum authored Jan 24, 2024
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>
```
  9b945daa
- [Feature] Simple API token authentication and pluggable middlewares (#1106) · 9c1352eb
  Erfan Al-Hossami authored Jan 23, 2024
  
  9c1352eb
22 Jan, 2024 1 commit
- migrate pydantic from v1 to v2 (#2531) · 71d63ed7
  Jannis Schönleber authored Jan 22, 2024
  
  71d63ed7
19 Jan, 2024 1 commit
- refactor complemention api for readability (#2499) · dd7e8f5f
  Simon Mo authored Jan 18, 2024
  
  dd7e8f5f
18 Jan, 2024 1 commit

[Experimental] Prefix Caching Support (#1669) · d10f8e1d

shiyi.c_98 authored Jan 17, 2024


Co-authored-by: DouHappy <2278958187@qq.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

d10f8e1d

17 Jan, 2024 1 commit
- OpenAI Server refactoring (#2360) · 14cc317b
  FlorianJoncour authored Jan 17, 2024
  
  14cc317b
12 Jan, 2024 1 commit
- Allow setting fastapi root_path argument (#2341) · ce036244
  Chirag Jain authored Jan 13, 2024
  
  ce036244
05 Jan, 2024 1 commit
- Ensure metrics are logged regardless of requests (#2347) · d0215a58
  Iskren Ivov Chernev authored Jan 05, 2024
  
  d0215a58
03 Jan, 2024 1 commit
- Remove unused const TIMEOUT_TO_PREVENT_DEADLOCK (#2321) · 74d8d776
  Ronen Schaffer authored Jan 04, 2024
  
  74d8d776
18 Dec, 2023 1 commit
- Add SSL arguments to API servers (#2109) · 08133c4d
  Harry Mellor authored Dec 18, 2023
  
  08133c4d
17 Dec, 2023 2 commits
- [Minor] Add more detailed explanation on `quantization` argument (#2145) · 30fb0956
  Woosuk Kwon authored Dec 17, 2023
  
  30fb0956
- Optimize model execution with CUDA graph (#1926) · 37ca5581
  Woosuk Kwon authored Dec 16, 2023
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  37ca5581
15 Dec, 2023 1 commit
- Add GPTQ support (#916) · 0fbfc4b8
  CHU Tianxiang authored Dec 15, 2023
  
  0fbfc4b8
10 Dec, 2023 2 commits
- Fix completion API echo and logprob combo (#1992) · 2e8fc0d4
  Simon Mo authored Dec 10, 2023
  
  2e8fc0d4
- Fix OpenAI server completion_tokens referenced before assignment (#1996) · 1aa13615
  Jin Shang authored Dec 10, 2023
  
  1aa13615
03 Dec, 2023 2 commits
- add custom server params (#1868) · 60dc62dc
  Roy authored Dec 04, 2023
  
  60dc62dc
- Add Production Metrics in Prometheus format (#1890) · 5313c2cb
  Simon Mo authored Dec 02, 2023
  
  5313c2cb
01 Dec, 2023 1 commit
- Support chat template and `echo` for chat API (#1756) · 66785cc0
  Adam Brusselback authored Nov 30, 2023
  
  66785cc0
30 Nov, 2023 1 commit
- Disable Logs Requests should Disable Logging of requests. (#1779) · c7821956
  Michael McCulloch authored Nov 29, 2023
```
Co-authored-by: Michael McCulloch <mjm.gitlab@fastmail.com>
```
  c7821956
27 Nov, 2023 1 commit
- Added echo function to OpenAI API server. (#1504) · 665cbcec
  Yunmo Chen authored Nov 27, 2023
  
  665cbcec
20 Nov, 2023 1 commit
- Migrate linter from `pylint` to `ruff` (#1665) · 5ffc0d13
  Simon Mo authored Nov 20, 2023
  
  5ffc0d13
18 Nov, 2023 1 commit
- Support download models from www.modelscope.cn (#1588) · edb30558
  liuyhwangyh authored Nov 18, 2023
  
  edb30558
16 Nov, 2023 1 commit
- Return usage for openai streaming requests (#1663) · 686f5e32
  Iskren Ivov Chernev authored Nov 17, 2023
  
  686f5e32
01 Nov, 2023 1 commit
- Add `/health` Endpoint for both Servers (#1540) · 7e90a2d1
  Fluder-Paradyne authored Nov 01, 2023
  
  7e90a2d1
30 Oct, 2023 1 commit
- Add support for `spaces_between_special_tokens` · 7013a801
  Dan Lord authored Oct 30, 2023
  
  7013a801
07 Oct, 2023 1 commit
- API server support ipv4 / ipv6 dualstack (#1288) · 09ff7f10
  Yunfeng Bai authored Oct 07, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  09ff7f10
03 Oct, 2023 2 commits
- Use monotonic time where appropriate (#1249) · acbed3ef
  Antoni Baum authored Oct 02, 2023
  
  acbed3ef
- add support for tokenizer revision (#1163) · 66d18a7f
  Federico Cassano authored Oct 02, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  66d18a7f
28 Sep, 2023 2 commits
- Provide default max model length (#1224) · f936657e
  Woosuk Kwon authored Sep 28, 2023
  
  f936657e
- Add `skip_special_tokens` sampling params (#1186) · 20f7cc4c
  Dan Lord authored Sep 27, 2023
  
  20f7cc4c
24 Sep, 2023 1 commit
- Align `max_tokens` behavior with openai (#852) · bbbf8656
  Wen Sun authored Sep 24, 2023
  
  bbbf8656
21 Sep, 2023 2 commits
- feat: support stop_token_ids parameter. (#1097) · f98b745a
  Ricardo Lu authored Sep 22, 2023
  
  f98b745a
- clean api code, remove redundant background task. (#1102) · 2d1e86f1
  Roy authored Sep 22, 2023
  
  2d1e86f1
20 Sep, 2023 1 commit
- Add gpu_memory_utilization and swap_space to LLM (#1090) · bc064457
  Woosuk Kwon authored Sep 19, 2023
  
  bc064457
18 Sep, 2023 1 commit
- added support for quantize on LLM module (#1080) · fbe66e1d
  orellavie1212 authored Sep 18, 2023
  
  fbe66e1d