Commits · bbec01c9aa219701320395133c8db848e635cde9 · change / sglang

15 Jun, 2024 1 commit
- Fix tp worker only checking req[0] for stream (#546) · bbec01c9
  Qubitium-modelcloud authored Jun 15, 2024
  
  bbec01c9
13 Jun, 2024 2 commits
- Higher priority for user input of max_prefill_tokens & format (#540) · fb9296f0
  Ying Sheng authored Jun 12, 2024
  
  fb9296f0
- Fix dependency & crash issues (#539) · 1374334d
  Ying Sheng authored Jun 12, 2024
  
  1374334d
12 Jun, 2024 3 commits
- Fix dependency (#538) · 94aead9e
  Lianmin Zheng authored Jun 12, 2024
  
  94aead9e
- Decode Incrementally (#517) · 9c902b19
  Liangsheng Yin authored Jun 12, 2024
  
  9c902b19
- Fix Regression: Disable p2p for 4090 (#531) · 111991fe
  ZhouXingg authored Jun 12, 2024
```
Co-authored-by: Qubitium <417764+Qubitium@users.noreply.github.com>
```
  111991fe
11 Jun, 2024 2 commits
- Add ChatGLM Model Support (#516) · a8c787d2
  Qubitium authored Jun 12, 2024
```
Co-authored-by: ZX <zx@lbx.dev>
```
  a8c787d2
- [Minor] Correct Optional type hints in api (#526) · 5f283991
  Fabian Preiß authored Jun 12, 2024
  
  5f283991
10 Jun, 2024 1 commit
- Fix missing numpy dependency in pyproject.toml (#524) · 542bc733
  Fabian Preiß authored Jun 10, 2024
  
  542bc733
08 Jun, 2024 3 commits
- Improve doc strings (#518) · f6dbd240
  Lianmin Zheng authored Jun 08, 2024
  
  f6dbd240
- Update version to 0.1.17 (#515) · e8a2327d
  Lianmin Zheng authored Jun 07, 2024
  
  e8a2327d
- Crash the server when error or OOM happens (#514) · 91f93f14
  Lianmin Zheng authored Jun 07, 2024
  
  91f93f14
07 Jun, 2024 5 commits
- Fix rid state map leak + Refractor .finished (#505) · f70f7258
  Qubitium authored Jun 08, 2024
```
Co-authored-by: ZX <zx@lbx.dev>
```
  f70f7258
- Improve logging & fix litellm dependency. (#512) · c0ae70c8
  Lianmin Zheng authored Jun 07, 2024
  
  c0ae70c8
- Litellm Backend (#502) · 87260b7b
  胡译文 authored Jun 08, 2024
  
  87260b7b
- remove redundant pad_input_ids function (#500) · 651a23ee
  Amos You authored Jun 07, 2024
  
  651a23ee
- Update vllm to v0.4.3 (#511) · bf3e271f
  Lianmin Zheng authored Jun 07, 2024
```
Co-authored-by: Qubitium <417764+Qubitium@users.noreply.github.com>
Co-authored-by: ZX <zx@lbx.dev>
```
  bf3e271f
04 Jun, 2024 1 commit
- [Minor] improve code style · 3bc01ac1
  Lianmin Zheng authored Jun 03, 2024
  
  3bc01ac1
01 Jun, 2024 1 commit
- Make the server random by default (#493) · 159cc741
  Lianmin Zheng authored May 31, 2024
  
  159cc741
31 May, 2024 1 commit
- Revert "Make the server random by default" (#492) · 83525a1d
  Ying Sheng authored May 31, 2024
  
  83525a1d
29 May, 2024 1 commit
- Do not set the default value of global random seed (#488) · 80a33ce8
  Lianmin Zheng authored May 29, 2024
  
  80a33ce8
28 May, 2024 2 commits

do not launch workers in parallel · 1a57e416
Lianmin Zheng authored May 27, 2024

1a57e416

Support data parallelism (static) (#480) · 0463f7fb

Ying Sheng authored May 27, 2024


Co-authored-by: Ying Sheng <ying.sheng@databricks.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

0463f7fb

27 May, 2024 2 commits
- improve logging & fix vllm version · 565d7274
  Lianmin Zheng authored May 27, 2024
  
  565d7274
- Improve benchmark scripts & add more models (#484) · 09de730d
  Lianmin Zheng authored May 27, 2024
  
  09de730d
26 May, 2024 2 commits
- Improve benchmark scripts & rename some scripts (#477) · 55c16436
  Lianmin Zheng authored May 26, 2024
  
  55c16436
- [Feat/Fix] Refactoring Llava models into single file (#475) · 2b605ab1
  Li Bo authored May 27, 2024
  
  2b605ab1
25 May, 2024 1 commit
- Optimize retract (#440) · f06e90c2
  Liangsheng Yin authored May 26, 2024
  
  f06e90c2
24 May, 2024 1 commit
- Improve logging & add logit cap (#471) · 2cea6146
  Lianmin Zheng authored May 24, 2024
  
  2cea6146
21 May, 2024 5 commits
- port fp8 mixtral (#460) · 0fafc560
  Lianmin Zheng authored May 21, 2024
  
  0fafc560
- Use model loader from vllm (#459) · 19d2135c
  Lianmin Zheng authored May 21, 2024
  
  19d2135c
- Rename api_num_spec_tokens -> num_api_spec_tokens (#458) · ced77c66
  Lianmin Zheng authored May 20, 2024
  
  ced77c66
- Abort disconnected requests (#457) · 8dbdc018
  Lianmin Zheng authored May 20, 2024
  
  8dbdc018
- Fix openai speculative execution (#456) · 3e684be7
  Ying Sheng authored May 20, 2024
  
  3e684be7
19 May, 2024 2 commits
- openai chat speculative execution (#250) · ec380dfd
  LiviaSun authored May 18, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  ec380dfd
- Fix the broken `--disable-radix-cache` (#451) · 5b647543
  Liangsheng Yin authored May 19, 2024
  
  5b647543
17 May, 2024 3 commits
- Improve error handling & abort disconnected requests (#449) · 8210ec60
  Lianmin Zheng authored May 17, 2024
  
  8210ec60
- Add PUT for generate api (#448) · 5be9eb8a
  Ying Sheng authored May 17, 2024
  
  5be9eb8a
- Simplify port allocation (#447) · c05956e5
  Lianmin Zheng authored May 16, 2024
  
  c05956e5
16 May, 2024 1 commit
- Add finish_reason to OpenAI API (#446) · d75dc20f
  Matthias Gerstgrasser authored May 16, 2024
  
  d75dc20f