Commits · eb1ae6ae0c7e044ebdb405992efebf7b28042370 · change / sglang

25 Jun, 2024 1 commit
- Add sglang.bench_latency for offline benchmark (#564) · eb1ae6ae
  Lianmin Zheng authored Jun 25, 2024
  
  eb1ae6ae
24 Jun, 2024 1 commit
- Update test_flashinfer (#560) · 05471f21
  Liangsheng Yin authored Jun 24, 2024
  
  05471f21
22 Jun, 2024 1 commit
- Add LlamaForClassification (#559) · 1fa15099
  Lianmin Zheng authored Jun 22, 2024
  
  1fa15099
13 Jun, 2024 1 commit
- Higher priority for user input of max_prefill_tokens & format (#540) · fb9296f0
  Ying Sheng authored Jun 12, 2024
  
  fb9296f0
07 Jun, 2024 1 commit
- Litellm Backend (#502) · 87260b7b
  胡译文 authored Jun 08, 2024
  
  87260b7b
28 May, 2024 1 commit

Support data parallelism (static) (#480) · 0463f7fb

Ying Sheng authored May 27, 2024


Co-authored-by: Ying Sheng <ying.sheng@databricks.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

0463f7fb

21 May, 2024 1 commit
- Fix openai speculative execution (#456) · 3e684be7
  Ying Sheng authored May 20, 2024
  
  3e684be7
14 May, 2024 1 commit
- Format code (#441) · 690d162d
  Liangsheng Yin authored May 14, 2024
  
  690d162d
13 May, 2024 1 commit
- Handle truncation errors (#436) · 5dc55a5f
  Lianmin Zheng authored May 13, 2024
  
  5dc55a5f
12 May, 2024 3 commits
- Misc fixes (#432) · 6e09cf6a
  Lianmin Zheng authored May 12, 2024
  
  6e09cf6a
- Fix logit processor bugs (#427) · aee4f523
  Lianmin Zheng authored May 12, 2024
  
  aee4f523
- Clean up (#422) · 7023f413
  Lianmin Zheng authored May 11, 2024
  
  7023f413
25 Apr, 2024 1 commit
- Minor: style improvement of radix_cache and memory_pool (#395) · 19818b9c
  Liangsheng Yin authored Apr 26, 2024
  
  19818b9c
23 Apr, 2024 1 commit
- Revert removing the unused imports (#385) · 150d7020
  Liangsheng Yin authored Apr 23, 2024
  
  150d7020
22 Apr, 2024 1 commit
- add `.isort.cfg` (#378) · 9acc6e35
  Liangsheng Yin authored Apr 22, 2024
  
  9acc6e35
16 Apr, 2024 2 commits
- Fix issue #367 – System message not supported for Anthropic (anthropic.BadRequestError) (#368) · 2b6d9991
  Fronx authored Apr 16, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  2b6d9991
- Fix commandr import; format code · 65501a9c
  Lianmin Zheng authored Apr 16, 2024
  
  65501a9c
09 Apr, 2024 1 commit
- Time cost utils (#355) · 62b3812b
  Liangsheng Yin authored Apr 09, 2024
  
  62b3812b
28 Mar, 2024 1 commit
- Logprobs Refractor (#331) · 3842eba5
  Liangsheng Yin authored Mar 28, 2024
  
  3842eba5
22 Mar, 2024 1 commit
- Use Anthropic messages API (#304) · e57f0792
  Jani Monoses authored Mar 22, 2024
  
  e57f0792
11 Mar, 2024 3 commits
- Update version to v0.1.13 (#280) · 4aa5dd2c
  Lianmin Zheng authored Mar 11, 2024
  
  4aa5dd2c
- Organize `server_args` (#277) · 1b355479
  Liangsheng Yin authored Mar 11, 2024
  
  1b355479
- Improve gemma and documentations (#278) · faba293a
  Lianmin Zheng authored Mar 11, 2024
  
  faba293a
11 Feb, 2024 2 commits
- Fix the chat template for llava-v1.6-34b & format code (#177) · c51020cf
  Lianmin Zheng authored Feb 11, 2024
  
  c51020cf
- Support extra field regex in OpenAI API (#172) · 50afed4e
  Cody Yu authored Feb 10, 2024
  
  50afed4e
09 Feb, 2024 1 commit
- import outlines (#168) · 37b42297
  Liangsheng Yin authored Feb 09, 2024
  
  37b42297
06 Feb, 2024 2 commits
- Format code & move functions (#155) · 23f05005
  Lianmin Zheng authored Feb 06, 2024
  
  23f05005
- Support decode token logprobs (#130) · a7334aee
  Cody Yu authored Feb 06, 2024
  
  a7334aee
05 Feb, 2024 1 commit
- jump-forward rename (#144) · 26f0bedc
  Liangsheng Yin authored Feb 05, 2024
  
  26f0bedc
30 Jan, 2024 3 commits
- [Feature] Adds basic support for image content in OpenAI chat routes (#113) · 1d0fbe8e
  Keith Stevens authored Jan 30, 2024
  
  1d0fbe8e
- Improve docs & Add JSON decode example (#121) · 97aa9b32
  Lianmin Zheng authored Jan 30, 2024
  
  97aa9b32
- Improve the control of streaming and improve the first token latency in streaming (#117) · 6f560c76
  Lianmin Zheng authored Jan 29, 2024
  
  6f560c76
25 Jan, 2024 1 commit
- support speculative execution for openai API (#48) · 23950056
  parasol-aser authored Jan 25, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  23950056
24 Jan, 2024 1 commit
- fast regex decode · 01ee0fbc
  Liangsheng Yin authored Jan 25, 2024
```
Auto-detect constant str path in regex FSM, then extend instead.
```
  01ee0fbc
23 Jan, 2024 1 commit
- Return logprob for choices (#87) · 9a16fea0
  Lianmin Zheng authored Jan 23, 2024
  
  9a16fea0
19 Jan, 2024 3 commits
- Fix the possible bug of decode out of memory (#36) · 40ab1f01
  Liangsheng Yin authored Jan 20, 2024
  
  40ab1f01
- Support v1/chat/completions (#50) · 23471f9a
  Cody Yu authored Jan 18, 2024
  
  23471f9a
- Support stream=True in v1/completions (#49) · 61d4c939
  Cody Yu authored Jan 18, 2024
  
  61d4c939
17 Jan, 2024 2 commits
- Improve docs & Rename Gemini -> VertexAI (#19) · bf51ddc6
  Lianmin Zheng authored Jan 17, 2024
  
  bf51ddc6
- Gemini Backend (#9) · fd7c4792
  shiyi.c_98 authored Jan 16, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  fd7c4792