Commits · f424e76d96e9cdc580cf648d7fdc75853a8530e1 · zhaoyu6 / sglang

20 Jul, 2024 3 commits
- Fix illegal tokens during sampling (#676) · f424e76d
  Liangsheng Yin authored Jul 20, 2024
  
  f424e76d
- Support random dataset in bench_serving.py (#669) · 35759efa
  Lianmin Zheng authored Jul 20, 2024
  
  35759efa
- Add support for OpenAI API parallel sampling (#640) · 49c5e0ec
  yichuan~ authored Jul 20, 2024
  
  49c5e0ec
19 Jul, 2024 3 commits
- Fix jump forward when streaming (#665) · 7620cd37
  Liangsheng Yin authored Jul 19, 2024
  
  7620cd37
- Improve docs (#662) · e87c7fd5
  Ying Sheng authored Jul 19, 2024
  
  e87c7fd5
- Detokenize incrementally when streaming (#653) · a9ef49c1
  Liangsheng Yin authored Jul 18, 2024
  
  a9ef49c1
07 Jul, 2024 1 commit
- Fix streaming (#600) · 0877f1e7
  Liangsheng Yin authored Jul 07, 2024
  
  0877f1e7
06 Jul, 2024 1 commit
- * fix(detokenizer_manager.py): fix truncated decoded output (#586) · 26908d95
  uylnap authored Jul 07, 2024
```
Co-authored-by: hnyls2002 <hnyls2002@gmail.com>
```
  26908d95
30 Jun, 2024 1 commit
- Minor fix in compiler & format (#545) · 11616fc6
  sglang authored Jun 29, 2024
  
  11616fc6
13 Jun, 2024 1 commit
- Higher priority for user input of max_prefill_tokens & format (#540) · fb9296f0
  Ying Sheng authored Jun 12, 2024
  
  fb9296f0
12 Jun, 2024 1 commit
- Decode Incrementally (#517) · 9c902b19
  Liangsheng Yin authored Jun 12, 2024
  
  9c902b19
08 Jun, 2024 1 commit
- Improve doc strings (#518) · f6dbd240
  Lianmin Zheng authored Jun 08, 2024
  
  f6dbd240
07 Jun, 2024 1 commit
- Fix rid state map leak + Refractor .finished (#505) · f70f7258
  Qubitium authored Jun 08, 2024
```
Co-authored-by: ZX <zx@lbx.dev>
```
  f70f7258
28 May, 2024 1 commit

Support data parallelism (static) (#480) · 0463f7fb

Ying Sheng authored May 27, 2024


Co-authored-by: Ying Sheng <ying.sheng@databricks.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

0463f7fb

25 May, 2024 1 commit
- Optimize retract (#440) · f06e90c2
  Liangsheng Yin authored May 26, 2024
  
  f06e90c2
19 May, 2024 1 commit
- openai chat speculative execution (#250) · ec380dfd
  LiviaSun authored May 18, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  ec380dfd
17 May, 2024 1 commit
- Improve error handling & abort disconnected requests (#449) · 8210ec60
  Lianmin Zheng authored May 17, 2024
  
  8210ec60
14 May, 2024 1 commit
- Format code (#441) · 690d162d
  Liangsheng Yin authored May 14, 2024
  
  690d162d
12 May, 2024 3 commits
- Allow `input_ids` in the input of the `/generate` endpoint (#363) · 04c0b214
  Shannon Shen authored May 12, 2024
  
  04c0b214
- Minor fix for the import path (#428) · abc548c7
  Lianmin Zheng authored May 12, 2024
  
  abc548c7
- Fix logit processor bugs (#427) · aee4f523
  Lianmin Zheng authored May 12, 2024
  
  aee4f523
30 Apr, 2024 1 commit
- SamplingParams add "spaces_between_special_tokens" argument (#392) · 183df472
  ZhouXingg authored May 01, 2024
  
  183df472
28 Mar, 2024 1 commit
- Logprobs Refractor (#331) · 3842eba5
  Liangsheng Yin authored Mar 28, 2024
  
  3842eba5
15 Feb, 2024 1 commit
- Refactor decoding logprob and add completion_tokens_wo_jump_forward (#189) · 63ba630b
  Cody Yu authored Feb 15, 2024
  
  63ba630b
06 Feb, 2024 2 commits
- Format code & move functions (#155) · 23f05005
  Lianmin Zheng authored Feb 06, 2024
  
  23f05005
- Support decode token logprobs (#130) · a7334aee
  Cody Yu authored Feb 06, 2024
  
  a7334aee
05 Feb, 2024 1 commit
- jump-forward rename (#144) · 26f0bedc
  Liangsheng Yin authored Feb 05, 2024
  
  26f0bedc
30 Jan, 2024 1 commit
- Improve docs & Add JSON decode example (#121) · 97aa9b32
  Lianmin Zheng authored Jan 30, 2024
  
  97aa9b32
26 Jan, 2024 1 commit
- Flush Cache API (#103) · 81561f8e
  Liangsheng Yin authored Jan 26, 2024
  
  81561f8e
24 Jan, 2024 2 commits
- fast regex decode · 01ee0fbc
  Liangsheng Yin authored Jan 25, 2024
```
Auto-detect constant str path in regex FSM, then extend instead.
```
  01ee0fbc
- Llava-hd Support (#92) · c6576e82
  shiyi.c_98 authored Jan 24, 2024
```
Co-authored-by: Haotian Liu <liuhaotian.cn@gmail.com>
```
  c6576e82
23 Jan, 2024 1 commit
- Return logprob for choices (#87) · 9a16fea0
  Lianmin Zheng authored Jan 23, 2024
  
  9a16fea0
08 Jan, 2024 1 commit

release initial code · 22085081

Lianmin Zheng authored Jan 08, 2024


Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: parasol-aser <3848358+parasol-aser@users.noreply.github.com>
Co-authored-by: LiviaSun <33578456+ChuyueSun@users.noreply.github.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

22085081