Commits · 3520f75fb14d1932fa226aea534937cc87c1b819 · zhaoyu6 / sglang

30 Jul, 2024 1 commit
- Remove inf value for chunked prefill size (#812) · 3520f75f
  Liangsheng Yin authored Jul 29, 2024
  
  3520f75f
29 Jul, 2024 4 commits
- Add support for OpenAI API : offline batch(file) processing (#699) · 084fa54d
  yichuan~ authored Jul 30, 2024
```
Co-authored-by: hnyls2002 <hnyls2002@gmail.com>
```
  084fa54d
- Chunked prefill (#800) · 7cd4f244
  Liangsheng Yin authored Jul 29, 2024
  
  7cd4f244
- Revert "Chunked prefill support" (#799) · 98111fbe
  Ying Sheng authored Jul 29, 2024
  
  98111fbe
- Chunked prefill support (#797) · 2ec39ab7
  Liangsheng Yin authored Jul 29, 2024
  
  2ec39ab7
28 Jul, 2024 3 commits
- chore: add copyright for srt (#790) · dd7e8b94
  Yineng Zhang authored Jul 28, 2024
  
  dd7e8b94
- Allow disabling flashinfer sampling kernel (#778) · 752e6430
  Lianmin Zheng authored Jul 27, 2024
  
  752e6430
- minor refactor: move check server args to server_args.py (#774) · e4db4e5b
  Mingyi authored Jul 27, 2024
  
  e4db4e5b
27 Jul, 2024 1 commit
- Deepseek v2 support (#693) · 679ebcbb
  Liangsheng Yin authored Jul 26, 2024
  
  679ebcbb
23 Jul, 2024 1 commit
- Use min new token ratio at start (#701) · 26868443
  Liangsheng Yin authored Jul 23, 2024
  
  26868443
22 Jul, 2024 1 commit
- Tune params (#696) · c3f1aac8
  Ying Sheng authored Jul 22, 2024
  
  c3f1aac8
21 Jul, 2024 1 commit
- Support gpt-bigcode model class (#681) · caaad53b
  Liangsheng Yin authored Jul 20, 2024
  
  caaad53b
20 Jul, 2024 1 commit
- refactor model loader: initial refactor (#664) · 06487f12
  Ying Sheng authored Jul 20, 2024
  
  06487f12
19 Jul, 2024 2 commits
- Update Readme (#660) · 51fda143
  Ying Sheng authored Jul 19, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  51fda143
- perf: reduce ttft and itl with stream_interval 1 (#658) · ac971ff6
  zhyncs authored Jul 20, 2024
  
  ac971ff6
18 Jul, 2024 1 commit
- Remove the dependency of rpyc (#646) · d774acad
  Mingyi authored Jul 18, 2024
  
  d774acad
15 Jul, 2024 1 commit
- Improve tensor parallel performance (#625) · 6a2941f4
  Ying Sheng authored Jul 15, 2024
```
Co-authored-by: Mingyi <wisclmy0611@gmail.com>
```
  6a2941f4
13 Jul, 2024 1 commit
- Enable cuda graph by default (#612) · 66581596
  Lianmin Zheng authored Jul 13, 2024
  
  66581596
12 Jul, 2024 1 commit
- Clean up the usage of flashinfer (#610) · af4e7910
  Lianmin Zheng authored Jul 12, 2024
  
  af4e7910
07 Jul, 2024 1 commit
- Add `--enable-p2p-check` option (#599) · 5304b4ef
  Liangsheng Yin authored Jul 06, 2024
  
  5304b4ef
05 Jul, 2024 1 commit
- Format (#593) · dc1b8bcf
  Ying Sheng authored Jul 05, 2024
  
  dc1b8bcf
04 Jul, 2024 1 commit
- fix flashinfer & http log level · 63fbef98
  Lianmin Zheng authored Jul 03, 2024
  
  63fbef98
03 Jul, 2024 1 commit
- Update install commands (#583) · c7709d3a
  Lianmin Zheng authored Jul 03, 2024
  
  c7709d3a
02 Jul, 2024 1 commit
- Turn on flashinfer by default (#578) · 9380f50f
  Ying Sheng authored Jul 02, 2024
  
  9380f50f
28 Jun, 2024 1 commit
- Expose dtype argument (#569) · badf3fa0
  Lianmin Zheng authored Jun 27, 2024
  
  badf3fa0
25 Jun, 2024 1 commit
- Add a new arguments log_level_http to control the HTTP logging (#563) · 2187f362
  Lianmin Zheng authored Jun 25, 2024
  
  2187f362
18 Jun, 2024 1 commit
- Multi-node Tensor Parallelism (#550) · 09593e9b
  Ying Sheng authored Jun 17, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  09593e9b
01 Jun, 2024 1 commit
- Make the server random by default (#493) · 159cc741
  Lianmin Zheng authored May 31, 2024
  
  159cc741
31 May, 2024 1 commit
- Revert "Make the server random by default" (#492) · 83525a1d
  Ying Sheng authored May 31, 2024
  
  83525a1d
29 May, 2024 1 commit
- Do not set the default value of global random seed (#488) · 80a33ce8
  Lianmin Zheng authored May 29, 2024
  
  80a33ce8
28 May, 2024 1 commit

Support data parallelism (static) (#480) · 0463f7fb

Ying Sheng authored May 27, 2024


Co-authored-by: Ying Sheng <ying.sheng@databricks.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

0463f7fb

26 May, 2024 1 commit
- Improve benchmark scripts & rename some scripts (#477) · 55c16436
  Lianmin Zheng authored May 26, 2024
  
  55c16436
21 May, 2024 1 commit
- port fp8 mixtral (#460) · 0fafc560
  Lianmin Zheng authored May 21, 2024
  
  0fafc560
13 May, 2024 2 commits
- support llava video (#426) · 0992d85f
  Yuanhan Zhang authored May 14, 2024
  
  0992d85f
- Cache optimizations (#418) · 39191c85
  Liangsheng Yin authored May 13, 2024
  
  39191c85
12 May, 2024 3 commits
- Move openai api server into a separate file (#429) · 3fc97f67
  Lianmin Zheng authored May 12, 2024
  
  3fc97f67
- Fix logit processor bugs (#427) · aee4f523
  Lianmin Zheng authored May 12, 2024
  
  aee4f523
- Clean up (#422) · 7023f413
  Lianmin Zheng authored May 11, 2024
  
  7023f413
09 Apr, 2024 1 commit
- Time cost utils (#355) · 62b3812b
  Liangsheng Yin authored Apr 09, 2024
  
  62b3812b
11 Mar, 2024 1 commit
- Add Support for API Key Authentication (#230) · d5ae2eba
  Alessio Dalla Piazza authored Mar 11, 2024
  
  d5ae2eba