Commits · 3de2f30a27b1d9ffef6dfddcdcc7877c2a2dc857 · change / sglang

17 Jul, 2024 2 commits
- Flashinfer sample kernel (#617) · 3de2f30a
  Liangsheng Yin authored Jul 17, 2024
  
  3de2f30a
- misc: add pre-commit config (#637) · 2e341cd4
  zhyncs authored Jul 18, 2024
  
  2e341cd4
16 Jul, 2024 1 commit
- Disable NCCL_NVLS by default (#631) · 0aa189f1
  Ying Sheng authored Jul 16, 2024
  
  0aa189f1
15 Jul, 2024 1 commit
- Improve tensor parallel performance (#625) · 6a2941f4
  Ying Sheng authored Jul 15, 2024
```
Co-authored-by: Mingyi <wisclmy0611@gmail.com>
```
  6a2941f4
13 Jul, 2024 1 commit
- Enable cuda graph by default (#612) · 66581596
  Lianmin Zheng authored Jul 13, 2024
  
  66581596
05 Jul, 2024 1 commit
- Format (#593) · dc1b8bcf
  Ying Sheng authored Jul 05, 2024
  
  dc1b8bcf
04 Jul, 2024 1 commit
- fix flashinfer & http log level · 63fbef98
  Lianmin Zheng authored Jul 03, 2024
  
  63fbef98
03 Jul, 2024 2 commits
- 2x performance improvement for large prefill & Fix workspace conflicts (#579) · 2a754e57
  Ying Sheng authored Jul 03, 2024
  
  2a754e57
- fix the broken server args (#585) · 96c503eb
  Liangsheng Yin authored Jul 04, 2024
  
  96c503eb
02 Jul, 2024 1 commit
- Fix flashinfer version (#576) · d9ac6392
  Yueyang Pan authored Jul 02, 2024
  
  d9ac6392
25 Jun, 2024 1 commit
- Add a new arguments log_level_http to control the HTTP logging (#563) · 2187f362
  Lianmin Zheng authored Jun 25, 2024
  
  2187f362
22 Jun, 2024 1 commit
- Clean up logits processor (#558) · 303ef888
  Lianmin Zheng authored Jun 22, 2024
  
  303ef888
21 Jun, 2024 1 commit
- Update flashinfer to 0.0.5 (#554) · b7e2f800
  Lianmin Zheng authored Jun 20, 2024
  
  b7e2f800
18 Jun, 2024 1 commit
- Multi-node Tensor Parallelism (#550) · 09593e9b
  Ying Sheng authored Jun 17, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  09593e9b
13 Jun, 2024 1 commit
- Higher priority for user input of max_prefill_tokens & format (#540) · fb9296f0
  Ying Sheng authored Jun 12, 2024
  
  fb9296f0
08 Jun, 2024 2 commits
- Improve doc strings (#518) · f6dbd240
  Lianmin Zheng authored Jun 08, 2024
  
  f6dbd240
- Crash the server when error or OOM happens (#514) · 91f93f14
  Lianmin Zheng authored Jun 07, 2024
  
  91f93f14
28 May, 2024 1 commit

Support data parallelism (static) (#480) · 0463f7fb

Ying Sheng authored May 27, 2024


Co-authored-by: Ying Sheng <ying.sheng@databricks.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

0463f7fb

24 May, 2024 1 commit
- Improve logging & add logit cap (#471) · 2cea6146
  Lianmin Zheng authored May 24, 2024
  
  2cea6146
21 May, 2024 1 commit
- Abort disconnected requests (#457) · 8dbdc018
  Lianmin Zheng authored May 20, 2024
  
  8dbdc018
19 May, 2024 1 commit
- openai chat speculative execution (#250) · ec380dfd
  LiviaSun authored May 18, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  ec380dfd
17 May, 2024 3 commits
- Improve error handling & abort disconnected requests (#449) · 8210ec60
  Lianmin Zheng authored May 17, 2024
  
  8210ec60
- Add PUT for generate api (#448) · 5be9eb8a
  Ying Sheng authored May 17, 2024
  
  5be9eb8a
- Simplify port allocation (#447) · c05956e5
  Lianmin Zheng authored May 16, 2024
  
  c05956e5
14 May, 2024 1 commit
- Format code (#441) · 690d162d
  Liangsheng Yin authored May 14, 2024
  
  690d162d
13 May, 2024 2 commits
- support llava video (#426) · 0992d85f
  Yuanhan Zhang authored May 14, 2024
  
  0992d85f
- Handle truncation errors (#436) · 5dc55a5f
  Lianmin Zheng authored May 13, 2024
  
  5dc55a5f
12 May, 2024 3 commits
- Move openai api server into a separate file (#429) · 3fc97f67
  Lianmin Zheng authored May 12, 2024
  
  3fc97f67
- Fix logit processor bugs (#427) · aee4f523
  Lianmin Zheng authored May 12, 2024
  
  aee4f523
- Clean up (#422) · 7023f413
  Lianmin Zheng authored May 11, 2024
  
  7023f413
11 May, 2024 1 commit
- Compat with latest VLLM 0.4.2 main + fork.number rename + Flashinfer 0.0.4 (#380) · 33b242df
  Qubitium authored May 12, 2024
```
Co-authored-by: ZX <zx@lbx.dev>
Co-authored-by: ZhouXingg <165115237+ZhouXingg@users.noreply.github.com>
```
  33b242df
25 Apr, 2024 1 commit
- Minor: style improvement of radix_cache and memory_pool (#395) · 19818b9c
  Liangsheng Yin authored Apr 26, 2024
  
  19818b9c
23 Apr, 2024 1 commit
- Revert removing the unused imports (#385) · 150d7020
  Liangsheng Yin authored Apr 23, 2024
  
  150d7020
22 Apr, 2024 1 commit
- add `.isort.cfg` (#378) · 9acc6e35
  Liangsheng Yin authored Apr 22, 2024
  
  9acc6e35
09 Apr, 2024 1 commit
- Time cost utils (#355) · 62b3812b
  Liangsheng Yin authored Apr 09, 2024
  
  62b3812b
28 Mar, 2024 1 commit
- Logprobs Refractor (#331) · 3842eba5
  Liangsheng Yin authored Mar 28, 2024
  
  3842eba5
11 Mar, 2024 4 commits
- Fix Runtime missing some ServerArgs options (#281) · b2eb0805
  Qubitium authored Mar 11, 2024
  
  b2eb0805
- Fix RuntimeEndpoint (#279) · 13662fd5
  Lianmin Zheng authored Mar 11, 2024
  
  13662fd5
- Add Support for API Key Authentication (#230) · d5ae2eba
  Alessio Dalla Piazza authored Mar 11, 2024
  
  d5ae2eba
- Organize `server_args` (#277) · 1b355479
  Liangsheng Yin authored Mar 11, 2024
  
  1b355479