Commits · 2422de5193bfb00ed73b767957448de528dfb14f · change / sglang

04 Oct, 2024 1 commit
- Make input_ids a torch.Tensor (#1568) · 45473d4b
  Lianmin Zheng authored Oct 04, 2024
  
  45473d4b
03 Oct, 2024 1 commit
- [Fix] Move ScheduleBatch out of SamplingInfo (#1556) · 317631ca
  Lianmin Zheng authored Oct 02, 2024
  
  317631ca
01 Oct, 2024 1 commit
- Fix bugs of `logprobs_nums` (#1548) · b88ea90d
  Liangsheng Yin authored Sep 30, 2024
  
  b88ea90d
30 Sep, 2024 4 commits
- Clean up batch data structures: Introducing ModelWorkerBatch (#1544) · 63ba2f8d
  Lianmin Zheng authored Sep 30, 2024
  
  63ba2f8d
- Rename InputMetadata -> ForwardBatch (#1543) · 36d5acfc
  Lianmin Zheng authored Sep 30, 2024
  
  36d5acfc
- Let ModelRunner take InputMetadata as input, instead of ScheduleBatch (#1541) · 3f0fe08d
  Lianmin Zheng authored Sep 29, 2024
  
  3f0fe08d
- Move scheduler code from tp_worker.py to scheduler.py (#1538) · f86c1e61
  Lianmin Zheng authored Sep 29, 2024
  
  f86c1e61
29 Sep, 2024 1 commit
- Organize image inputs (#1531) · fd9ad817
  Liangsheng Yin authored Sep 28, 2024
  
  fd9ad817
23 Sep, 2024 1 commit
- Fix the overhead due to penalizer in bench_latency (#1496) · 2854a5ea
  Lianmin Zheng authored Sep 23, 2024
  
  2854a5ea
17 Sep, 2024 2 commits
- Enable MLA by default (#1447) · c6b6d2e7
  Ke Bao authored Sep 17, 2024
  
  c6b6d2e7
- Simplify sampler and its error handling (#1441) · 2fa5cec7
  Lianmin Zheng authored Sep 16, 2024
  
  2fa5cec7
15 Sep, 2024 1 commit
- [Fix] Fix logprob and normalized_logprob (#1428) · 9ba1f097
  Lianmin Zheng authored Sep 15, 2024
  
  9ba1f097
14 Sep, 2024 1 commit
- Optimize conflicts between CUDA graph and vocab mask tensors (#1392) · 70b68029
  Liangsheng Yin authored Sep 13, 2024
  
  70b68029
13 Sep, 2024 1 commit
- Make stop reason a dict instead of str (#1407) · b912de11
  Lianmin Zheng authored Sep 12, 2024
  
  b912de11
12 Sep, 2024 1 commit
- [Feature] Initial support for multi-LoRA serving (#1307) · 71221692
  Ying Sheng authored Sep 12, 2024
  
  71221692
11 Sep, 2024 3 commits
- Refactor attention backend (#1381) · fec185ce
  Lianmin Zheng authored Sep 11, 2024
  
  fec185ce
- Organize flashinfer indices update (#1378) · 144bc70f
  Liangsheng Yin authored Sep 10, 2024
  
  144bc70f
- Deprecate --disable-flashinfer and introduce --attention-backend (#1380) · 46094e0c
  Lianmin Zheng authored Sep 10, 2024
  
  46094e0c
10 Sep, 2024 1 commit
- Fix vocab mask update bug (#1376) · fbb4754c
  Liangsheng Yin authored Sep 10, 2024
  
  fbb4754c
09 Sep, 2024 2 commits
- Unify forward mode (#1360) · 69b3bb9a
  Liangsheng Yin authored Sep 09, 2024
  
  69b3bb9a
- [Feat] Add modalities for vision server when handling pixel values for llava (#1346) · 662ecd93
  Kaichen Zhang - NTU authored Sep 09, 2024
  
  662ecd93
03 Sep, 2024 1 commit
- [Fix] Fix select by ensuring each request has at least one token (#1318) · 1e495e08
  Lianmin Zheng authored Sep 03, 2024
  
  1e495e08
29 Aug, 2024 1 commit
- Sampler cudagraph (#1253) · 381dd57b
  Liangsheng Yin authored Aug 28, 2024
  
  381dd57b
28 Aug, 2024 2 commits
- [Fix] Fix llava on multi images (#1247) · bf53bf51
  Lianmin Zheng authored Aug 28, 2024
  
  bf53bf51
- hotfix: revert sampler CUDA Graph (#1242) · f25f4dfd
  Yineng Zhang authored Aug 28, 2024
  
  f25f4dfd
27 Aug, 2024 1 commit
- [FIX] Wrong logger (#1230) · 909f3436
  havetc authored Aug 27, 2024
  
  909f3436
26 Aug, 2024 2 commits
- [FEAT] JSON constrained support (#1125) · 9935f97b
  havetc authored Aug 26, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  9935f97b
- Move sampler into CUDA graph (#1201) · 75ce37f4
  Liangsheng Yin authored Aug 26, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  75ce37f4
24 Aug, 2024 1 commit
- Cleanup readme, llava examples, usage examples and nccl init (#1194) · f6af3a65
  Lianmin Zheng authored Aug 24, 2024
  
  f6af3a65
22 Aug, 2024 1 commit
- Fix broken penalty (#1184) · 364d3d72
  Liangsheng Yin authored Aug 22, 2024
  
  364d3d72
21 Aug, 2024 3 commits
- Improve code style of sampler (#1168) · 83e23c69
  Liangsheng Yin authored Aug 21, 2024
  
  83e23c69
- Support min-p sampling (#1167) · 068e9eae
  intervitens authored Aug 22, 2024
  
  068e9eae
- Improve multi-node stability (#1171) · bea2bb9e
  Lianmin Zheng authored Aug 20, 2024
  
  bea2bb9e
20 Aug, 2024 1 commit
- Improve docs and warnings (#1164) · a8ae6403
  Lianmin Zheng authored Aug 20, 2024
  
  a8ae6403
16 Aug, 2024 2 commits
- Mixed style of chunked prefill (#1013) · 3694f8f9
  Liangsheng Yin authored Aug 16, 2024
  
  3694f8f9
- Fix the deadlock in multi-node tp (#1122) · 5a261bd0
  Lianmin Zheng authored Aug 16, 2024
  
  5a261bd0
15 Aug, 2024 1 commit
- Support `stop_token_ids` in sglang API (#1092) · 73cf6834
  Liangsheng Yin authored Aug 14, 2024
  
  73cf6834
14 Aug, 2024 1 commit
- Use `dtype` to control generate (#1082) · a34dd86a
  Liangsheng Yin authored Aug 14, 2024
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  a34dd86a
12 Aug, 2024 1 commit
- Fix the prefix indices (#1037) · 7de60345
  Liangsheng Yin authored Aug 11, 2024
  
  7de60345
10 Aug, 2024 1 commit
- Fix `input_ids` && rename to `fill_ids` (#1021) · 43fbb6d9
  Liangsheng Yin authored Aug 10, 2024
  
  43fbb6d9