Commits · 5e1558f1f26f0fc060ea261c9e81b767dc8e3fb9 · zhaoyu6 / sglang

"docs/vscode:/vscode.git/clone" did not exist on "a98d86d93b09d8a2eaf33911de3f985561f38aeb"

21 Oct, 2024 1 commit
- Update `max_req_len` and `max_req_input_len` (#1748) · 5e1558f1
  Liangsheng Yin authored Oct 21, 2024
  
  5e1558f1
20 Oct, 2024 3 commits
- Split the overlapped version of TpModelWorkerClient into a separate file (#1726) · b48edff6
  Lianmin Zheng authored Oct 20, 2024
  
  b48edff6
- Unify the memory pool api and tp worker API (#1724) · 59cbf476
  Lianmin Zheng authored Oct 19, 2024
  
  59cbf476
- Simplify the interface of tp_worker (#1718) · 12cad0fe
  Lianmin Zheng authored Oct 19, 2024
  
  12cad0fe
19 Oct, 2024 1 commit
- Fix the race condition in overlap mode (#1712) · 769bf11c
  Lianmin Zheng authored Oct 19, 2024
  
  769bf11c
16 Oct, 2024 1 commit
- Launch a thread to overlap CPU and GPU (#1687) · dbec2f18
  Lianmin Zheng authored Oct 16, 2024
  
  dbec2f18
13 Oct, 2024 1 commit
- Simplify the event loop and expose `--num-continuous-decode-steps` as an argument (#1652) · 7ee6c259
  Lianmin Zheng authored Oct 12, 2024
  
  7ee6c259
30 Sep, 2024 4 commits
- Clean up batch data structures: Introducing ModelWorkerBatch (#1544) · 63ba2f8d
  Lianmin Zheng authored Sep 30, 2024
  
  63ba2f8d
- Rename InputMetadata -> ForwardBatch (#1543) · 36d5acfc
  Lianmin Zheng authored Sep 30, 2024
  
  36d5acfc
- Let ModelRunner take InputMetadata as input, instead of ScheduleBatch (#1541) · 3f0fe08d
  Lianmin Zheng authored Sep 29, 2024
  
  3f0fe08d
- Move scheduler code from tp_worker.py to scheduler.py (#1538) · f86c1e61
  Lianmin Zheng authored Sep 29, 2024
  
  f86c1e61
29 Sep, 2024 2 commits
- Improve process creation (#1534) · 04868543
  Lianmin Zheng authored Sep 29, 2024
  
  04868543
- Organize image inputs (#1531) · fd9ad817
  Liangsheng Yin authored Sep 28, 2024
  
  fd9ad817
28 Sep, 2024 1 commit
- [Feature] Support reward model LxzGordon/URM-LLaMa-3.1-8B (#1525) · 9aa6553d
  Ying Sheng authored Sep 27, 2024
  
  9aa6553d
23 Sep, 2024 1 commit
- Fix the overhead due to penalizer in bench_latency (#1496) · 2854a5ea
  Lianmin Zheng authored Sep 23, 2024
  
  2854a5ea
17 Sep, 2024 2 commits
- fix schedule bug (#1450) · 36078fb2
  Liangsheng Yin authored Sep 17, 2024
  
  36078fb2
- Clean up model loader (#1440) · 27b557ae
  Lianmin Zheng authored Sep 16, 2024
  
  27b557ae
16 Sep, 2024 1 commit
- Add constrained_json_whitespace_pattern to ServerArgs (#1438) · 93dffd69
  zifeitong authored Sep 16, 2024
  
  93dffd69
15 Sep, 2024 1 commit
- [Fix] Fix logprob and normalized_logprob (#1428) · 9ba1f097
  Lianmin Zheng authored Sep 15, 2024
  
  9ba1f097
14 Sep, 2024 1 commit
- Optimize conflicts between CUDA graph and vocab mask tensors (#1392) · 70b68029
  Liangsheng Yin authored Sep 13, 2024
  
  70b68029
13 Sep, 2024 1 commit
- Make stop reason a dict instead of str (#1407) · b912de11
  Lianmin Zheng authored Sep 12, 2024
  
  b912de11
12 Sep, 2024 1 commit
- [Feature] Initial support for multi-LoRA serving (#1307) · 71221692
  Ying Sheng authored Sep 12, 2024
  
  71221692
11 Sep, 2024 1 commit
- Deprecate --disable-flashinfer and introduce --attention-backend (#1380) · 46094e0c
  Lianmin Zheng authored Sep 10, 2024
  
  46094e0c
10 Sep, 2024 1 commit
- [Minor] move triton attention kernels into a separate folder (#1379) · 3a6e8b6d
  Lianmin Zheng authored Sep 10, 2024
  
  3a6e8b6d
09 Sep, 2024 3 commits
- Unify forward mode (#1360) · 69b3bb9a
  Liangsheng Yin authored Sep 09, 2024
  
  69b3bb9a
- [Minor] Many cleanup (#1357) · e4d68afc
  Lianmin Zheng authored Sep 09, 2024
  
  e4d68afc
- [Feat] Add modalities for vision server when handling pixel values for llava (#1346) · 662ecd93
  Kaichen Zhang - NTU authored Sep 09, 2024
  
  662ecd93
08 Sep, 2024 1 commit
- Fix some online scheduling delay (#1345) · 05bea688
  Liangsheng Yin authored Sep 07, 2024
  
  05bea688
05 Sep, 2024 1 commit
- Optimize schedule (#1339) · ab4a83b2
  Liangsheng Yin authored Sep 05, 2024
  
  ab4a83b2
01 Sep, 2024 1 commit
- [Chore] Rename model_overide_args to model_override_args (#1284) · 08360553
  Kai-Hsun Chen authored Sep 01, 2024
```
Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  08360553
29 Aug, 2024 1 commit
- Sampler cudagraph (#1253) · 381dd57b
  Liangsheng Yin authored Aug 28, 2024
  
  381dd57b
28 Aug, 2024 2 commits
- [Fix] Fix llava on multi images (#1247) · bf53bf51
  Lianmin Zheng authored Aug 28, 2024
  
  bf53bf51
- hotfix: revert sampler CUDA Graph (#1242) · f25f4dfd
  Yineng Zhang authored Aug 28, 2024
  
  f25f4dfd
26 Aug, 2024 2 commits
- [FEAT] JSON constrained support (#1125) · 9935f97b
  havetc authored Aug 26, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  9935f97b
- Move sampler into CUDA graph (#1201) · 75ce37f4
  Liangsheng Yin authored Aug 26, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  75ce37f4
25 Aug, 2024 2 commits
- [Minor] Improve the function organization in TokenizerManager & improve loggers (#1208) · 90227800
  Lianmin Zheng authored Aug 25, 2024
  
  90227800
- Support Alibaba-NLP/gte-Qwen2-7B-instruct embedding Model (#1186) · 30b4f771
  Chayenne authored Aug 26, 2024
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  30b4f771
23 Aug, 2024 1 commit

[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2... · a5b14ad0

Kaichen Zhang - NTU authored Aug 24, 2024


[Feat/WIP] add llava-onevision, with support for (1) siglip encoder, (2) qwen2 decoder (3) openai api compatible server. (#1123)
Co-authored-by: Bo Li <drluodian@gmail.com>

a5b14ad0

21 Aug, 2024 2 commits
- Improve code style of sampler (#1168) · 83e23c69
  Liangsheng Yin authored Aug 21, 2024
  
  83e23c69
- Improve multi-node stability (#1171) · bea2bb9e
  Lianmin Zheng authored Aug 20, 2024
  
  bea2bb9e