Commits · c614dbdf9518704328b45057aa41d403685d8f1c · change / sglang

05 Mar, 2025 1 commit
- [Eagle] Refactor eagle speculative decoding (#3986) · d3d4d767
  Ying Sheng authored Mar 05, 2025
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  d3d4d767
03 Mar, 2025 1 commit

Support penalty in overlap mode; return logprob with chunked prefill; improve... · ac238727

Lianmin Zheng authored Mar 03, 2025


Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

ac238727

28 Feb, 2025 1 commit
- [Feature] SPMD for SGLang + Verl (#3852) · e3e0bc50
  fzyzcjy authored Mar 01, 2025
  
  e3e0bc50
19 Jan, 2025 1 commit
- fix missing revision arg when loading tokenizer (#2982) · 8b6a4486
  giorgiopiatti-dfinity authored Jan 19, 2025
  
  8b6a4486
16 Jan, 2025 1 commit
- Support multi-node DP attention (#2925) · 8b6ce52e
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: dhou-xai <dhou@x.ai>
```
  8b6ce52e
02 Jan, 2025 2 commits
- Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) · ad20b795
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  ad20b795
- Speed up `update_weights_from_tensor` (#2695) · 9183c23e
  fzyzcjy authored Jan 02, 2025
  
  9183c23e
31 Dec, 2024 1 commit
- Eagle speculative decoding part 2: Fix cuda graph + DP attention hanging (#2684) · b0524c37
  Lianmin Zheng authored Dec 31, 2024
```
Co-authored-by: yukavio <kavioyu@gmail.com>
```
  b0524c37
28 Dec, 2024 1 commit
- Add `update_weights_from_tensor` (#2631) · fd28640d
  fzyzcjy authored Dec 29, 2024
  
  fd28640d
02 Dec, 2024 2 commits
- Update model_loader deps and qqq quantization deps (#2220) (#2318) · 85e1a6f3
  Yineng Zhang authored Dec 02, 2024
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  85e1a6f3
- Online weight updates from torch.distributed (#2279) · 983bfcf3
  Chayenne authored Dec 01, 2024
  
  983bfcf3
30 Nov, 2024 2 commits
- Add get weights by parameter name for llama (#2266) · 7d1485d3
  Chayenne authored Nov 29, 2024
  
  7d1485d3
- udate weights from disk (#2265) · 7d5d1d3d
  Chayenne authored Nov 29, 2024
  
  7d5d1d3d
22 Nov, 2024 1 commit
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
19 Nov, 2024 2 commits
- Use cuda event wait and synchronization instead of busy waiting (#2089) · b7a065ea
  Lianmin Zheng authored Nov 19, 2024
  
  b7a065ea
- Simplify logits penalizer (#2086) · b1104538
  Lianmin Zheng authored Nov 18, 2024
  
  b1104538
18 Nov, 2024 1 commit
- Support cuda graph for DP attention (#2061) · 62832bb2
  Ke Bao authored Nov 18, 2024
  
  62832bb2
16 Nov, 2024 1 commit
- Support DP MLA (#1970) · 976bc302
  Ke Bao authored Nov 16, 2024
  
  976bc302
03 Nov, 2024 1 commit
- Unify the model type checking (#1905) · 0abbf289
  Lianmin Zheng authored Nov 03, 2024
  
  0abbf289
21 Oct, 2024 1 commit
- Update `max_req_len` and `max_req_input_len` (#1748) · 5e1558f1
  Liangsheng Yin authored Oct 21, 2024
  
  5e1558f1
20 Oct, 2024 3 commits
- Split the overlapped version of TpModelWorkerClient into a separate file (#1726) · b48edff6
  Lianmin Zheng authored Oct 20, 2024
  
  b48edff6
- Unify the memory pool api and tp worker API (#1724) · 59cbf476
  Lianmin Zheng authored Oct 19, 2024
  
  59cbf476
- Simplify the interface of tp_worker (#1718) · 12cad0fe
  Lianmin Zheng authored Oct 19, 2024
  
  12cad0fe
19 Oct, 2024 1 commit
- Fix the race condition in overlap mode (#1712) · 769bf11c
  Lianmin Zheng authored Oct 19, 2024
  
  769bf11c
16 Oct, 2024 1 commit
- Launch a thread to overlap CPU and GPU (#1687) · dbec2f18
  Lianmin Zheng authored Oct 16, 2024
  
  dbec2f18
13 Oct, 2024 1 commit
- Simplify the event loop and expose `--num-continuous-decode-steps` as an argument (#1652) · 7ee6c259
  Lianmin Zheng authored Oct 12, 2024
  
  7ee6c259
30 Sep, 2024 4 commits
- Clean up batch data structures: Introducing ModelWorkerBatch (#1544) · 63ba2f8d
  Lianmin Zheng authored Sep 30, 2024
  
  63ba2f8d
- Rename InputMetadata -> ForwardBatch (#1543) · 36d5acfc
  Lianmin Zheng authored Sep 30, 2024
  
  36d5acfc
- Let ModelRunner take InputMetadata as input, instead of ScheduleBatch (#1541) · 3f0fe08d
  Lianmin Zheng authored Sep 29, 2024
  
  3f0fe08d
- Move scheduler code from tp_worker.py to scheduler.py (#1538) · f86c1e61
  Lianmin Zheng authored Sep 29, 2024
  
  f86c1e61
29 Sep, 2024 2 commits
- Improve process creation (#1534) · 04868543
  Lianmin Zheng authored Sep 29, 2024
  
  04868543
- Organize image inputs (#1531) · fd9ad817
  Liangsheng Yin authored Sep 28, 2024
  
  fd9ad817
28 Sep, 2024 1 commit
- [Feature] Support reward model LxzGordon/URM-LLaMa-3.1-8B (#1525) · 9aa6553d
  Ying Sheng authored Sep 27, 2024
  
  9aa6553d
23 Sep, 2024 1 commit
- Fix the overhead due to penalizer in bench_latency (#1496) · 2854a5ea
  Lianmin Zheng authored Sep 23, 2024
  
  2854a5ea
17 Sep, 2024 2 commits
- fix schedule bug (#1450) · 36078fb2
  Liangsheng Yin authored Sep 17, 2024
  
  36078fb2
- Clean up model loader (#1440) · 27b557ae
  Lianmin Zheng authored Sep 16, 2024
  
  27b557ae
16 Sep, 2024 1 commit
- Add constrained_json_whitespace_pattern to ServerArgs (#1438) · 93dffd69
  zifeitong authored Sep 16, 2024
  
  93dffd69
15 Sep, 2024 1 commit
- [Fix] Fix logprob and normalized_logprob (#1428) · 9ba1f097
  Lianmin Zheng authored Sep 15, 2024
  
  9ba1f097
14 Sep, 2024 1 commit
- Optimize conflicts between CUDA graph and vocab mask tensors (#1392) · 70b68029
  Liangsheng Yin authored Sep 13, 2024
  
  70b68029
13 Sep, 2024 1 commit
- Make stop reason a dict instead of str (#1407) · b912de11
  Lianmin Zheng authored Sep 12, 2024
  
  b912de11