Commits · 61a4680494902efeef00ce7559dca3869bda3881 · change / sglang · GitLab

31 Jul, 2025 1 commit
- [Feature] Hybrid EP and TP (#8590) · 7a1f7fc5
  Cheng Wan authored Jul 31, 2025
  
  7a1f7fc5
25 Jul, 2025 1 commit
- DP Enhancement (#8280) · c0fb25e9
  Cheng Wan authored Jul 24, 2025
  
  c0fb25e9
05 Jul, 2025 1 commit
- Fix incorrect spec_num_draft_tokens in draft_extend (#7757) · 6c903611
  Cheng Wan authored Jul 05, 2025
  
  6c903611
01 Jul, 2025 1 commit

support llama4 eagle3 (#6985) · 886d3449

lukec authored Jul 01, 2025


Co-authored-by: shuaills <shishuaiuoe@gmail.com>
Co-authored-by: Shenggui Li <somerlee.9@gmail.com>
Co-authored-by: Yingyi Huang <yingyihuang2000@outlook.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>

886d3449

24 Jun, 2025 1 commit

Perormance: Enable cuda graph for dp idle batch (#7269) · ed0a0b69

u4lr451 authored Jun 24, 2025


Co-authored-by: austindeng <austindeng@tencent.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: ch-wan <cwan39@gatech.edu>

ed0a0b69

23 Jun, 2025 1 commit
- Fix dtype for idle input in spec decoding (#7456) · f8d48fd3
  Cheng Wan authored Jun 23, 2025
  
  f8d48fd3
17 Jun, 2025 1 commit

feat: mtp support dp-attention (#6081) · 10d60cd4

u4lr451 authored Jun 17, 2025


Co-authored-by: austindeng <austindeng@tencent.com>
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
Co-authored-by: ch-wan <cwan39@gatech.edu>

10d60cd4

16 Jun, 2025 3 commits
- Use seq_len_fill_value in the cuda graph runners (#7233) · c64290dc
  Lianmin Zheng authored Jun 16, 2025
  
  c64290dc
- [Eagle] Fix kernel call after updating speculative sampling kernels (#7231) · 53a525bf
  Lianmin Zheng authored Jun 16, 2025
  
  53a525bf
- [EAGLE] Refactor code for page size > 1 & more simplifications (#7213) · b1286a11
  Lianmin Zheng authored Jun 16, 2025
  
  b1286a11
15 Jun, 2025 2 commits
- Revert "[EAGLE] Refactor code for page size > 1 & more simplifications" (#7210) · fff10809
  Lianmin Zheng authored Jun 15, 2025
  
  fff10809
- [EAGLE] Refactor code for page size > 1 & more simplifications (#7163) · 5f1ab327
  Lianmin Zheng authored Jun 14, 2025
  
  5f1ab327
10 Jun, 2025 1 commit
- Open AI API hidden states (#6716) · b56de8f9
  kyle-pena-kuzco authored Jun 10, 2025
  
  b56de8f9
09 Jun, 2025 1 commit
- Simplify prepare_extend_after_decode (#6987) · dc0705a5
  Lianmin Zheng authored Jun 09, 2025
  
  dc0705a5
08 Jun, 2025 1 commit
- Sync cuda graph runners (#6976) · 0c1f03a2
  Lianmin Zheng authored Jun 08, 2025
  
  0c1f03a2
02 Jun, 2025 1 commit
- Add draft extend CUDA graph for flashinfer backend (#6805) · a2cb5913
  Ke Bao authored Jun 02, 2025
  
  a2cb5913
29 May, 2025 1 commit
- Add draft extend CUDA graph for Triton backend (#6705) · 7e412900
  Ke Bao authored May 29, 2025
  
  7e412900
27 May, 2025 1 commit
- Support EAGLE draft extend CUDA graph (#6606) · 63195028
  Ke Bao authored May 27, 2025
```
Co-authored-by: Sehoon Kim <sehoonkim@berkeley.edu>
```
  63195028
22 May, 2025 1 commit
- Apply constraint grammar to EAGLE (#6499) · 6ce0ed07
  Ke Bao authored May 22, 2025
```
Co-authored-by: merrymercy <lianminzheng@gmail.com>
```
  6ce0ed07
17 May, 2025 1 commit
- Use monotonic clock for interval measurement (#6211) · 3cf1473a
  Lifu Huang authored May 17, 2025
```
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
```
  3cf1473a
15 May, 2025 1 commit

[Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109) · 2e4babdb

quinnrong94 authored May 15, 2025


Co-authored-by: Yingyi <yingyihuang2000@outlook.com>
Co-authored-by: neiltian <neiltian@tencent.com>
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
Co-authored-by: kexueyu <kexueyu@tencent.com>
Co-authored-by: vincentmeng <vincentmeng@tencent.com>
Co-authored-by: pengmeng <pengmeng@tencent.com>

2e4babdb

12 May, 2025 3 commits
- Revert "fix some typos" (#6244) · e8e18dcd
  Lianmin Zheng authored May 12, 2025
  
  e8e18dcd
- fix some typos (#6209) · d738ab52
  applesaucethebun authored May 12, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  d738ab52
- Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201) · fba8eccd
  Lianmin Zheng authored May 12, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  fba8eccd
11 May, 2025 1 commit
- Add typo checker in pre-commit (#6179) · 2ce87935
  applesaucethebun authored May 11, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  2ce87935
01 May, 2025 1 commit
- [PP] Add pipeline parallelism (#5724) · 11383cec
  Ying Sheng authored Apr 30, 2025
  
  11383cec
21 Apr, 2025 1 commit
- [misc] remove is_cuda_available (#5319) · 97cb762b
  JieXin Liang authored Apr 21, 2025
  
  97cb762b
18 Apr, 2025 1 commit
- Fix: Incorrect parameters passed to forward_batch_generation (#5506) (#5511) · 211c7b31
  u4lr451 authored Apr 18, 2025
  
  211c7b31
09 Apr, 2025 1 commit
- Optimize topk operation in llama4 (#5128) · 86a876d8
  fzyzcjy authored Apr 09, 2025
  
  86a876d8
05 Apr, 2025 1 commit
- [Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052) · efbae697
  Baizhou Zhang authored Apr 05, 2025
  
  efbae697
02 Apr, 2025 1 commit

Add Eagle Speculative Decoding to FA3 Backend (#4951) · e983e432

Qingquan Song authored Apr 02, 2025


Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: zcnrex <zcnrex@gmail.com>

e983e432

30 Mar, 2025 1 commit
- Support page size > 1 + eagle (#4908) · b26bc86b
  Lianmin Zheng authored Mar 30, 2025
  
  b26bc86b
28 Mar, 2025 1 commit
- [Misc] Fix issues reported by torchfix (#4837) · e84f4ba0
  Brayden Zhong authored Mar 27, 2025
  
  e84f4ba0
18 Mar, 2025 1 commit
- [Feature] Support EAGLE 3 (#4247) · 9e0186f3
  James Liu authored Mar 18, 2025
  
  9e0186f3
16 Mar, 2025 1 commit
- [Eagle] Remove the greedy branch and some redundant code (#4363) · 1b859295
  Ying Sheng authored Mar 16, 2025
```
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  1b859295
09 Mar, 2025 1 commit
- Support nextn for flashinfer mla attention backend (#4218) · 9fb48f95
  Baizhou Zhang authored Mar 09, 2025
  
  9fb48f95
08 Mar, 2025 1 commit

[EAGLE] many fixes for eagle (#4195) · d4017a6b

Lianmin Zheng authored Mar 07, 2025


Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>

d4017a6b

06 Mar, 2025 1 commit

Fix a draft model accuracy bug in eagle; support step=1; return logprob in eagle (#4134) · bc1534ff

Lianmin Zheng authored Mar 06, 2025


Co-authored-by: Sehoon Kim <kssteven418@gmail.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>

bc1534ff

05 Mar, 2025 1 commit
- [Eagle] Refactor eagle speculative decoding (#3986) · d3d4d767
  Ying Sheng authored Mar 05, 2025
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  d3d4d767
04 Mar, 2025 1 commit
- Simplify eagle tests and TP sync in grammar backend (#4066) · 77a3954b
  Lianmin Zheng authored Mar 04, 2025
  
  77a3954b