Commits · 2e4babdb0a19fd3bd5a863cba083d36c17506b40 · zhaoyu6 / sglang · GitLab

15 May, 2025 1 commit

[Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109) · 2e4babdb

quinnrong94 authored May 15, 2025


Co-authored-by: Yingyi <yingyihuang2000@outlook.com>
Co-authored-by: neiltian <neiltian@tencent.com>
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
Co-authored-by: kexueyu <kexueyu@tencent.com>
Co-authored-by: vincentmeng <vincentmeng@tencent.com>
Co-authored-by: pengmeng <pengmeng@tencent.com>

2e4babdb

12 May, 2025 3 commits
- Revert "fix some typos" (#6244) · e8e18dcd
  Lianmin Zheng authored May 12, 2025
  
  e8e18dcd
- fix some typos (#6209) · d738ab52
  applesaucethebun authored May 12, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  d738ab52
- Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201) · fba8eccd
  Lianmin Zheng authored May 12, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  fba8eccd
11 May, 2025 1 commit
- Add typo checker in pre-commit (#6179) · 2ce87935
  applesaucethebun authored May 11, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  2ce87935
01 May, 2025 1 commit
- [PP] Add pipeline parallelism (#5724) · 11383cec
  Ying Sheng authored Apr 30, 2025
  
  11383cec
21 Apr, 2025 1 commit
- [misc] remove is_cuda_available (#5319) · 97cb762b
  JieXin Liang authored Apr 21, 2025
  
  97cb762b
18 Apr, 2025 1 commit
- Fix: Incorrect parameters passed to forward_batch_generation (#5506) (#5511) · 211c7b31
  u4lr451 authored Apr 18, 2025
  
  211c7b31
09 Apr, 2025 1 commit
- Optimize topk operation in llama4 (#5128) · 86a876d8
  fzyzcjy authored Apr 09, 2025
  
  86a876d8
05 Apr, 2025 1 commit
- [Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052) · efbae697
  Baizhou Zhang authored Apr 05, 2025
  
  efbae697
02 Apr, 2025 1 commit

Add Eagle Speculative Decoding to FA3 Backend (#4951) · e983e432

Qingquan Song authored Apr 02, 2025


Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: zcnrex <zcnrex@gmail.com>

e983e432

30 Mar, 2025 1 commit
- Support page size > 1 + eagle (#4908) · b26bc86b
  Lianmin Zheng authored Mar 30, 2025
  
  b26bc86b
28 Mar, 2025 1 commit
- [Misc] Fix issues reported by torchfix (#4837) · e84f4ba0
  Brayden Zhong authored Mar 27, 2025
  
  e84f4ba0
18 Mar, 2025 1 commit
- [Feature] Support EAGLE 3 (#4247) · 9e0186f3
  James Liu authored Mar 18, 2025
  
  9e0186f3
16 Mar, 2025 1 commit
- [Eagle] Remove the greedy branch and some redundant code (#4363) · 1b859295
  Ying Sheng authored Mar 16, 2025
```
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  1b859295
09 Mar, 2025 1 commit
- Support nextn for flashinfer mla attention backend (#4218) · 9fb48f95
  Baizhou Zhang authored Mar 09, 2025
  
  9fb48f95
08 Mar, 2025 1 commit

[EAGLE] many fixes for eagle (#4195) · d4017a6b

Lianmin Zheng authored Mar 07, 2025


Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>

d4017a6b

06 Mar, 2025 1 commit

Fix a draft model accuracy bug in eagle; support step=1; return logprob in eagle (#4134) · bc1534ff

Lianmin Zheng authored Mar 06, 2025


Co-authored-by: Sehoon Kim <kssteven418@gmail.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Sehoon Kim <sehoon@x.ai>

bc1534ff

05 Mar, 2025 1 commit
- [Eagle] Refactor eagle speculative decoding (#3986) · d3d4d767
  Ying Sheng authored Mar 05, 2025
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  d3d4d767
04 Mar, 2025 2 commits
- Simplify eagle tests and TP sync in grammar backend (#4066) · 77a3954b
  Lianmin Zheng authored Mar 04, 2025
  
  77a3954b
- [Feature] Add test for speculative_token_map (#4016) · 0d4e3228
  William authored Mar 04, 2025
  
  0d4e3228
03 Mar, 2025 2 commits
- Share target model embed and head weights for nextn (#4033) · 9fafa62d
  Ke Bao authored Mar 04, 2025
  
  9fafa62d
- [feat] add small vocab table for eagle's draft model[1]. (#3822) · 7fbab730
  Zhousx authored Mar 03, 2025
```
Co-authored-by: Achazwl <323163497@qq.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  7fbab730
14 Feb, 2025 1 commit
- Support NextN (MTP) speculative decoding for DeepSeek-V3/R1 (#3582) · 862dd76c
  Ke Bao authored Feb 15, 2025
  
  862dd76c
10 Feb, 2025 2 commits
- [Eagle] reduce one draft forward (#3468) · d23cb9a0
  Ying Sheng authored Feb 10, 2025
  
  d23cb9a0
- Support Eagle2 for Triton backend (#3466) · 2d611323
  Ke Bao authored Feb 10, 2025
  
  2d611323
08 Feb, 2025 1 commit
- fix EAGLE 2 non greedy case (#3407) · fad315cb
  Yineng Zhang authored Feb 09, 2025
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  fad315cb
04 Feb, 2025 1 commit
- upgrade flashinfer v0.2.0.post2 (#3288) · d39899e8
  Yineng Zhang authored Feb 04, 2025
```
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>
```
  d39899e8
03 Feb, 2025 1 commit

refactor EAGLE 2 (#3269) · 013021b6

Yineng Zhang authored Feb 03, 2025


Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>

013021b6

21 Jan, 2025 1 commit
- Misc fixes for eagle (flush_cache, CPU overhead) (#3014) · 287d07a6
  Lianmin Zheng authored Jan 20, 2025
  
  287d07a6
13 Jan, 2025 1 commit
- [Fix]eagle2 health_generate is first request,apiserver will core (#2853) · 4093aa46
  justdoit authored Jan 13, 2025
  
  4093aa46
10 Jan, 2025 1 commit
- [Eagle2] Fix multiple concurrent request crashes (#2730) · a47bf391
  justdoit authored Jan 11, 2025
  
  a47bf391
06 Jan, 2025 1 commit
- Clean up eagle code (#2756) · b8574f69
  Lianmin Zheng authored Jan 06, 2025
  
  b8574f69
02 Jan, 2025 1 commit

Eagle speculative decoding part 4: Add EAGLE2 worker (#2150) · 815dce05

yukavio authored Jan 02, 2025


Co-authored-by: kavioyu <kavioyu@tencent.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>

815dce05