Commits · 2e4babdb0a19fd3bd5a863cba083d36c17506b40 · change / sglang

15 May, 2025 1 commit

[Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109) · 2e4babdb

quinnrong94 authored May 15, 2025


Co-authored-by: Yingyi <yingyihuang2000@outlook.com>
Co-authored-by: neiltian <neiltian@tencent.com>
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
Co-authored-by: kexueyu <kexueyu@tencent.com>
Co-authored-by: vincentmeng <vincentmeng@tencent.com>
Co-authored-by: pengmeng <pengmeng@tencent.com>

2e4babdb

14 May, 2025 1 commit
- docs: fix a bad redirect (#6300) · 9a91fa0e
  Brayden Zhong authored May 14, 2025
  
  9a91fa0e
12 May, 2025 3 commits
- Revert "fix some typos" (#6244) · e8e18dcd
  Lianmin Zheng authored May 12, 2025
  
  e8e18dcd
- fix some typos (#6209) · d738ab52
  applesaucethebun authored May 12, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  d738ab52
- Performing Vocabulary Parallelism for LM Head across Attention TP Groups (#5558) · 25c83fff
  Cheng Wan authored May 12, 2025
```
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
```
  25c83fff
11 May, 2025 4 commits
- Improve structured outputs: fix race condition, server crash, metrics and style (#6188) · 01bdbf7f
  Lianmin Zheng authored May 11, 2025
  
  01bdbf7f
- [Docs] minor Qwen3 and reasoning parser docs fix (#6032) · 94d42b67
  Adarsh Shirawalmath authored May 11, 2025
  
  94d42b67
- doc: fix the erroneous documents and example codes about... · 69276f61
  mlmz authored May 11, 2025
```
doc: fix the erroneous documents and example codes about Alibaba-NLP/gme-Qwen2-VL-2B-Instruct (#6199)
```
  69276f61
- Add typo checker in pre-commit (#6179) · 2ce87935
  applesaucethebun authored May 11, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  2ce87935
10 May, 2025 3 commits
- Revert "feat: add thinking_budget (#6089)" (#6181) · 66fc63d6
  Yineng Zhang authored May 10, 2025
  
  66fc63d6
- [Docs]Delete duplicate content (#6146) · 921e4a81
  Ximingwang-09 authored May 11, 2025
```
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
```
  921e4a81
- Fix and Clean up chat-template requirement for VLM (#6114) · 9d8ec2e6
  XinyuanTong authored May 10, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  9d8ec2e6
09 May, 2025 1 commit
- feat: add thinking_budget (#6089) · 63484f9f
  thyecust authored May 09, 2025
  
  63484f9f
08 May, 2025 2 commits
- [Feature] Add FlashAttention3 as a backend for VisionAttention (#5764) · fa7d7fd9
  Zhu Chen authored May 09, 2025
```
Co-authored-by: othame <chenzhu_912@zju.edu.cn>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>
```
  fa7d7fd9
- Update doc for MLA attention backends (#6034) · 8f508cc7
  Baizhou Zhang authored May 07, 2025
  
  8f508cc7
07 May, 2025 1 commit
- [Doc]Fix description for dp_size argument (#6063) · fee37d9e
  Baizhou Zhang authored May 07, 2025
  
  fee37d9e
05 May, 2025 2 commits
- feat: append more comprehensive fields in messages instead of merely role and content (#5996) · a68ed766
  mlmz authored May 06, 2025
  
  a68ed766
- Fix "Avoid computing lse in Ragged Prefill when there's no prefix match" (#5555) · 22da3d97
  Wenxuan Tan authored May 05, 2025
  
  22da3d97
04 May, 2025 1 commit
- Tool Call: Add `chat_template_kwargs` documentation (#5679) · 95c231e5
  vzed authored May 04, 2025
  
  95c231e5
02 May, 2025 1 commit
- Remove token in token out in Native API (#5967) · 73dcf2b3
  Chayenne authored May 01, 2025
  
  73dcf2b3
30 Apr, 2025 1 commit
- feat: support pythonic tool call and index in tool call streaming (#5725) · 2b06484b
  Chang Su authored Apr 29, 2025
  
  2b06484b
29 Apr, 2025 2 commits
- [Doc] Tables instead of bulletpoints for sampling doc (#5841) · ae523675
  simveit authored Apr 29, 2025
  
  ae523675
- Feat: support cuda graph for LoRA (#4115) · 8c0cfca8
  Qiaolin Yu authored Apr 29, 2025
```
Co-authored-by: Beichen Ma <mabeichen12@gmail.com>
```
  8c0cfca8
28 Apr, 2025 4 commits
- [CI] test chunked prefill more (#5798) · 849c83a0
  Lianmin Zheng authored Apr 28, 2025
  
  849c83a0
- [Doc] Recover history of server_arguments.md (#5851) · f48b007c
  Baizhou Zhang authored Apr 28, 2025
  
  f48b007c
- [Docs] Replace lists with tables for cleanup and readability in server_arguments (#5276) · 966eb908
  Michael Yao authored Apr 28, 2025
  
  966eb908
- Add Cutlass MLA attention backend (#5390) · 84810da4
  Trevor Morris authored Apr 27, 2025
  
  84810da4
27 Apr, 2025 1 commit
- [Minor] fix documentations (#5756) · 155890e4
  Lianmin Zheng authored Apr 26, 2025
  
  155890e4
24 Apr, 2025 1 commit
- [Doc] Fix a link to Weilin Zhao (#5706) · b5be5694
  Michael Yao authored Apr 25, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  b5be5694
22 Apr, 2025 1 commit
- [Doc] Fix a 404 link to llama-405b (#5615) · 92bb64bc
  Michael Yao authored Apr 22, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  92bb64bc
21 Apr, 2025 2 commits
- smaller and non gated models for docs (#5378) · 8de53da9
  simveit authored Apr 21, 2025
  
  8de53da9
- [Feature] Prefill assistant response - add continue_final_message parameter (#4226) · 8b39274e
  Adarsh Shirawalmath authored Apr 21, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  8b39274e
20 Apr, 2025 1 commit
- Add document for LoRA serving (#5521) · 072b4d03
  Baizhou Zhang authored Apr 20, 2025
  
  072b4d03
18 Apr, 2025 3 commits
- Revert "Avoid computing lse in Ragged Prefill when there's no prefix.… (#5544) · a6f892e5
  Yineng Zhang authored Apr 18, 2025
  
  a6f892e5
- Avoid computing lse in Ragged Prefill when there's no prefix. (#5476) · bfa39224
  Wenxuan Tan authored Apr 18, 2025
```
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  bfa39224
- [docs] Fix several consistency issues in sampling_params.md (#5373) · a0fc5bc1
  Michael Yao authored Apr 18, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  a0fc5bc1
17 Apr, 2025 4 commits
- Doc: fix problems of the 'Execute Notebooks / run-all-notebooks' ci caused by... · f13d65a7
  mlmz authored Apr 18, 2025
```
Doc: fix problems of the 'Execute Notebooks / run-all-notebooks' ci caused by the unstability of deepseek-ai/DeepSeek-R1-Distill-Qwen-7B (#5503)
```
  f13d65a7
- Deprecate enable-flashinfer-mla and enable-flashmla (#5480) · 6fb29ffd
  Baizhou Zhang authored Apr 17, 2025
  
  6fb29ffd
- Deprecate disable-mla (#5481) · 4fb05583
  Baizhou Zhang authored Apr 17, 2025
  
  4fb05583
- Update attention_backend.md: plural form (#5489) · 92d1561b
  Didier Durand authored Apr 17, 2025
  
  92d1561b