Commits · a42736bbb8feeb047c240c062a0ba9de03c2ab40 · change / sglang

16 Apr, 2025 1 commit
- Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113) · a42736bb
  Baizhou Zhang authored Apr 15, 2025
  
  a42736bb
12 Apr, 2025 1 commit
- [VLM] Adopt fast image processor by default (#5065) · 34ef6c81
  Mick authored Apr 12, 2025
  
  34ef6c81
05 Apr, 2025 1 commit
- [Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052) · efbae697
  Baizhou Zhang authored Apr 05, 2025
  
  efbae697
03 Apr, 2025 2 commits
- Revert "Replace enable_flashinfer_mla argument with attention_backend" (#5048) · 74885a84
  Lianmin Zheng authored Apr 03, 2025
  
  74885a84
- Replace enable_flashinfer_mla argument with attention_backend (#5005) · e8999b13
  Baizhou Zhang authored Apr 03, 2025
  
  e8999b13
01 Apr, 2025 1 commit

[Feature] Support DeepEP Low Latency (#4767) · 23c764b1

Jinyan Chen authored Apr 02, 2025


Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: ch-wan <cwan39@gatech.edu>

23c764b1

28 Mar, 2025 1 commit
- Support (1 <= dp < tp) in the dp attention in DeepEP (#4770) · 7f19e083
  tarinkk authored Mar 27, 2025
```
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
```
  7f19e083
27 Mar, 2025 1 commit
- [k8s] Clarified the usage of shared memory. (#4341) · f60f2931
  Jiří Suchomel authored Mar 27, 2025
  
  f60f2931
19 Mar, 2025 1 commit

[Feature] Integrate DeepEP into SGLang (#4232) · f44db16c

Jinyan Chen authored Mar 19, 2025


Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>

f44db16c

15 Mar, 2025 1 commit
- docs: hot fix torch compile cache (#4442) · e1a5e7e4
  Chayenne authored Mar 14, 2025
  
  e1a5e7e4
13 Mar, 2025 2 commits
- [docs] Update outdated description about `torch.compile` (#3844) · 14344caa
  Jun Liu authored Mar 13, 2025
  
  14344caa
- docs: add parameter --log-requests-level (#4335) · 01603318
  Peter Pan authored Mar 13, 2025
  
  01603318
06 Mar, 2025 2 commits
- Docs: add torch compile cache (#4151) · ebddb65a
  Chayenne authored Mar 06, 2025
```
Co-authored-by: ybyang <ybyang7@iflytek.com>
```
  ebddb65a
- fix cross-reference error and spelling mistakes (#4101) · d2d0d061
  samzong authored Mar 06, 2025
```
Signed-off-by: samzong <samzong.lu@gmail.com>
```
  d2d0d061
05 Mar, 2025 3 commits
- Add examples for server token-in-token-out (#4103) · 357671e2
  Qiaolin Yu authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  357671e2
- [Revision] Add fast decode plan for flashinfer mla (#4012) · fc91d08a
  Baizhou Zhang authored Mar 05, 2025
  
  fc91d08a
- example: add vlm to token in & out example (#3941) · 583d6af7
  Mick authored Mar 05, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  583d6af7
03 Mar, 2025 7 commits
- Add examples in sampling parameters (#4039) · 146ac8df
  Chayenne authored Mar 03, 2025
  
  146ac8df
- Misc clean up; Remove the support of jump forward (#4032) · 935cda94
  Lianmin Zheng authored Mar 03, 2025
  
  935cda94
- Support penalty in overlap mode; return logprob with chunked prefill; improve... · ac238727
  Lianmin Zheng authored Mar 03, 2025
```
Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
```
  ac238727
- Add examples to token-in-token-out for LLM (#4010) · 728e175f
  Chayenne authored Mar 02, 2025
  
  728e175f
- Revert "Add fast decode plan for flashinfer mla" (#4008) · 9e1014cf
  Lianmin Zheng authored Mar 02, 2025
  
  9e1014cf
- Add fast decode plan for flashinfer mla (#3987) · fa561067
  Baizhou Zhang authored Mar 02, 2025
  
  fa561067
- [feat] add small vocab table for eagle's draft model[1]. (#3822) · 7fbab730
  Zhousx authored Mar 03, 2025
```
Co-authored-by: Achazwl <323163497@qq.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  7fbab730
01 Mar, 2025 1 commit

[Feature]Support ragged prefill in flashinfer mla backend (#3967) · 90a4b7d9

Baizhou Zhang authored Feb 28, 2025


Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>

90a4b7d9

27 Feb, 2025 2 commits
- [Doc] Add experimental tag for flashinfer mla (#3925) · 3e02526b
  Baizhou Zhang authored Feb 27, 2025
  
  3e02526b
- [doc] Update document for flashinfer mla (#3907) · 71ed0183
  Baizhou Zhang authored Feb 26, 2025
  
  71ed0183
26 Feb, 2025 1 commit
- Docs: improve link to docs (#3860) · 44a2c4bd
  simveit authored Feb 26, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  44a2c4bd
25 Feb, 2025 1 commit
- [Doc] Fix typo in server-argument description (#3641) · 3758d209
  Yuanheng Zhao authored Feb 25, 2025
  
  3758d209
20 Feb, 2025 1 commit
- [Docs] Modify ep related server args and remove cublas part of deepseek (#3732) · ac053100
  Baizhou Zhang authored Feb 20, 2025
  
  ac053100
15 Feb, 2025 1 commit
- doc: emphasize and notify the usage of chat_template (#3589) · 7711ac6e
  Mick authored Feb 15, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  7711ac6e
11 Feb, 2025 1 commit
- fix server_arguments typo (#3499) · 1e2cf2b5
  Didier Durand authored Feb 11, 2025
  
  1e2cf2b5
04 Feb, 2025 1 commit
- [Feature] Define backends and add Triton backend for Lora (#3161) · 70817a7e
  Baizhou Zhang authored Feb 03, 2025
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  70817a7e
03 Feb, 2025 1 commit
- Update server args doc (#3273) · 7b5a3741
  simveit authored Feb 04, 2025
```
Co-authored-by: Shi Shuai <126407087+shuaills@users.noreply.github.com>
```
  7b5a3741
23 Jan, 2025 1 commit

Docs: Update doc for server arguments (#2742) · 1c4e0d24

simveit authored Jan 23, 2025


Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>

1c4e0d24

16 Jan, 2025 1 commit
- Support multi-node DP attention (#2925) · 8b6ce52e
  Lianmin Zheng authored Jan 16, 2025
```
Co-authored-by: dhou-xai <dhou@x.ai>
```
  8b6ce52e
08 Jan, 2025 2 commits
- Docs：Update the style of llma 3.1 405B docs (#2789) · 2e6346fc
  Chayenne authored Jan 08, 2025
  
  2e6346fc
- Docs: Rewrite docs for LLama 405B and ModelSpace (#2773) · 977f785d
  mlmz authored Jan 08, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  977f785d
01 Jan, 2025 1 commit
- [Docs] Add Support for Pydantic Structured Output Format (#2697) · 062c48d2
  Shi Shuai authored Jan 01, 2025
  
  062c48d2
30 Dec, 2024 1 commit
- [Docs] clean up structured outputs docs (#2654) · 8c3b420e
  Lianmin Zheng authored Dec 29, 2024
  
  8c3b420e