Commits · d58e35447203c293933ad300c066c8f581cb0935 · zhaoyu6 / sglang · GitLab

19 Apr, 2025 1 commit
- simplify the control logic for using shared experts fusion (#5504) · d58e3544
  Xiaoyu Zhang authored Apr 20, 2025
  
  d58e3544
18 Apr, 2025 1 commit
- Introduce moe_dense_tp_size to fix dense layer errors in DeepSeek V3 + 4x8xH100 (#4836) · 53dcf388
  fzyzcjy authored Apr 18, 2025
  
  53dcf388
17 Apr, 2025 2 commits
- Refactor DeepSeek decoder layer branches (#5205) · 8beb356f
  fzyzcjy authored Apr 17, 2025
  
  8beb356f
- Deprecate disable-mla (#5481) · 4fb05583
  Baizhou Zhang authored Apr 17, 2025
  
  4fb05583
16 Apr, 2025 2 commits
- Clean up imports (#5467) · 177320a5
  Lianmin Zheng authored Apr 16, 2025
  
  177320a5
- Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113) · a42736bb
  Baizhou Zhang authored Apr 15, 2025
  
  a42736bb
15 Apr, 2025 1 commit
- [fix/misc] remove duplicate row in deepseek v2 model (#5279) · ee9d6ca6
  Yangcheng Li authored Apr 15, 2025
  
  ee9d6ca6
14 Apr, 2025 1 commit
- [perf] experimental enhance fp8 per-tensor quant (#5370) · bdde2375
  JieXin Liang authored Apr 15, 2025
  
  bdde2375
13 Apr, 2025 1 commit
- [DeepEP] Reduce routed scaling overhead (#5277) · adca585b
  yulei authored Apr 14, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  adca585b
12 Apr, 2025 1 commit
- Fix w8a8_int8 model shared experts fusion load weights error (#5120) · 1b1b47a9
  lambert0312 authored Apr 12, 2025
  
  1b1b47a9
10 Apr, 2025 1 commit
- Fix DeepSeek error when using DeepEP mode (#5190) · e3c4bd31
  fzyzcjy authored Apr 10, 2025
  
  e3c4bd31
09 Apr, 2025 3 commits
- Support Llama4 fp8 inference (#5194) · 40652482
  HandH1998 authored Apr 09, 2025
```
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: zhyncs <me@zhyncs.com>
```
  40652482
- fix: use DeepEPDispatcher on CUDA (#5180) · 90caf06c
  Yineng Zhang authored Apr 08, 2025
  
  90caf06c
- [Fix] DeepEP Compatibility with Low Latency (#5068) · bc3f6db2
  Jinyan Chen authored Apr 09, 2025
```
Co-authored-by: ch-wan <cwan39@gatech.edu>
```
  bc3f6db2
08 Apr, 2025 1 commit
- Fix loading KV quantization scale; Enable modelopt kv cache (#4686) · 2695ab05
  Yun Dai authored Apr 08, 2025
```
Co-authored-by: qingquansong <ustcsqq@gmail.com>
```
  2695ab05
05 Apr, 2025 1 commit
- [Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052) · efbae697
  Baizhou Zhang authored Apr 05, 2025
  
  efbae697
04 Apr, 2025 3 commits
- fix dummy-load deepseekv2 (#4535) · 7ed77d6b
  inkcherry authored Apr 05, 2025
  
  7ed77d6b
- Add DeepSeek V3/R1 shared experts fusion (#4918) · 924ca7c9
  Xiaoyu Zhang authored Apr 04, 2025
  
  924ca7c9
- Small refactor DeepEPDispatcher into subclasses (#4994) · febe21ce
  fzyzcjy authored Apr 04, 2025
  
  febe21ce
03 Apr, 2025 3 commits
- Revert "Replace enable_flashinfer_mla argument with attention_backend" (#5048) · 74885a84
  Lianmin Zheng authored Apr 03, 2025
  
  74885a84
- Small refactor DeepEPMode to clean up code a bit (#4992) · 8e10fec9
  fzyzcjy authored Apr 03, 2025
  
  8e10fec9
- Replace enable_flashinfer_mla argument with attention_backend (#5005) · e8999b13
  Baizhou Zhang authored Apr 03, 2025
  
  e8999b13
01 Apr, 2025 1 commit

[Feature] Support DeepEP Low Latency (#4767) · 23c764b1

Jinyan Chen authored Apr 02, 2025


Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: ch-wan <cwan39@gatech.edu>

23c764b1

31 Mar, 2025 2 commits
- refactor: multimodal data (#4754) · 5cb552b1
  Mick authored Apr 01, 2025
  
  5cb552b1
- Fix DeepSeek bug causing 2.2% MMLU drop when TP!=DP (#4883) · a303325f
  fzyzcjy authored Mar 31, 2025
```
Co-authored-by: ch-wan <cwan39@gatech.edu>
```
  a303325f
30 Mar, 2025 1 commit
- Fix 2-gpu CI test and suppress some warnings (#4930) · 9adf178c
  Lianmin Zheng authored Mar 30, 2025
  
  9adf178c
29 Mar, 2025 1 commit
- [Feature] Support FA3 backend for MLA (#4831) · 20c90be2
  Baizhou Zhang authored Mar 28, 2025
  
  20c90be2
28 Mar, 2025 2 commits
- Clean up `import vllm` in quantization/__init__.py (#4834) · 74e0ac1d
  Lianmin Zheng authored Mar 28, 2025
  
  74e0ac1d
- Support (1 <= dp < tp) in the dp attention in DeepEP (#4770) · 7f19e083
  tarinkk authored Mar 27, 2025
```
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
```
  7f19e083
26 Mar, 2025 1 commit
- Support compressed tensors fp8w8a8 (#4743) · 04e3ff69
  Xiaoyu Zhang authored Mar 27, 2025
  
  04e3ff69
25 Mar, 2025 1 commit
- Add endpoints to dump selected expert ids (#4435) · 199bb01d
  yuhsaun-t authored Mar 24, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  199bb01d
23 Mar, 2025 2 commits
- Support async in DeepEP (#4610) · ca75741e
  fzyzcjy authored Mar 23, 2025
```
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
```
  ca75741e
- Multiple tiny code cleanups (#4608) · c6d549e7
  fzyzcjy authored Mar 23, 2025
  
  c6d549e7
22 Mar, 2025 1 commit
- Optimize Permute Kernel in DeepEP (#4643) · c2bd094d
  xutizhou authored Mar 23, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  c2bd094d
19 Mar, 2025 2 commits
- avoid cudaStreamSynchronize in DeepSeekV2AttentionMLA (#4577) · df7014a8
  strgrb authored Mar 20, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  df7014a8
- [Feature] Integrate DeepEP into SGLang (#4232) · f44db16c
  Jinyan Chen authored Mar 19, 2025
```
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
```
  f44db16c
18 Mar, 2025 2 commits
- Reduce computation and communication in DP attention (#4521) · 3196999f
  Cheng Wan authored Mar 18, 2025
  
  3196999f
- cleanup deps 3/n (#4541) · c16b33cc
  Yineng Zhang authored Mar 18, 2025
  
  c16b33cc
17 Mar, 2025 1 commit

[Feature] Support Deepseek-VL2 (#2798) · d6d21640

萝卜菜 authored Mar 17, 2025


Co-authored-by: Edenzzzz <wtan45@wisc.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>

d6d21640

16 Mar, 2025 1 commit
- cleanup deps 2/n (#4464) · 65b7c9b7
  Yineng Zhang authored Mar 15, 2025
  
  65b7c9b7