Commits · 8fc910db031020f5f3de847f32cbbd92461fbd8c · zhaoyu6 / sglang

05 Jul, 2025 1 commit
- DP Attention with Auto DeepEP Dispatch (#7222) · 8fc910db
  Cheng Wan authored Jul 05, 2025
  
  8fc910db
03 Jul, 2025 1 commit
- Fix num_tokens_pre_allocated in disaggregation log (#7714) · 1bebd315
  Ziming Huang authored Jul 03, 2025
  
  1bebd315
28 Jun, 2025 2 commits

[PD] Respect sampling_params.max_new_tokens when PD disaggregation is activated (#7598) · eb429b88
Shangming Cai authored Jun 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
eb429b88

Hybrid kv cache for LLaMA4 (#6563) · eb6c2c16

tarinkk authored Jun 27, 2025


Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>

eb6c2c16

25 Jun, 2025 1 commit
- [PD][Spec] Fix hidden state transfer for spec decode (#7516) · 7b9a174a
  Shangming Cai authored Jun 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  7b9a174a
22 Jun, 2025 1 commit
- [minor] simplify the `TokenToKVPoolAllocator` (#7414) · 05c9bc89
  Liangsheng Yin authored Jun 22, 2025
  
  05c9bc89
21 Jun, 2025 1 commit
- [fix] PD disaggregation when enable mtp and tp!=dp (#7420) · 02bf31ef
  Atream authored Jun 22, 2025
  
  02bf31ef
20 Jun, 2025 1 commit
- [Feature] Comprehensive Hybrid Parallelism Support (#6389) · e879d8b7
  Cheng Wan authored Jun 20, 2025
  
  e879d8b7
19 Jun, 2025 2 commits
- [PD] Transfer hidden states for mtp when disaggregation (#7242) · 4f838c09
  Atream authored Jun 20, 2025
  
  4f838c09
- Multi-Stage Awake: Support Resume and Pause KV Cache and Weights separately (#7099) · 3774f078
  Stefan He authored Jun 19, 2025
  
  3774f078
15 Jun, 2025 3 commits
- Minor PD style fix (#7215) · 96be97bf
  Byron Hsu authored Jun 15, 2025
  
  96be97bf
- [PD] use int32 for kv indices & get num_reserved_decode_tokens from server_args (#7214) · 88f9c347
  Byron Hsu authored Jun 15, 2025
  
  88f9c347
- [PD] Support decode retract and update decode.py (#7196) · db0cc57e
  Byron Hsu authored Jun 14, 2025
  
  db0cc57e
14 Jun, 2025 1 commit
- [PD] Update prefill.py (#7190) · 7d316991
  Byron Hsu authored Jun 14, 2025
  
  7d316991
12 Jun, 2025 1 commit
- [Fix] Reduce busy polling when scheduler is idle (#6026) · bd7cfbd2
  Povilas Kanapickas authored Jun 13, 2025
  
  bd7cfbd2
10 Jun, 2025 1 commit
- Add decode req pool (#6980) · c2b16795
  Byron Hsu authored Jun 09, 2025
  
  c2b16795
09 Jun, 2025 1 commit
- feat: add direct routing strategy to DP worker (#6884) · f1569876
  ishandhanani authored Jun 09, 2025
  
  f1569876
27 May, 2025 1 commit
- [PD] Handle P/D failure and reconnect without affecting other instances (#6263) · 3ce94f71
  shangmingc authored May 27, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  3ce94f71
24 May, 2025 1 commit
- [PD] Support structured output (#6560) · 2d831c6e
  Byron Hsu authored May 23, 2025
  
  2d831c6e
23 May, 2025 3 commits
- [PD] Support logprob & Add failure test (#6558) · 8233cc10
  Byron Hsu authored May 23, 2025
  
  8233cc10
- [PD] support spec decode (#6507) · d2e0881a
  Byron Hsu authored May 23, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  d2e0881a
- [PD] Fix failure abort (#6535) · 0a4fc73b
  Byron Hsu authored May 22, 2025
  
  0a4fc73b
22 May, 2025 2 commits
- Fix missing http status import for PD failure handler (#6520) · 58f10679
  shangmingc authored May 22, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  58f10679
- [PD] Abort request if transfer fails (#6504) · 3bde1010
  Byron Hsu authored May 21, 2025
  
  3bde1010
12 May, 2025 1 commit
- [PD] Add support for different TP sizes per DP rank (#5922) · f1c89600
  shangmingc authored May 13, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  f1c89600
11 May, 2025 1 commit
- Add typo checker in pre-commit (#6179) · 2ce87935
  applesaucethebun authored May 11, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  2ce87935
06 May, 2025 1 commit
- Fix not "import os" (#6057) · 6d4d3bc8
  Liangsheng Yin authored May 06, 2025
  
  6d4d3bc8
05 May, 2025 1 commit
- [PD] Allow customizing reserved tokens to avoid KV cache waste (#6002) · 3008db9c
  fzyzcjy authored May 05, 2025
  
  3008db9c
28 Apr, 2025 1 commit
- [PD] support pd fake transfer for warmup (#5726) · c6c62640
  ybyang authored Apr 29, 2025
  
  c6c62640
25 Apr, 2025 2 commits
- [PD]: Support Muti Prefill in one node (#5704) · 11e27d09
  IAN authored Apr 26, 2025
```
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  11e27d09
- [PD] Better logs (#5715) · c55550cb
  Liangsheng Yin authored Apr 25, 2025
  
  c55550cb
23 Apr, 2025 1 commit
- Integrating PD disaggregation with DP attention and DeepEP (#5435) · 711efe78
  Cheng Wan authored Apr 23, 2025
```
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  711efe78
21 Apr, 2025 1 commit
- [PD] Support decode overlap schedule (#5608) · e65b9f21
  Byron Hsu authored Apr 21, 2025
  
  e65b9f21
20 Apr, 2025 1 commit
- [PD] Support page size > 1 (#5561) · ab4b5606
  Byron Hsu authored Apr 19, 2025
  
  ab4b5606
19 Apr, 2025 1 commit
- [PD] Remove the requirement of config file for mooncake backend (#5460) · dca90f1d
  shangmingc authored Apr 19, 2025
  
  dca90f1d
16 Apr, 2025 1 commit
- Fix several minor issues in PD disaggregation (#5444) · 6aca5834
  Cheng Wan authored Apr 15, 2025
  
  6aca5834
15 Apr, 2025 2 commits
- Distinguish bootstrap key only in decode server (#5422) · 33b16ad1
  Liangsheng Yin authored Apr 15, 2025
  
  33b16ad1
- [PD] Fix dynamic port support and MLA buffer for Mooncake (#5415) · ffde65a0
  shangmingc authored Apr 15, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
```
  ffde65a0
13 Apr, 2025 1 commit
- [PD] Add transfer backend abstraction (#5328) · a9499885
  Byron Hsu authored Apr 13, 2025
  
  a9499885
10 Apr, 2025 1 commit

[PD] Support KV transfer with mooncake (#4880) · 4c31ae9f

Teng Ma authored Apr 10, 2025


Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Co-authored-by: shangmingc <csmthu@gmail.com>

4c31ae9f