Commits · 50eda8398e87ab67d14f1952ac9c66274b7dfd89 · zhaoyu6 / sglang

25 Apr, 2025 1 commit
- [PD] Better logs (#5715) · c55550cb
  Liangsheng Yin authored Apr 25, 2025
  
  c55550cb
21 Apr, 2025 4 commits
- [PD] Support prefill overlap + Ensure no race condition (#5609) · bf98d2e3
  Byron Hsu authored Apr 21, 2025
  
  bf98d2e3
- [PD] Support decode overlap schedule (#5608) · e65b9f21
  Byron Hsu authored Apr 21, 2025
  
  e65b9f21
- upstream hicache fixes (#5570) · 70645f4d
  Zhiqiang Xie authored Apr 20, 2025
  
  70645f4d
- Tiny add Engine.flush_cache API (#5241) · 11951820
  fzyzcjy authored Apr 21, 2025
  
  11951820
19 Apr, 2025 1 commit
- Make profiler output file names consistent (#5548) · f6a71139
  fzyzcjy authored Apr 19, 2025
  
  f6a71139
16 Apr, 2025 2 commits
- Fix several minor issues in PD disaggregation (#5444) · 6aca5834
  Cheng Wan authored Apr 15, 2025
  
  6aca5834
- [Bug fix] need record start time in pd mode (#5425) · dd83e7e9
  ybyang authored Apr 16, 2025
  
  dd83e7e9
15 Apr, 2025 1 commit
- [PD] Fix dynamic port support and MLA buffer for Mooncake (#5415) · ffde65a0
  shangmingc authored Apr 15, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: ybyang <ybyang7@iflytek.com>
```
  ffde65a0
13 Apr, 2025 2 commits
- [PD] Add transfer backend abstraction (#5328) · a9499885
  Byron Hsu authored Apr 13, 2025
  
  a9499885
- Fix typo: infight -> inflight (#5357) · f7655790
  Liangsheng Yin authored Apr 14, 2025
  
  f7655790
12 Apr, 2025 1 commit
- [VLM] Adopt fast image processor by default (#5065) · 34ef6c81
  Mick authored Apr 12, 2025
  
  34ef6c81
11 Apr, 2025 1 commit
- Support `--enable-llama4-multimodal` (#5254) · 038bc5d5
  Cheng Wan authored Apr 11, 2025
  
  038bc5d5
10 Apr, 2025 3 commits
- Update deps for mllama4 (#5215) · 1078396f
  Ke Bao authored Apr 11, 2025
  
  1078396f
- [PD] Support KV transfer with mooncake (#4880) · 4c31ae9f
  Teng Ma authored Apr 10, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Xuchun Shang <xuchun.shang@linux.alibaba.com>
Co-authored-by: shangmingc <csmthu@gmail.com>
```
  4c31ae9f
- [metrics] Add in queue metrics (#4444) · 5db37c86
  Stefan He authored Apr 09, 2025
  
  5db37c86
09 Apr, 2025 1 commit
- Let `bench_one_batch` support `enable_dp_attention` (#4058) · 61970b08
  fzyzcjy authored Apr 09, 2025
  
  61970b08
08 Apr, 2025 1 commit
- feat: disable grammar restrictions within reasoning sections (#4984) · 7c5658c1
  mlmz authored Apr 08, 2025
```
Co-authored-by: tianhaoyu <thy@mail.ecust.edu.cn>
Co-authored-by: DarkSharpness <2040703891@qq.com>
```
  7c5658c1
02 Apr, 2025 1 commit
- Large page size aligned hierarchical caching (#4581) · e119f042
  Zhiqiang Xie authored Apr 01, 2025
  
  e119f042
31 Mar, 2025 1 commit
- refactor: multimodal data (#4754) · 5cb552b1
  Mick authored Apr 01, 2025
  
  5cb552b1
30 Mar, 2025 1 commit
- Support page size > 1 + eagle (#4908) · b26bc86b
  Lianmin Zheng authored Mar 30, 2025
  
  b26bc86b
28 Mar, 2025 5 commits
- Fix wrong variable name when stopping memory profile (#4772) · c483377e
  Fr4nk1in authored Mar 29, 2025
  
  c483377e
- Support with_stack and record_shapes in profiler (#4740) · 8c04f0f2
  fzyzcjy authored Mar 28, 2025
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  8c04f0f2
- Support controlling nsys start and end range programmatically (#4688) · 53a2c3b4
  fzyzcjy authored Mar 28, 2025
  
  53a2c3b4
- [Fix] `self.worker` assignment in `TpModelWorker` and refactor references (#4788) · 42a45df0
  XinyuanTong authored Mar 27, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  42a45df0
- Support (1 <= dp < tp) in the dp attention in DeepEP (#4770) · 7f19e083
  tarinkk authored Mar 27, 2025
```
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
```
  7f19e083
26 Mar, 2025 2 commits
- Support compressed tensors fp8w8a8 (#4743) · 04e3ff69
  Xiaoyu Zhang authored Mar 27, 2025
  
  04e3ff69
- Warn users when release_memory_occupation is called without memory saver enabled (#4566) · 26f07294
  fzyzcjy authored Mar 26, 2025
  
  26f07294
25 Mar, 2025 3 commits
- Fix test_expert_distribution failure (#4752) · eb934bdf
  fzyzcjy authored Mar 25, 2025
  
  eb934bdf
- Add endpoints to dump selected expert ids (#4435) · 199bb01d
  yuhsaun-t authored Mar 24, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  199bb01d
- model: Minicpmo (#3023) · 1e86457c
  Mick authored Mar 25, 2025
  
  1e86457c
21 Mar, 2025 1 commit

[PD] Release initial code (#4654) · c7c7dbeb

Byron Hsu authored Mar 21, 2025


Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: makro
Co-authored-by: dhou-xai

c7c7dbeb

18 Mar, 2025 1 commit
- Unit test for Hierarchical Caching (#4486) · a98290ae
  Zhiqiang Xie authored Mar 17, 2025
  
  a98290ae
17 Mar, 2025 2 commits
- Fix data parallel + tensor parallel (#4499) · 5493c334
  Lianmin Zheng authored Mar 17, 2025
  
  5493c334
- [Fix] use `torch.inference_mode()` instead of `torch.no_grad()` (#4372) · 0212d2e2
  JieXin Liang authored Mar 17, 2025
  
  0212d2e2
16 Mar, 2025 1 commit
- [Eagle] Remove the greedy branch and some redundant code (#4363) · 1b859295
  Ying Sheng authored Mar 16, 2025
```
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  1b859295
14 Mar, 2025 1 commit
- feat(remote_model): support variable remote backend for model loader (#3964) · 1ce4878d
  wangyu authored Mar 14, 2025
```
Signed-off-by: wangyu <wangyu.steph@bytedance.com>
```
  1ce4878d
13 Mar, 2025 3 commits
- Hot fix for hicache with new page aligned radixtree (#4397) · fbdb5050
  Zhiqiang Xie authored Mar 13, 2025
  
  fbdb5050
- Improve DP attention (#4390) · 8e66fbec
  Lianmin Zheng authored Mar 13, 2025
```
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  8e66fbec
- Support page size > 1 (#4356) · c76040e3
  Lianmin Zheng authored Mar 12, 2025
  
  c76040e3