Commits · 155214952b059396e07c5c65d51753e292d79fe7 · zhaoyu6 / sglang

18 May, 2025 2 commits
- refactor: Extract repeated member variables in KVCache subclasses to base class. (#6323) · 15521495
  wangxiyu191 authored May 19, 2025
  
  15521495
- chore: tiny remove duplicated code (#6392) · 9d24c3ff
  doujiang24 authored May 18, 2025
```
Signed-off-by: doujiang24 <doujiang24@gmail.com>
```
  9d24c3ff
09 May, 2025 1 commit

KV‑Cache (MHA, MLA): add missing start_layer / end_layer fields to... · b29a026e

Simon (Jiyou) Li authored May 10, 2025

KV‑Cache (MHA, MLA): add missing start_layer / end_layer fields to MHATokenToKVPoolHost and MLATokenToKVPoolHost (#6016)
Co-authored-by: 继优 <jiyou.ljy@alibaba-inc.com>
Co-authored-by: chus-chus <chus-chus@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

b29a026e

02 May, 2025 1 commit
- Fix set kv cache multi-stream (#5975) · 6579cd7d
  Ke Bao authored May 03, 2025
  
  6579cd7d
01 May, 2025 1 commit
- [PP] Add pipeline parallelism (#5724) · 11383cec
  Ying Sheng authored Apr 30, 2025
  
  11383cec
27 Apr, 2025 1 commit
- Fuse MLA set kv cache kernel (#5748) · 799c4bb5
  Ke Bao authored Apr 27, 2025
  
  799c4bb5
21 Apr, 2025 3 commits
- Enhance GPU memory settings (#5604) · e69a2190
  Liangsheng Yin authored Apr 22, 2025
  
  e69a2190
- upstream hicache fixes (#5570) · 70645f4d
  Zhiqiang Xie authored Apr 20, 2025
  
  70645f4d
- [PD] Fix large page size + chunk prefill (#5588) · c951d312
  Byron Hsu authored Apr 20, 2025
  
  c951d312
20 Apr, 2025 2 commits
- [PD] Fix DeepSeek cannot be run on latest master (#5568) · 80ac527d
  fzyzcjy authored Apr 20, 2025
  
  80ac527d
- [PD] Support page size > 1 (#5561) · ab4b5606
  Byron Hsu authored Apr 19, 2025
  
  ab4b5606
14 Apr, 2025 1 commit
- [PD Bug] fix MLA get_contiguous_buf_infos error (#5384) · e9fc2ac7
  ybyang authored Apr 14, 2025
  
  e9fc2ac7
13 Apr, 2025 1 commit
- fix MLATokenToKVPoolHost get_size_per_token bug (#5161) · 5fbafbb8
  huangtingwei authored Apr 14, 2025
```
Co-authored-by: AniZpZ <zhuangsen.zp@antgroup.com>
```
  5fbafbb8
10 Apr, 2025 1 commit
- [PD] Add get_contiguous_buf_infos interface for MLATokenToKVPool (#5204) · 7e4f72dd
  Teng Ma authored Apr 10, 2025
  
  7e4f72dd
02 Apr, 2025 1 commit
- Large page size aligned hierarchical caching (#4581) · e119f042
  Zhiqiang Xie authored Apr 01, 2025
  
  e119f042
30 Mar, 2025 1 commit
- Support page size > 1 + eagle (#4908) · b26bc86b
  Lianmin Zheng authored Mar 30, 2025
  
  b26bc86b
24 Mar, 2025 1 commit
- Unify variable naming: replace is_in_free_group with is_not_in_free_group (#4698) · 93cf7fc5
  c1lovez1 authored Mar 24, 2025
  
  93cf7fc5
23 Mar, 2025 1 commit
- Move mem_state update into debug mode (#4525) · 4d253057
  Zhiqiang Xie authored Mar 23, 2025
  
  4d253057
21 Mar, 2025 1 commit

[PD] Release initial code (#4654) · c7c7dbeb

Byron Hsu authored Mar 21, 2025


Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: makro
Co-authored-by: dhou-xai

c7c7dbeb

18 Mar, 2025 1 commit
- Unit test for Hierarchical Caching (#4486) · a98290ae
  Zhiqiang Xie authored Mar 17, 2025
  
  a98290ae
17 Mar, 2025 1 commit
- Fix: Complete int32 to int64 conversion (#4465) · f5bbf603
  Zhiqiang Xie authored Mar 16, 2025
  
  f5bbf603
16 Mar, 2025 3 commits
- [Fix] use `torch.cat` instead of `torch.concat` to prevent entering the... · 1a3fa75f
  JieXin Liang authored Mar 16, 2025
```
[Fix] use `torch.cat` instead of `torch.concat` to prevent entering the `Autograd` backends. (#4466)
```
  1a3fa75f
- Fix minor style (#4460) · 2c4f5cca
  Lianmin Zheng authored Mar 15, 2025
  
  2c4f5cca
- Fix typos (#4368) · 15843047
  Wang Ran (汪然) authored Mar 16, 2025
  
  15843047
15 Mar, 2025 1 commit
- [Fix] Fix errors when using the device except cuda. (#4455) · 86d9baed
  Chen Shengzhi authored Mar 16, 2025
  
  86d9baed
14 Mar, 2025 1 commit

Hierarchical Caching supports MLA (#4009) · 0e0ec702

Lu Changqi authored Mar 14, 2025


Signed-off-by: Changqi Lu <luchangqi.123@bytedance.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

0e0ec702

13 Mar, 2025 3 commits
- Fix auto merge & add back get_flat_data_by_layer (#4393) · a5a892ff
  Lianmin Zheng authored Mar 13, 2025
  
  a5a892ff
- Fix a regression introduced by overlapping KV cache writing (#4375) · 4fea040c
  Lianmin Zheng authored Mar 13, 2025
  
  4fea040c
- Support page size > 1 (#4356) · c76040e3
  Lianmin Zheng authored Mar 12, 2025
  
  c76040e3
12 Mar, 2025 1 commit
- Hierarchical Caching Refactoring and Fixing TP issue (#4082) · 10b544ae
  Zhiqiang Xie authored Mar 12, 2025
  
  10b544ae
07 Mar, 2025 1 commit
- Memory pool fix for upstream change about eagle (#4170) · 9376ac36
  Zhiqiang Xie authored Mar 07, 2025
  
  9376ac36
06 Mar, 2025 1 commit
- Add a pointer to the real KV cache pool (#4113) · aee30630
  Zhiqiang Xie authored Mar 05, 2025
  
  aee30630
05 Mar, 2025 2 commits
- [Eagle] Refactor eagle speculative decoding (#3986) · d3d4d767
  Ying Sheng authored Mar 05, 2025
```
Co-authored-by: Ke Bao <ISPObaoke@163.com>
```
  d3d4d767
- [Minor] more code cleanup (#4077) · e074d84e
  Lianmin Zheng authored Mar 04, 2025
  
  e074d84e
24 Feb, 2025 1 commit

Hierarchical Caching for SGLang (#2693) · 6c7a152c

Zhiqiang Xie authored Feb 23, 2025


Co-authored-by: Wenxuan Tan <wenxuan.tan@wisc.edu>
Co-authored-by: Yineng Zhang <me@zhyncs.com>

6c7a152c

20 Jan, 2025 1 commit
- Fix perf regression on small batch sizes (#3008) · dc188132
  Lianmin Zheng authored Jan 20, 2025
  
  dc188132
19 Jan, 2025 1 commit
- Remove the unused write_with_records (#2972) · 7906d1d2
  Lianmin Zheng authored Jan 18, 2025
  
  7906d1d2
13 Jan, 2025 3 commits
- Add a new api configure_logging to allow dumping the requests (#2875) · 46d44318
  Lianmin Zheng authored Jan 13, 2025
  
  46d44318
- CUDA-graph-compatible releasing and resuming KV cache and model weight memory (#2630) · 923f5183
  fzyzcjy authored Jan 14, 2025
  
  923f5183
- Fix linear.py and improve weight loading (#2851) · 72c77763
  Lianmin Zheng authored Jan 13, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  72c77763