Commits · eb6c2c166322352b30fb1adb6d011dcfb5c6916c · change / sglang · GitLab

28 Jun, 2025 1 commit

Hybrid kv cache for LLaMA4 (#6563) · eb6c2c16

tarinkk authored Jun 27, 2025


Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>

eb6c2c16

26 Jun, 2025 1 commit
- chore: bump v0.4.8.post1 (#7559) · 69183f88
  Yineng Zhang authored Jun 26, 2025
  
  69183f88
24 Jun, 2025 1 commit
- chore: bump v0.4.8 (#7493) · 7c3a12c0
  Yineng Zhang authored Jun 23, 2025
  
  7c3a12c0
16 Jun, 2025 1 commit
- chore: bump v0.4.7.post1 (#7248) · f9dc9dd2
  Yineng Zhang authored Jun 16, 2025
  
  f9dc9dd2
10 Jun, 2025 2 commits
- Improve perf tuning docs (#7071) · 90bd3e32
  Lianmin Zheng authored Jun 10, 2025
  
  90bd3e32
- chore: bump v0.4.7 (#7038) · 4f723edd
  Yineng Zhang authored Jun 10, 2025
  
  4f723edd
09 Jun, 2025 1 commit
- Fix torch profiler bugs for bench_offline_throughput.py (#6557) · 98c00a2d
  Yueyang Pan authored Jun 09, 2025
  
  98c00a2d
06 Jun, 2025 1 commit
- AITER backend extension and workload optimizations (#6838) · b819381f
  HAI authored Jun 05, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
```
  b819381f
28 May, 2025 1 commit
- [Feature] Support Flashinfer fp8 blockwise GEMM kernel on Blackwell (#6479) · 791b3bfa
  Baizhou Zhang authored May 28, 2025
  
  791b3bfa
27 May, 2025 1 commit
- update toc for doc and dockerfile code style format (#6450) · 7a0bbe6a
  linzhuo authored May 27, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  7a0bbe6a
26 May, 2025 1 commit
- Auto handle PD disaggregation in bench_serving (#6587) · 25be63d0
  fzyzcjy authored May 26, 2025
```
Co-authored-by: yizhang2077 <1109276519@qq.com>
```
  25be63d0
25 May, 2025 1 commit
- Fix some issues with current docs. (#6588) · e235be16
  simveit authored May 25, 2025
  
  e235be16
15 May, 2025 1 commit

[Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109) · 2e4babdb

quinnrong94 authored May 15, 2025


Co-authored-by: Yingyi <yingyihuang2000@outlook.com>
Co-authored-by: neiltian <neiltian@tencent.com>
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
Co-authored-by: kexueyu <kexueyu@tencent.com>
Co-authored-by: vincentmeng <vincentmeng@tencent.com>
Co-authored-by: pengmeng <pengmeng@tencent.com>

2e4babdb

13 May, 2025 1 commit
- [Llama4] Add docs note about enable multimodal (#6235) · 3c32895c
  Brayden Zhong authored May 12, 2025
  
  3c32895c
12 May, 2025 3 commits
- Revert "fix some typos" (#6244) · e8e18dcd
  Lianmin Zheng authored May 12, 2025
  
  e8e18dcd
- [Docs] Add docs for `SGLANG_` and `SGL_` environment variables (#6206) · 12319a67
  Brayden Zhong authored May 12, 2025
  
  12319a67
- fix some typos (#6209) · d738ab52
  applesaucethebun authored May 12, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  d738ab52
08 May, 2025 1 commit
- Update doc for MLA attention backends (#6034) · 8f508cc7
  Baizhou Zhang authored May 07, 2025
  
  8f508cc7
02 May, 2025 1 commit
- feat: Refactor DeepSeekV3 function call (#5908) · 170d1f21
  Chang Su authored May 01, 2025
  
  170d1f21
01 May, 2025 1 commit
- Update ci test and doc for MTP api change (#5952) · ebaba856
  Ke Bao authored May 02, 2025
  
  ebaba856
27 Apr, 2025 3 commits
- [Docs] update grafana setup guide in production metrics (#5643) · 86317c09
  Huapeng Zhou authored Apr 28, 2025
```
Co-authored-by: NoahM <88418672+zhudianGG@users.noreply.github.com>
```
  86317c09
- support for the DeepSeek model by enabling streaming response parsing (#5592) · a21ef363
  Frankey_8080 authored Apr 27, 2025
  
  a21ef363
- [Minor] fix documentations (#5756) · 155890e4
  Lianmin Zheng authored Apr 26, 2025
  
  155890e4
22 Apr, 2025 1 commit
- Turn on DeepGemm By Default and Update Doc (#5628) · ce5412b6
  Baizhou Zhang authored Apr 22, 2025
  
  ce5412b6
21 Apr, 2025 2 commits
- [Feat.] Enable grafana to show metrics (#4718) · 57131dd9
  Huapeng Zhou authored Apr 21, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  57131dd9
- add function call parser for DeepSeek V3 (#5224) · fac17acf
  Yi Zhou authored Apr 21, 2025
  
  fac17acf
20 Apr, 2025 2 commits
- Super tiny fix typo (#5559) · 9c434777
  fzyzcjy authored Apr 21, 2025
  
  9c434777
- [Doc]Add instruction for profiling with bench_one_batch (#5581) · b54b5a96
  Baizhou Zhang authored Apr 20, 2025
  
  b54b5a96
17 Apr, 2025 2 commits
- Deprecate enable-flashinfer-mla and enable-flashmla (#5480) · 6fb29ffd
  Baizhou Zhang authored Apr 17, 2025
  
  6fb29ffd
- Deprecate disable-mla (#5481) · 4fb05583
  Baizhou Zhang authored Apr 17, 2025
  
  4fb05583
16 Apr, 2025 2 commits
- [doc] Update benchmark_and_profiling.md (#5449) · 06a1656e
  Xiaoyu Zhang authored Apr 16, 2025
  
  06a1656e
- Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113) · a42736bb
  Baizhou Zhang authored Apr 15, 2025
  
  a42736bb
15 Apr, 2025 1 commit
- [Fix] Turn off DeepGEMM by default (#5263) · f6772f14
  Baizhou Zhang authored Apr 14, 2025
  
  f6772f14
11 Apr, 2025 1 commit
- [Docs] Remove the older supported docs section (#5301) · a0a9f6d6
  Adarsh Shirawalmath authored Apr 12, 2025
  
  a0a9f6d6
09 Apr, 2025 1 commit
- docs: remove the use of Downward API for LWS_WORKER_INDEX (#5110) · f2b70afd
  Kay Yan authored Apr 09, 2025
```
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
```
  f2b70afd
08 Apr, 2025 1 commit
- Add Llama4 user guide (#5133) · ade714a6
  Ke Bao authored Apr 08, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  ade714a6
07 Apr, 2025 1 commit

Add Llama4 support (#5092) · f04c80dc

Chang Su authored Apr 07, 2025


Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: ispobock <ispobaoke@163.com>

f04c80dc

05 Apr, 2025 1 commit
- [Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052) · efbae697
  Baizhou Zhang authored Apr 05, 2025
  
  efbae697
03 Apr, 2025 2 commits
- Revert "Replace enable_flashinfer_mla argument with attention_backend" (#5048) · 74885a84
  Lianmin Zheng authored Apr 03, 2025
  
  74885a84
- Replace enable_flashinfer_mla argument with attention_backend (#5005) · e8999b13
  Baizhou Zhang authored Apr 03, 2025
  
  e8999b13