Commits · ab1a4fa5cbb0cf5d2ea9a2f3b36f16fd129cf945 · change / sglang · GitLab

14 Jun, 2025 1 commit
- [fix] fix cutlass_mla_backend with cuda_graph and add sm_scale for sgl-kernel cutlass_mla (#7184) · ab1a4fa5
  JieXin Liang authored Jun 15, 2025
  
  ab1a4fa5
13 Jun, 2025 3 commits
- Minor style change of triton backend (#7165) · be2d985d
  Lianmin Zheng authored Jun 13, 2025
  
  be2d985d
- Remove 200us slow concat kernel (part 2: srt) (#7020) · c49c1d92
  fzyzcjy authored Jun 14, 2025
  
  c49c1d92
- Remove unnecessary metadata_expand.max_seq_len_k operations in fa3 to… (#7140) · 22a6b9fc
  Binyao Jiang authored Jun 12, 2025
  
  22a6b9fc
11 Jun, 2025 2 commits
- vlm: adapt internvl to VisionAttention (#6870) · 83d87685
  Mick authored Jun 11, 2025
  
  83d87685
- Simplify the heuristics for setting --mem-fraction-static (#7054) · 6b12d6a8
  Lianmin Zheng authored Jun 10, 2025
  
  6b12d6a8
10 Jun, 2025 2 commits
- [WA] fix output data is nan in CI test "test_moe_eval_accuracy_large.py" (#7021) · 8ea7df61
  kk authored Jun 11, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
```
  8ea7df61
- Minor cleanup of fa3 backend (#6999) · 2dae104d
  Lianmin Zheng authored Jun 10, 2025
  
  2dae104d
09 Jun, 2025 2 commits
- Fix cutlass MLA gets almost zero accuracy (#6998) · e58423b2
  fzyzcjy authored Jun 10, 2025
  
  e58423b2
- Use torch.compile to fuse flash attention decode metadata preparation (#6973) · 9d5fa68b
  Lianmin Zheng authored Jun 08, 2025
  
  9d5fa68b
06 Jun, 2025 2 commits
- [Feature] Support Flashinfer fmha on Blackwell (#6930) · 5f91c825
  Jianan Ji authored Jun 06, 2025
  
  5f91c825
- AITER backend extension and workload optimizations (#6838) · b819381f
  HAI authored Jun 05, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
```
  b819381f
02 Jun, 2025 1 commit
- Add draft extend CUDA graph for flashinfer backend (#6805) · a2cb5913
  Ke Bao authored Jun 02, 2025
  
  a2cb5913
31 May, 2025 1 commit

Add intel_amx backend for Radix Attention for CPU (#6408) · 888cb175

YanbingJiang authored May 31, 2025


Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>

888cb175

30 May, 2025 1 commit
- Support sliding window in triton backend (#6509) · 22630ca2
  Jianan Ji authored May 30, 2025
  
  22630ca2
29 May, 2025 1 commit
- Add draft extend CUDA graph for Triton backend (#6705) · 7e412900
  Ke Bao authored May 29, 2025
  
  7e412900
28 May, 2025 1 commit
- Minor refactor two-batch overlap (#6682) · ae6a5b29
  fzyzcjy authored May 29, 2025
  
  ae6a5b29
27 May, 2025 1 commit
- Support EAGLE draft extend CUDA graph (#6606) · 63195028
  Ke Bao authored May 27, 2025
```
Co-authored-by: Sehoon Kim <sehoonkim@berkeley.edu>
```
  63195028
25 May, 2025 1 commit
- Support overlapping two batches (#4068) · 0d477880
  fzyzcjy authored May 25, 2025
  
  0d477880
24 May, 2025 1 commit
- Fix GPU OOM (#6564) · 7a5e6ce1
  kk authored May 25, 2025
```
Co-authored-by: michael <michael.zhang@amd.com>
```
  7a5e6ce1
21 May, 2025 1 commit
- aiter attention-backend (default enabled on AMD/ROCm) (#6381) · 5c0b38f3
  HAI authored May 20, 2025
  
  5c0b38f3
20 May, 2025 1 commit
- [doc] add note for get_num_kv_splits in triton_backend (#6444) · 69af3ec3
  JieXin Liang authored May 20, 2025
  
  69af3ec3
18 May, 2025 3 commits
- [OAI] Add rid tracing for v1/embeddings and fix rid type in Chat (#6397) · 066cf445
  Chang Su authored May 18, 2025
  
  066cf445
- [fix] fix fa3 forward_decode with spec_decode (#6395) · 1f30c05d
  JieXin Liang authored May 19, 2025
```
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
```
  1f30c05d
- refactor: minor refactors regarding multimodal processing (#6187) · 01dd39ba
  Mick authored May 18, 2025
  
  01dd39ba
17 May, 2025 1 commit
- perf: Optimize local attention memory allocation in FlashAttentionBackend (#6356) · 205d5cb4
  Chang Su authored May 17, 2025
  
  205d5cb4
15 May, 2025 1 commit

[Feat] Support FlashMLA backend with MTP and FP8 KV cache (#6109) · 2e4babdb

quinnrong94 authored May 15, 2025


Co-authored-by: Yingyi <yingyihuang2000@outlook.com>
Co-authored-by: neiltian <neiltian@tencent.com>
Co-authored-by: lukec <118525388+sleepcoo@users.noreply.github.com>
Co-authored-by: kexueyu <kexueyu@tencent.com>
Co-authored-by: vincentmeng <vincentmeng@tencent.com>
Co-authored-by: pengmeng <pengmeng@tencent.com>

2e4babdb

14 May, 2025 1 commit
- perf: optimize local_block_table memory allocation (#6273) · 912788c0
  Chang Su authored May 13, 2025
  
  912788c0
12 May, 2025 3 commits
- Revert "fix some typos" (#6244) · e8e18dcd
  Lianmin Zheng authored May 12, 2025
  
  e8e18dcd
- fix some typos (#6209) · d738ab52
  applesaucethebun authored May 12, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  d738ab52
- Log if cuda graph is used & extend cuda graph capture to cuda-graph-max-bs (#6201) · fba8eccd
  Lianmin Zheng authored May 12, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  fba8eccd
11 May, 2025 1 commit
- Add typo checker in pre-commit (#6179) · 2ce87935
  applesaucethebun authored May 11, 2025
```
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  2ce87935
09 May, 2025 2 commits
- [Bugfix] Fix Llama4 gibberish output with long context and CUDA graph (#6162) · 1940cdec
  Chang Su authored May 09, 2025
  
  1940cdec
- opt flashinfer mla cat (#5822) · e30c273b
  xu-yfei authored May 09, 2025
```
Co-authored-by: xuyongfei.xyf <xuyongfei.xyf@antgroup.com>
```
  e30c273b
08 May, 2025 2 commits
- [Feature] Add FlashAttention3 as a backend for VisionAttention (#5764) · fa7d7fd9
  Zhu Chen authored May 09, 2025
```
Co-authored-by: othame <chenzhu_912@zju.edu.cn>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>
```
  fa7d7fd9
- optimize pad operations in fa3 to accelarate 100+us (#6077) · 79961afa
  Minglei Zhu authored May 07, 2025
  
  79961afa
05 May, 2025 2 commits
- feat: Add a unified merge_state API (#5428) · 82653f66
  DefTruth authored May 06, 2025
  
  82653f66
- Fix "Avoid computing lse in Ragged Prefill when there's no prefix match" (#5555) · 22da3d97
  Wenxuan Tan authored May 05, 2025
  
  22da3d97
04 May, 2025 1 commit
- [Fix] Suppress dynamo logging when using flashinfer backend with torch compile (#5992) · bf203cb7
  Baizhou Zhang authored May 04, 2025
  
  bf203cb7
02 May, 2025 1 commit
- FA3 speed up: skip len operation and get batch size directly from forward batch (#5969) · 1acca3a2
  Lifu Huang authored May 02, 2025
```
Signed-off-by: Lifu Huang <lifu.hlf@gmail.com>
```
  1acca3a2