Commits · 4a9f3eef90a5cbf768bea0a70432e4f1c507e8ec · change / sglang · GitLab

09 Aug, 2025 18 commits
- Tiny Llama4 type error in constructor (#6752) · 4a9f3eef
  Brayden Zhong authored Aug 09, 2025
  
  4a9f3eef
- feature(hicache): Support hf3fs-hicache reusing kvcache across different instances (#8673) · 1b7afad0
  hzh0425 authored Aug 09, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  1b7afad0
- Support glm4.1v and glm4.5v (#8798) · f29aba8c
  Binyao Jiang authored Aug 09, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Chang Su <csu272@usc.edu>
```
  f29aba8c
- feat: update flashinfer ar oneshot params (#8687) · faa25df1
  eigen authored Aug 09, 2025
  
  faa25df1
- Fix qwen2 audio not working bug (#8600) · 7b81f956
  Binyao Jiang authored Aug 09, 2025
  
  7b81f956
- Fix redundant kernel in sink dtype conversion (#8966) · d3e67deb
  fzyzcjy authored Aug 09, 2025
  
  d3e67deb
- Add CI for gpt-oss model on hopper (#8851) · 442534aa
  fzyzcjy authored Aug 09, 2025
  
  442534aa
- chore(deps): update minimum python to 3.10 (#8984) · de8b8b6e
  ishandhanani authored Aug 09, 2025
  
  de8b8b6e
- optimize: reduce shulffle and quantization overhead in cutlass_moe sm90 (#8962) · 3f2e315f
  tql.99 authored Aug 09, 2025
```
Co-authored-by: 戚余航 <qiyuhang@bytedance.com>
```
  3f2e315f
- Fix incorrect default get_hidden_dim logic (#8987) · 6e215118
  Lifu Huang authored Aug 09, 2025
  
  6e215118
- [hotfix] use the original implementation in 8785 (#8994) · a47baff1
  Cheng Wan authored Aug 08, 2025
  
  a47baff1
- Revert "[bug fix] Ensure local token and global token buffers are pointing to... · fd7e15b7
  Cheng Wan authored Aug 08, 2025
```
Revert "[bug fix] Ensure local token and global token buffers are pointing to different storage " (#8993)
```
  fd7e15b7
- [Fix] Fix wrong backend chosen in hybrid backend (#8989) · fc42ff7b
  DarkSharpness authored Aug 08, 2025
  
  fc42ff7b
- Molly/ci gnr server (#8667) · 7c0db868
  DiweiSun authored Aug 09, 2025
  
  7c0db868
- Clean up server_args.py to have a dedicated function for model specific adjustments (#8983) · 706bd69c
  Lianmin Zheng authored Aug 08, 2025
  
  706bd69c
- [AMD] Update SGLang image fallback logic for AMD CI (#8980) · 23f2afb2
  michael-amd authored Aug 08, 2025
  
  23f2afb2
- Add unit test for flashinfer fp4 moe (#8330) · a60f88b5
  Trevor Morris authored Aug 08, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  a60f88b5
- [1/2][resubmit] sgl-kernel: Fuse routed scaling factor into moe_fused_gate (select_experts) (#8770) · 591c232f
  Trevor Morris authored Aug 08, 2025
  
  591c232f
08 Aug, 2025 22 commits
- Minor Optimizations in Schedule Batch (#8724) · f352b793
  Lianmin Zheng authored Aug 08, 2025
```
Co-authored-by: Suruchi Shah <surshah@linkedin.com>
```
  f352b793
- [Fix] Add a workflow to cancel all pending CI runs (#8988) · 6642e3a2
  Lianmin Zheng authored Aug 08, 2025
  
  6642e3a2
- Create cancel-all-pr-test-runs (#8986) · 67a7d1f6
  Lianmin Zheng authored Aug 08, 2025
  
  67a7d1f6
- [bug fix] Ensure local token and global token buffers are pointing to different storage (#8785) · 92cbef59
  Elfie Guo authored Aug 08, 2025
  
  92cbef59
- Update qwen3_coder_detector.py for streaming (#8371) · b3359dc9
  maocheng23 authored Aug 08, 2025
  
  b3359dc9
- [router] fix radix tree integration issues in PD router (#8982) · 7b7e5615
  Simo Lin authored Aug 08, 2025
  
  7b7e5615
- [router] reduce contention, fix double-count race (#8978) · 1a8706c8
  Simo Lin authored Aug 08, 2025
  
  1a8706c8
- chore(ci): update Python version from 3.9 to 3.10 in sgl-kernel workflow (#8981) · 7d3af603
  ishandhanani authored Aug 08, 2025
  
  7d3af603
- chore(gb200): update to CUDA 12.9 and improve build process (#8772) · 4e7f0252
  ishandhanani authored Aug 08, 2025
  
  4e7f0252
- [router] add metrics for worker and policy (#8971) · 36bfddec
  Tony Lu authored Aug 09, 2025
```
Signed-off-by: Tony Lu <tonyluj@gmail.com>
```
  36bfddec
- Fix kimi k2 function call format (#8968) · 91e2f902
  Lianmin Zheng authored Aug 08, 2025
  
  91e2f902
- [router] harden retries + metrics; fix streaming load; header filtering (#8972) · a59cbea9
  Simo Lin authored Aug 08, 2025
  
  a59cbea9
- refine aiter_backend for mtp (#7279) · 53f7874a
  valarLip authored Aug 09, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
```
  53f7874a
- [router] router circuit breaker core (#8941) · 61a46804
  Simo Lin authored Aug 08, 2025
  
  61a46804
- chore: bump v0.5.0rc0 (#8959) · 9020f7fc
  Yineng Zhang authored Aug 08, 2025
  
  9020f7fc
- [RL] fix skip_server_warmup and rl health_generate logic (#8757) · dd650e0e
  Zilin Zhu authored Aug 08, 2025
  
  dd650e0e
- Revert "Support Multi Process Tokenizer Manager" (#8960) · a9471542
  Lianmin Zheng authored Aug 08, 2025
  
  a9471542
- chore: update flashinfer (#8958) · 41357e51
  Yineng Zhang authored Aug 08, 2025
  
  41357e51
- Simple prefetch policy (#8692) · e2fd2b9c
  pansicheng authored Aug 08, 2025
  
  e2fd2b9c
- Support Multi Process Tokenizer Manager (#6555) · 7490e3f6
  ybyang authored Aug 08, 2025
```
Signed-off-by: ybyang <ybyang7@iflytek.com>
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: lw9527 <952799980@qq.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
```
  7490e3f6
- add zai-org/GLM-4.5-Air-FP8 model into nightly CI (#8894) · 6ee6619b
  Minglei Zhu authored Aug 08, 2025
  
  6ee6619b
- chore: bump sgl-kernel v0.3.3 (#8957) · 54ea57f2
  Yineng Zhang authored Aug 08, 2025
  
  54ea57f2