Commits · 4373df55258e1eff6bf2b3899c78acf90eccfa83 · zhaoyu6 / sglang

06 Aug, 2025 5 commits

add flashinfer mxfp4 (#8847) · 4373df55
Xiaoyu Zhang authored Aug 07, 2025

4373df55

Support v1/responses and use harmony in serving_chat (#8837) · 92cc32d9

Chang Su authored Aug 06, 2025


Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>

92cc32d9

Support mxfp4 for GPT-OSS (#8843) · 168033d5

Ying Sheng authored Aug 06, 2025


Co-authored-by: Co-author fzyzcjy <ch271828n@outlook.com>
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
Co-authored-by: zhuofan1123 <zhuofanl@nvidia.com>
Co-authored-by: liz-badada <jinyanc@nvidia.com>
Co-authored-by: xutizhou <xutingz@nvidia.com>
Co-authored-by: linhu-nv <linhu@nvidia.com>

168033d5

[Feature] improve TBO: two chunk overlap (#8144) · ca47e24f
HouseWest authored Aug 06, 2025

ca47e24f
Turn off hybrid cache by default (#8839) · 8128e08d
Ke Bao authored Aug 06, 2025

8128e08d

05 Aug, 2025 4 commits
- Add initial support for gpt-oss (#8824) · c1d2061f
  Ying Sheng authored Aug 05, 2025
  
  c1d2061f
- feat: add trtllm-gen mha from direct call (#8782) · 40e3b2be
  eigen authored Aug 05, 2025
```
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  40e3b2be
- Support OCP MXFP4 quantization on AMD GPUs (#8255) · d4bf5a85
  kk authored Aug 05, 2025
```
Co-authored-by: wunhuang <wunhuang@amd.com>
Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>
```
  d4bf5a85
- [Fix] Fix several issues preventing gemma3n LoRA support. (#8776) · 7cb20754
  Lifu Huang authored Aug 04, 2025
  
  7cb20754
04 Aug, 2025 1 commit
- [NVIDIA] Add Low Latency NVFP4 decode kernels from Flashinfer (#8552) · 915140fd
  azhurkevich authored Aug 04, 2025
```
Co-authored-by: Cheng Wan <cwan@x.ai>
```
  915140fd
03 Aug, 2025 4 commits
- Fix the missing 'lof' choice of --schedule-policy server args (#7114) · ed6f7597
  Yingchun Lai authored Aug 04, 2025
  
  ed6f7597
- [feat] support minimum token load balance in dp attention (#7379) · f7b2853f
  Guanhua Wang authored Aug 03, 2025
  
  f7b2853f
- Support limiting max loaded loras in CPU. (#8650) · 8675bdf2
  Lifu Huang authored Aug 03, 2025
  
  8675bdf2
- [FIX] Fix the nightly CI by disabling swa mem pool for gemma2 (#8693) · e314b084
  Lianmin Zheng authored Aug 02, 2025
  
  e314b084
01 Aug, 2025 4 commits
- Add support for NCCL symmetric memory for TP allreduces (#8238) · 82e6c3a6
  Nicolas Castet authored Aug 01, 2025
  
  82e6c3a6
- [5/N] MoE Refactor: Update MoE parallelism arguments (#8658) · 6c88f6c8
  Cheng Wan authored Aug 01, 2025
  
  6c88f6c8
- Interface change for kvcache io to support page first layout (#8318) · dd7ca006
  Zhiqiang Xie authored Jul 31, 2025
  
  dd7ca006
- [NVIDIA] Enable Flashinfer MoE blockscale fp8 backend for TP MoE (#8450) · aa4c66b5
  Kaixi Hou authored Jul 31, 2025
```
Co-authored-by: kushanam <42385577+kushanam@users.noreply.github.com>
```
  aa4c66b5
31 Jul, 2025 8 commits

TRTLLM Gen MLA Decode Kernel Integration (same as #7938) (#8632) · 4b04998d
Faraz authored Jul 31, 2025
```
Signed-off-by: Faraz Khoubsirat <58580514+farazkh80@users.noreply.github.com>
```
4b04998d
[bugfix] Fix flashinfer cutlass EP moe after MoE refactor (#8630) · b7170cc8
Trevor Morris authored Jul 31, 2025

b7170cc8

SGLang HiCache NIXL Connector (#8488) · 2cd2e27f

Vishwanath Venkatesan authored Jul 31, 2025


Signed-off-by: Vishwanath Venkatesan <vvenkatesan@nvidia.com>
Co-authored-by: Moein Khazraee <moein@nvidia.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>

2cd2e27f

[Optimization][Perf] Disable the GC during CUDA graph capture to speed up by up to 3x (#8577) · 4acf6902
Brayden Zhong authored Jul 31, 2025

4acf6902
[Feature] Hybrid EP and TP (#8590) · 7a1f7fc5
Cheng Wan authored Jul 31, 2025

7a1f7fc5

model: support Step3V (#8583) · 51c38163

Chang Su authored Jul 31, 2025


Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: nnnobody-code <nnnobody@foxmail.com>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Qiaolin-Yu <qy254@cornell.edu>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>

51c38163

Support l3 cache (mooncake store) for hiradix cache (#7211) · d9049592

huangtingwei authored Jul 31, 2025


Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: AniZpZ <zhuangsen.zp@antgroup.com>
Co-authored-by: zuoyuan <zhangzuo21@mails.tsinghua.edu.cn>
Co-authored-by: @wangyueneng.wyn <wangyueneng.wyn@antgroup.com>
Co-authored-by: JinYan Su <jinyansu792@gmail.com>

d9049592

Add hf3fs support for hicache storage (based on #7704) (#7280) · 29980334
pansicheng authored Jul 31, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
29980334

29 Jul, 2025 1 commit
- Split the scheduler into multiple mixin classes to reduce the file size (#8483) · a4c3b121
  Lianmin Zheng authored Jul 29, 2025
  
  a4c3b121
28 Jul, 2025 3 commits
- feat: throttle requests at scheduler based on --max_queued_requests (#7565) · 747dd450
  harrisonlimh authored Jul 28, 2025
  
  747dd450
- GLM-4.5 Model Support (#8224) · 6d6a8bc2
  Yuxuan Zhang authored Jul 28, 2025
```
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
```
  6d6a8bc2
- [feat] Support different attention backends for prefill and decode (#6338) · 28103384
  Qiaolin Yu authored Jul 27, 2025
```
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  28103384
27 Jul, 2025 1 commit
- [NVIDIA] Add Flashinfer MoE blockscale fp8 backend (#8036) · 85486b6f
  Kaixi Hou authored Jul 27, 2025
  
  85486b6f
26 Jul, 2025 1 commit
- Tiny assert EPLB is used together with expert parallel (#8381) · da0c0260
  fzyzcjy authored Jul 26, 2025
  
  da0c0260
25 Jul, 2025 3 commits
- Fix FP4 MoE accuracy from missing routed_scaling_factor (#8333) · 58c468f4
  Trevor Morris authored Jul 25, 2025
  
  58c468f4
- Clean up server_args, triton cache manager (#8332) · ed2e313e
  Lianmin Zheng authored Jul 25, 2025
  
  ed2e313e
- [Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs in Qwen3 detector (#8357) · f8260f25
  Chang Su authored Jul 25, 2025
  
  f8260f25
23 Jul, 2025 1 commit
- Introduce Stable LoRA ID System for Overlapped Updates and Prefix Caching (#8261) · 8abd3e77
  Lifu Huang authored Jul 23, 2025
  
  8abd3e77
22 Jul, 2025 1 commit
- Preliminary Support for Qwen3XMLDetector (#8260) · 0dfe2491
  yhyang201 authored Jul 23, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  0dfe2491
19 Jul, 2025 3 commits
- Support start up LoRA server without initial adapters (#8019) · 4e3defe5
  Lifu Huang authored Jul 19, 2025
  
  4e3defe5
- Clean up server args (#8161) · bb0e8a32
  Lianmin Zheng authored Jul 19, 2025
  
  bb0e8a32
- Support NVFP4 quantized dense models on AMD CDNA2/CDNA3 GPUs (#7302) · d918ab79
  Haohui Mai authored Jul 18, 2025
```
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
```
  d918ab79