Commits · 094fbdacd5bb27e9fdb9336cd81f36ad4fcc11d9 · change / sglang · GitLab

31 May, 2025 5 commits
- Fix incorrect LoRA weight loading for fused gate_up_proj (#6734) · 094fbdac
  Lifu Huang authored May 31, 2025
  
  094fbdac
- Add intel_amx backend for Radix Attention for CPU (#6408) · 888cb175
  YanbingJiang authored May 31, 2025
```
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  888cb175
- Support token-level quantization for EP MoE (#6782) · ced3c07a
  Cheng Wan authored May 30, 2025
  
  ced3c07a
- feat(tool call): Enhance Llama32Detector for improved JSON parsing in non-stream (#6784) · f18b068f
  Chang Su authored May 30, 2025
  
  f18b068f
- update llama4 chat template and pythonic parser (#6679) · 4fac524b
  Chao Yang authored May 30, 2025
```
Co-authored-by: Chang Su <chang.s.su@oracle.com>
```
  4fac524b
30 May, 2025 6 commits
- Fix one bug in the grouped-gemm triton kernel (#6772) · b581b225
  Cheng Wan authored May 30, 2025
  
  b581b225
- Fix shared experts fusion error (#6289) · 69dd878b
  Li Hui authored May 30, 2025
  
  69dd878b
- Support sliding window in triton backend (#6509) · 22630ca2
  Jianan Ji authored May 30, 2025
  
  22630ca2
- Fix aiohttp 'Chunk too big' in bench_serving (#6737) · d279d499
  Yuhong Guo authored May 30, 2025
  
  d279d499
- [PD] Optimize time out logic and add env var doc for mooncake (#6761) · 6cb00c63
  shangmingc authored May 30, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  6cb00c63
- Improve EPLB logical to physical dispatch map (#6727) · 2c3b71d6
  fzyzcjy authored May 30, 2025
  
  2c3b71d6
29 May, 2025 14 commits
- [fix][RL] Fix DeepSeekV3ForCausalLM.post_load_weights for multiple update weight (#6265) · 51cdd81f
  Zilin Zhu authored May 30, 2025
  
  51cdd81f
- Fix mem_fraction_static for AMD CI (#6748) · 73def253
  Baizhou Zhang authored May 29, 2025
  
  73def253
- Support picking variants of EPLB algorithms (#6728) · 3ab7d9b5
  fzyzcjy authored May 29, 2025
  
  3ab7d9b5
- Super tiny enable sole usage of expert distribution metrics and update doc (#6680) · 7e5071c9
  fzyzcjy authored May 29, 2025
  
  7e5071c9
- PD Rust LB (PO2) (#6437) · 78689d33
  Liangsheng Yin authored May 29, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  78689d33
- [PD] Support completion endpoint (#6729) · 1dc6864f
  shangmingc authored May 29, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  1dc6864f
- refactor apply_w8a8_block_fp8_linear in fp (#6545) · 485a023b
  ChangyiYang authored May 29, 2025
  
  485a023b
- Add draft extend CUDA graph for Triton backend (#6705) · 7e412900
  Ke Bao authored May 29, 2025
  
  7e412900
- refactor(tool call): Fix BaseFormatDetector tool_index issue and refactor... · c673727e
  Chang Su authored May 29, 2025
```
refactor(tool call): Fix BaseFormatDetector tool_index issue and refactor `parse_streaming_increment` (#6715)
```
  c673727e
- Tune memory arguments on B200 (#6718) · f2bd3515
  Baizhou Zhang authored May 29, 2025
  
  f2bd3515
- [PD] bug fix: Update status if nixl receiver send a a dummy req. (#6720) · c459536b
  dongmao zhang authored May 29, 2025
  
  c459536b
- [fix] more mem for draft_extend cuda_graph (#6726) · 535c8386
  JieXin Liang authored May 29, 2025
  
  535c8386
- [feat] triton kernel for get_last_loc (#6676) · 2163586e
  JieXin Liang authored May 29, 2025
  
  2163586e
- Fix PP for Qwen3 MoE (#6709) · e06b0761
  iLeGend authored May 29, 2025
  
  e06b0761
28 May, 2025 13 commits
- [Feature] Support Flashinfer fp8 blockwise GEMM kernel on Blackwell (#6479) · 791b3bfa
  Baizhou Zhang authored May 28, 2025
  
  791b3bfa
- Speed up when having padding tokens two-batch overlap (#6668) · 31589e17
  fzyzcjy authored May 29, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  31589e17
- Minor refactor two-batch overlap (#6682) · ae6a5b29
  fzyzcjy authored May 29, 2025
  
  ae6a5b29
- Overlap two kernels in DeepSeek with communication (#6711) · 4839999b
  fzyzcjy authored May 29, 2025
  
  4839999b
- Fuse routed_scaling_factor in DeepSeek (#6710) · 541a985f
  fzyzcjy authored May 29, 2025
  
  541a985f
- [PD] Remove Unnecessary Exception Handling for FastQueue.get() (#6712) · 5170b010
  Hongbo Xu authored May 29, 2025
  
  5170b010
- [Bugfix] Fix ChatCompletion endpoint of mini_lb when stream is set (#6703) · e9fd11c0
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  e9fd11c0
- [Bugfix] Fix slice operation when chunk size mismatch (#6697) · c7588d59
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  c7588d59
- [PD Perf] replace Queue to FastQueue (#6649) · 6b231325
  ybyang authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  6b231325
- [PD] Abort unbootstrapped prefill requests through timeout (#6685) · b1c8d4e9
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  b1c8d4e9
- [Bugfix] Fix missing abort finish reason for PD with ChatCompletion (#6693) · fba03b29
  shangmingc authored May 28, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  fba03b29
- fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool... · 461a7302
  Chang Su authored May 28, 2025
```
fix(deepseekv3): Fix DeepSeekV3Detector tool_index assignment and multi-tool call streaming support (#6655)
```
  461a7302
- Refine pre_reorder_triton_kernel slightly to improve performance (#6627) · c087ddd6
  Yuan Luo authored May 28, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  c087ddd6
27 May, 2025 2 commits
- feat: Add warnings for invalid tool_choice and UTs (#6582) · 41ba767f
  Chang Su authored May 27, 2025
  
  41ba767f
- fix(tool call): Fix tool_index in PythonicDetector and issues with mixed... · bdb962d7
  Chang Su authored May 27, 2025
```
fix(tool call): Fix tool_index in PythonicDetector and issues with mixed output in non-streaming (#6678)
```
  bdb962d7