Commits · 53bd00d975ff1a0a4683a4f006c19636d529199f · change / sglang

09 Oct, 2025 1 commit
- [Generative Score API] Multi-Item scoring with custom attention mask. (#10979) · 53bd00d9
  Sundara Raman Ramachandran authored Oct 08, 2025
  
  53bd00d9
08 Oct, 2025 3 commits
- [2/2] Support MHA prefill with FlashAttention 4. (#10937) · edefab0c
  Lifu Huang authored Oct 08, 2025
```
Co-authored-by: Hieu Pham <hyhieu@gmail.com>
```
  edefab0c
- [8/N] MoE Refactor: deprecate `EPMoE` (#11211) · 3c06b673
  Cheng Wan authored Oct 07, 2025
  
  3c06b673
- [oai serving chat] Add argument `--sampling-defaults` and fix... · 7ba3de0e
  Chang Su authored Oct 07, 2025
```
[oai serving chat] Add argument `--sampling-defaults` and fix `ChatCompletionRequest` defaults (#11304)
```
  7ba3de0e
07 Oct, 2025 1 commit
- Update tool parser and related documentation (#11223) · e3c7f091
  Xinyuan Tong authored Oct 07, 2025
  
  e3c7f091
06 Oct, 2025 3 commits

Rename max_micro_batch_size -> pp_max_micro_batch_size (#11279) · 708f4ff4
Lianmin Zheng authored Oct 06, 2025

708f4ff4
Enable native ModelOpt quantization support (1/3) (#7149) · 155cbb51
Zhiyu authored Oct 06, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
```
155cbb51

Support DeepSeek V3.2 Exp (#11061) · efbc687c

fzyzcjy authored Oct 06, 2025


Co-authored-by: Stefan He <11166516+hebiao064@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <95566987+hnyls2002@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <56809903+fridge003@users.noreply.github.com>
Co-authored-by: DarkSharpness <76582120+darksharpness@users.noreply.github.com>
Co-authored-by: ZhengdQin <46387172+zhengdqin@users.noreply.github.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>

efbc687c

05 Oct, 2025 1 commit
- [Feat] Support Torch Symm Mem AllReduce (#10571) · 590f2da0
  Yuan Luo authored Oct 06, 2025
```
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
```
  590f2da0
03 Oct, 2025 3 commits
- Tiny move files to utils folder (#11166) · fdc4e1e5
  fzyzcjy authored Oct 03, 2025
  
  fdc4e1e5
- [Feature] Option to save model weights to CPU when memory saver mode is enabled (#10873) · 8c574902
  Matt Nappo authored Oct 03, 2025
```
Co-authored-by: molocule <34072934+molocule@users.noreply.github.com>
```
  8c574902
- [1/n] Enable DCA CUDA graph capture (#9537) · a2faf894
  b8zhong authored Oct 02, 2025
  
  a2faf894
02 Oct, 2025 4 commits
- Minor fixes for server_args, parallel_state, and test_deterministic.py (#11159) · 6a261aac
  Lianmin Zheng authored Oct 02, 2025
  
  6a261aac
- fix cpp JIT compilation issue of ngram speculative decoding (#10837) · 948278f1
  b8zhong authored Oct 02, 2025
  
  948278f1
- Support single batch overlap (#10422) · 5e786cca
  fzyzcjy authored Oct 02, 2025
  
  5e786cca
- Fix ngram spec with page size > 1 (#11135) · 25e7dbe8
  Liangsheng Yin authored Oct 02, 2025
  
  25e7dbe8
01 Oct, 2025 1 commit
- [Feature] Add EIC as sglang HiCache Storage backend (#10271) · 86cb4db0
  Shisong Ma authored Oct 01, 2025
```
Co-authored-by: mashisong <mashisong@bytedance.com>
```
  86cb4db0
30 Sep, 2025 4 commits
- Fix DSR1 accuracy for flashinfer_trtllm MoE with FP8 quantization (#11081) · a6cc86df
  Trevor Morris authored Sep 30, 2025
  
  a6cc86df
- [CPU] Adding Memory Capacity Acquisition Functionality (#11102) · 229d2b95
  Zaili Wang authored Sep 30, 2025
  
  229d2b95
- Enable optional FP32 compute for LM Head (#10729) · d17986f8
  narutolhy authored Sep 29, 2025
```
Thanks to MiniMax Team and Chenyang Zhao's support.
```
  d17986f8
- Remove hybrid_linear_attn attention backend and refactor attention registry (#10816) · 2bc61dd1
  li-kesen authored Sep 30, 2025
```
Co-authored-by: Yi Zhang <1109276519@qq.com>
```
  2bc61dd1
29 Sep, 2025 4 commits
- Fix mem fraction static for nightly tests (#11076) · dda34c2f
  Lianmin Zheng authored Sep 29, 2025
  
  dda34c2f
- Use more general heuristics to set the default value of --mem-fraction-static (#10975) · a17e70f5
  Lianmin Zheng authored Sep 29, 2025
```
Co-authored-by: sglang-bot <sglangbot@gmail.com>
```
  a17e70f5
- [speculative decoding] rename lookahead to ngram (#11010) · 24f7cb1e
  Zhihao Zhang authored Sep 29, 2025
```
Co-authored-by: a4zhangfei <a4zhangfei@qq.com>
```
  24f7cb1e
- [HiCacheStorage] mooncake store support page_first_direct layout (#10591) · e05555fa
  huangtingwei authored Sep 29, 2025
  
  e05555fa
28 Sep, 2025 1 commit
- fix: show failed models in nightly ci (#10986) · 2e763398
  Mick authored Sep 29, 2025
  
  2e763398
27 Sep, 2025 2 commits

fix: make inference deterministic for large TP (#10930) · 62e2e99d

Xinyuan Tong authored Sep 27, 2025


Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

62e2e99d

[HiCache]: Support dynamic loading backends for hicache (#10551) · c8a5d12a
hzh0425 authored Sep 27, 2025
```
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
```
c8a5d12a

26 Sep, 2025 2 commits
- Restruct gpu_memory_settings in a unify function and relax max_cuda_graph_bs (#10372) · 05a35266
  Xiaoyu Zhang authored Sep 27, 2025
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: sglang-bot <sglangbot@gmail.com>
```
  05a35266
- [PD-HiCache]: Support Async Offloading KVCache In Decode Side (#10192) · 7ec5b4e8
  hzh0425 authored Sep 26, 2025
```
Signed-off-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
```
  7ec5b4e8
25 Sep, 2025 2 commits
- Rename customer label -> custom label (#10899) · f68dd998
  Lianmin Zheng authored Sep 25, 2025
```
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  f68dd998
- integrate AIBrix KVcache (#10376) · fce17048
  yi wang authored Sep 25, 2025
  
  fce17048
24 Sep, 2025 1 commit

chore: Initial support for input config files (#10534) · d7b20dd6

kushanam authored Sep 24, 2025


Co-authored-by: root <root@umbriel-b200-017.ipp4a1.colossus.nvidia.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>

d7b20dd6

23 Sep, 2025 2 commits
- Clean up server args (#10770) · ea338676
  Lianmin Zheng authored Sep 23, 2025
  
  ea338676
- [Ascend] optimize Qwen-vl on Ascend (#10556) · 095093ee
  ronnie_zheng authored Sep 23, 2025
```
Co-authored-by: wangqihui01 <wangqh10@163.com>
```
  095093ee
22 Sep, 2025 3 commits
- Add warnings and remove dependency for deterministic inference (#10724) · aa1c5cf5
  Baizhou Zhang authored Sep 22, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  aa1c5cf5
- [2/2] Support deterministic inference for temperature > 0 (#10678) · e2ac7888
  Qiaolin Yu authored Sep 21, 2025
```
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
```
  e2ac7888
- Support deterministic inference with triton backend (#10694) · 134b4f7e
  Ethan (Yusheng) Su authored Sep 21, 2025
  
  134b4f7e
21 Sep, 2025 2 commits
- [3/4] Speed up CSGMV backend perf by 10% through dynamic chunking + kernel optimization (#10592) · 08ecd0aa
  Lifu Huang authored Sep 20, 2025
  
  08ecd0aa
- Super tiny fix extra logs (#10697) · 720c1c8c
  fzyzcjy authored Sep 21, 2025
  
  720c1c8c