Commits · eb6c2c166322352b30fb1adb6d011dcfb5c6916c · change / sglang

28 Jun, 2025 2 commits

Hybrid kv cache for LLaMA4 (#6563) · eb6c2c16

tarinkk authored Jun 27, 2025


Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>

eb6c2c16

Fix: Minicpm (#7612) · 357921aa
Xinyuan Tong authored Jun 27, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
357921aa

27 Jun, 2025 7 commits
- [router] add centralized configuration module for sgl-router (#7588) · c071198c
  Simo Lin authored Jun 27, 2025
  
  c071198c
- Fix broken CI TestVILAServer (#7610) · d7374d74
  Lifu Huang authored Jun 27, 2025
  
  d7374d74
- Move multimodal processors into a separate folder (#7581) · ce3a3e87
  Lianmin Zheng authored Jun 27, 2025
  
  ce3a3e87
- feat: support compatibility between MTP and two-batch-overlap (#7225) · 41650b0d
  Qiaolin Yu authored Jun 27, 2025
```
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
```
  41650b0d
- Updates transformers and timm dependencies (#7577) · 1b951620
  Xinyuan Tong authored Jun 27, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  1b951620
- [CI] Add CI Testing for Prefill-Decode Disaggregation with Router (#7540) · 29bd4c81
  Keyang Ru authored Jun 27, 2025
  
  29bd4c81
- Add e2e test for multi instance multi stage memory release/resume occupuation (#7208) · 031f64aa
  Ata Fatahi authored Jun 26, 2025
```
Signed-off-by: Ata Fatahi <immrata@gmail.com>
```
  031f64aa
26 Jun, 2025 11 commits
- Fix MTP error when enabling two-batch overlap (#7569) · 3d7cdb2e
  fzyzcjy authored Jun 27, 2025
  
  3d7cdb2e
- Updates Gemma3n MLP layer to adapt latest transformers version (#7573) · 604efe07
  Xinyuan Tong authored Jun 26, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  604efe07
- [Fix] incorrect assert in EPLB (#7575) · 1b8cf77b
  Cheng Wan authored Jun 26, 2025
  
  1b8cf77b
- [PD][NIXL] Set is_sorted=False to fix NIXL_ERR_NOT_FOUND (#7330) · bb9b608c
  Trevor Morris authored Jun 26, 2025
  
  bb9b608c
- chore: bump v0.4.8.post1 (#7559) · 69183f88
  Yineng Zhang authored Jun 26, 2025
  
  69183f88
- chore: remove vlm unnecessary import (#7541) · 9b00990b
  Xinyuan Tong authored Jun 26, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
```
  9b00990b
- chore: improve ci bug reporting (#7542) · 4d67025a
  Mick authored Jun 26, 2025
  
  4d67025a
- Update seed in CPU UTs to avoid flaky failure with single test (#7544) · 0e05fe8c
  YanbingJiang authored Jun 26, 2025
  
  0e05fe8c
- Add Tencent HunYuanMoEV1 model support (#7549) · 2390a2bc
  Meng, Peng authored Jun 26, 2025
  
  2390a2bc
- [CMake] Fix sgl-kernel CMakeLists for Blackwell (#7543) · 16d76b9f
  Ruihang Lai authored Jun 25, 2025
  
  16d76b9f
- [PD] Raise error for incompatible mooncake version and some minor fixes (#7527) · 5c214257
  Shangming Cai authored Jun 26, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  5c214257
25 Jun, 2025 20 commits
- Fix gathered_buffer issues in tbo (#7531) · b8df43ab
  Qiaolin Yu authored Jun 25, 2025
  
  b8df43ab
- Fix FP8 KV Cache Support in FA3 Backend (#7148) · a1c1ebe9
  Yuhong Guo authored Jun 25, 2025
  
  a1c1ebe9
- minor: 'role' must be system/assistant/tool, but case insensitive for now (#7499) · fe2a0f96
  mlmz authored Jun 25, 2025
  
  fe2a0f96
- feat: add return hidden_states at async generation (#7507) · 20beb370
  eigen authored Jun 25, 2025
  
  20beb370
- Fix typo of flash_cache (#7513) · 00fbd8a4
  Stefan He authored Jun 25, 2025
  
  00fbd8a4
- take aiter get_rope back (#7521) · 802815e4
  valarLip authored Jun 25, 2025
  
  802815e4
- enable aiter fp8 blockscale quant (#7520) · 4c6675c4
  valarLip authored Jun 25, 2025
  
  4c6675c4
- [PD] Add different TP sizes support for no-MLA models (#6793) · e21aa1df
  Hongbo Xu authored Jun 25, 2025
```
Co-authored-by: shangmingc <csmthu@gmail.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  e21aa1df
- feat: send kvmetrics from sglang scheduler (#6721) · f3cbd245
  zixuanzhang226 authored Jun 25, 2025
  
  f3cbd245
- npu fused op (#7386) · 506a2d59
  ll819214 authored Jun 25, 2025
```
Co-authored-by: Li Junwen <lijunwen13@hisilicon.com>
```
  506a2d59
- [CI] Upgrade mooncake to v0.3.4.post2 to fix potential slice failed bug (#7522) · a07f8ae4
  Shangming Cai authored Jun 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  a07f8ae4
- [CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel... · 7eb47b0f
  Chunyuan WU authored Jun 25, 2025
```
[CPU] [BF16] Call fused_experts_cpu, weight_packed_linear and bmm_cpu kernel in DeepSeek model (#6641)
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  7eb47b0f
- fix: force synchronization between TP workers when update_weights (#6626) · bc2e5645
  DangKai authored Jun 25, 2025
```
Co-authored-by: dangkai.dk <dangkai.dk@alibaba-inc.com>
```
  bc2e5645
- [ci] add router benchmark script and CI (#7498) · 3abc3036
  Simo Lin authored Jun 25, 2025
  
  3abc3036
- clean duplicate code (#7512) · afeed465
  linzhuo authored Jun 25, 2025
  
  afeed465
- EPLB support for MTP (#7510) · 587b4c6e
  yilian49 authored Jun 25, 2025
  
  587b4c6e
- [PD][Spec] Fix hidden state transfer for spec decode (#7516) · 7b9a174a
  Shangming Cai authored Jun 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  7b9a174a
- [OAI] patch origin request_id logic (#7508) · 03c039c4
  ybyang authored Jun 25, 2025
  
  03c039c4
- Fuse sorted_token_ids padding to moe_align_block_size kernel (#7437) · 57ab7769
  Ke Bao authored Jun 25, 2025
  
  57ab7769
- misc: Improvement to serving_chat.py and add more ut (#7489) · 112b496a
  Chang Su authored Jun 24, 2025
  
  112b496a