Commits · c998d04b46920f06d945fbef9023884a768723fc · change / sglang

24 Apr, 2025 1 commit
- vlm: enable radix cache for qwen-vl models (#5349) · c998d04b
  Mick authored Apr 24, 2025
```
Co-authored-by: Xinyuan Tong <justinning0323@outlook.com>
```
  c998d04b
23 Apr, 2025 2 commits
- tune the threshold of gemma-2-27b-it in test_nightly_gsm8k_eval.py (#5677) · de071366
  Lianmin Zheng authored Apr 23, 2025
  
  de071366
- fix gemma3 unit test (#5670) · 8777a1d2
  Yineng Zhang authored Apr 23, 2025
  
  8777a1d2
21 Apr, 2025 5 commits
- upstream hicache fixes (#5570) · 70645f4d
  Zhiqiang Xie authored Apr 20, 2025
  
  70645f4d
- Add Speculative Decoding Eagle3 topk > 1 (#5318) · 188f0955
  Qingquan Song authored Apr 20, 2025
```
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
```
  188f0955
- Feat: Implement JSON Mode (response_format.type="json_object") (#4733) · 9f3bd2ad
  kyle-pena-kuzco authored Apr 20, 2025
```
Co-authored-by: Kyle Pena <kylepena@kyles-macbook-pro.turkey-marlin.ts.net>
```
  9f3bd2ad
- [Feature] Prefill assistant response - add continue_final_message parameter (#4226) · 8b39274e
  Adarsh Shirawalmath authored Apr 21, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  8b39274e
- Add test config yamls for Deepseek v3 (#5433) · 5156d5a4
  Baizhou Zhang authored Apr 20, 2025
  
  5156d5a4
19 Apr, 2025 1 commit
- restruct compressed_tensors_w8a8_fp8 (#5475) · bf86c5e9
  Xiaoyu Zhang authored Apr 19, 2025
  
  bf86c5e9
17 Apr, 2025 2 commits

Feat/support encoder model (like bert) (#4887) · 3bface15
woodx authored Apr 17, 2025

3bface15

[Model Support] unsloth/Phi-4-mini bnb model (#4982) · 8f783c19

eigen authored Apr 16, 2025


Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>

8f783c19

16 Apr, 2025 4 commits
- Clean up imports (#5467) · 177320a5
  Lianmin Zheng authored Apr 16, 2025
  
  177320a5
- Support MHA with chunked prefix cache for DeepSeek chunked prefill (#5113) · a42736bb
  Baizhou Zhang authored Apr 15, 2025
  
  a42736bb
- Support BNB quantization for llama/mllama (#5038) · bc24205b
  ryang authored Apr 16, 2025
```
Co-authored-by: Yuhao Yang <yyh073@foxmail.com>
```
  bc24205b
- Fix ignore_eos parameter when loading a chat template (#5264) · 27a009bb
  Chang Su authored Apr 15, 2025
  
  27a009bb
15 Apr, 2025 2 commits
- [misc] fix ci flaky case (#5352) · f88f7e19
  JieXin Liang authored Apr 15, 2025
  
  f88f7e19
- fix: update test config (#5392) · ac5b78ba
  Yineng Zhang authored Apr 14, 2025
  
  ac5b78ba
14 Apr, 2025 2 commits
- Support for Qwen2.5-VL Model in bitsandbytes Format (#5003) · 072df753
  yhyang201 authored Apr 14, 2025
  
  072df753
- Fix DeepSeek DP Attention + torch compile (#5367) · defede50
  fzyzcjy authored Apr 14, 2025
```
Co-authored-by: ispobock <ispobaoke@163.com>
```
  defede50
13 Apr, 2025 1 commit
- feat: update experiment_runner (#5360) · 39d90449
  Yineng Zhang authored Apr 13, 2025
  
  39d90449
12 Apr, 2025 2 commits
- Support server based rollout in Verlengine (#4848) · bc92107b
  tianlian yi authored Apr 13, 2025
```
Co-authored-by: Jin Pan <jpan236@wisc.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Jinn <47354855+jhinpan@users.noreply.github.com>
```
  bc92107b
- Adjust ci test threshold (#5271) · 5ad05719
  Ke Bao authored Apr 12, 2025
  
  5ad05719
10 Apr, 2025 1 commit
- Update deps for mllama4 (#5215) · 1078396f
  Ke Bao authored Apr 11, 2025
  
  1078396f
09 Apr, 2025 5 commits
- update grok test (#5171) · 7f875f12
  saienduri authored Apr 09, 2025
  
  7f875f12
- model: support mllama4 (#5144) · fbebcb7a
  Mick authored Apr 10, 2025
  
  fbebcb7a
- [ci] fix ci test fused_moe op (#5102) · 87eddedf
  Xiaoyu Zhang authored Apr 09, 2025
  
  87eddedf
- Support Llama4 fp8 inference (#5194) · 40652482
  HandH1998 authored Apr 09, 2025
```
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
Co-authored-by: zhyncs <me@zhyncs.com>
```
  40652482
- Tiny disable model that does not work (#5175) · 39efad4f
  fzyzcjy authored Apr 09, 2025
  
  39efad4f
08 Apr, 2025 4 commits
- [feat&refactor] Enhance multimodal input support with refactor io_struct (#4938) · d09a51f1
  XinyuanTong authored Apr 08, 2025
```
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
```
  d09a51f1
- Use public model for FA3 speculative decode testing (#5152) · fd5a55cf
  Yubo Wang authored Apr 08, 2025
  
  fd5a55cf
- Add unit test on page_size > 1 and mla and integration test for Flash Attention 3 (#4760) · 804d9f2e
  Yubo Wang authored Apr 07, 2025
  
  804d9f2e
- [modelopt] automatically inspect if model is ModelOpt quantized and set quantization method (#5145) · 9731eca7
  Yun Dai authored Apr 07, 2025
  
  9731eca7
05 Apr, 2025 1 commit
- [Revision] Replace enable_flashinfer_mla argument with attention_backend (#5052) · efbae697
  Baizhou Zhang authored Apr 05, 2025
  
  efbae697
04 Apr, 2025 1 commit

[2/3] fix dsv3 awq issue (#4625) · d95269f9

AniZpZ authored Apr 04, 2025


Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
Co-authored-by: laixinn <xielx@shanghaitech.edu.cn>

d95269f9

03 Apr, 2025 2 commits
- Revert "Replace enable_flashinfer_mla argument with attention_backend" (#5048) · 74885a84
  Lianmin Zheng authored Apr 03, 2025
  
  74885a84
- Replace enable_flashinfer_mla argument with attention_backend (#5005) · e8999b13
  Baizhou Zhang authored Apr 03, 2025
  
  e8999b13
02 Apr, 2025 1 commit
- Large page size aligned hierarchical caching (#4581) · e119f042
  Zhiqiang Xie authored Apr 01, 2025
  
  e119f042
31 Mar, 2025 3 commits
- refactor: multimodal data (#4754) · 5cb552b1
  Mick authored Apr 01, 2025
  
  5cb552b1
- Fix oom error for large page size (#4913) · a169b9f8
  Zhiqiang Xie authored Mar 30, 2025
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  a169b9f8
- [Fix] Improve Lora tests and reduce CI runtime (#4925) · 42873eac
  Baizhou Zhang authored Mar 30, 2025
  
  42873eac