Commits · ef3c2dd08efe0b866847d6b671ee1104373ace32 · change / sglang · GitLab

17 Mar, 2025 16 commits
- Support Online Quantization for W8A8 (#4485) · ef3c2dd0
  Stefan He authored Mar 17, 2025
  
  ef3c2dd0
- Support serving DeepSeek-R1-Channel-INT8 with 32 L40S. (#4418) · 75b65648
  Wenbo Yang authored Mar 17, 2025
  
  75b65648
- config: Update fused moe config (#4493) · 0f52fb55
  Mick authored Mar 17, 2025
  
  0f52fb55
- [Feature] Support Deepseek-VL2 (#2798) · d6d21640
  萝卜菜 authored Mar 17, 2025
```
Co-authored-by: Edenzzzz <wtan45@wisc.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>
```
  d6d21640
- [Fix] use `torch.inference_mode()` instead of `torch.no_grad()` (#4372) · 0212d2e2
  JieXin Liang authored Mar 17, 2025
  
  0212d2e2
- Fix router test (#4483) · 8cc300f5
  Byron Hsu authored Mar 16, 2025
  
  8cc300f5
- Constraint Decoding: Set xgrammar as the default grammar backend (#4386) · 452db508
  mlmz authored Mar 17, 2025
  
  452db508
- Add endpoint for file support, purely to speed up processing of input_embeds. (#2797) · d1112d85
  Rin Intachuen authored Mar 17, 2025
  
  d1112d85
- Feature: support code completion (#3612) · 48efec7b
  woodx authored Mar 17, 2025
  
  48efec7b
- [ROCm] enable moe topk softmax in amd (#4448) · 9b8333d9
  yiakwy-xpu-ml-framework-team authored Mar 17, 2025
  
  9b8333d9
- Fix: Complete int32 to int64 conversion (#4465) · f5bbf603
  Zhiqiang Xie authored Mar 16, 2025
  
  f5bbf603
- Fix: modelscope env comment (#4474) · 5cbd709e
  huiwq1990 authored Mar 17, 2025
```
Signed-off-by: huiwq1990 <huiwq1990@163.com>
```
  5cbd709e
- Initialize image processor for skip-tokenizer-init codepath (#4479) · 2e4a1e2d
  Yinghai Lu authored Mar 16, 2025
```
Co-authored-by: Alex Kirillov <alex@iterationlab.org>
```
  2e4a1e2d
- Urgent model support: support gemma-3-it (#4424) · 9d02bb3e
  Mick authored Mar 17, 2025
  
  9d02bb3e
- Benchmark: Statistical Analysis of the Output Stability of the Deepseek Model (#4202) · 402db5c5
  ZelinTan authored Mar 17, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  402db5c5
- Update CODEOWNERS (#4484) · 754a0e82
  Lianmin Zheng authored Mar 16, 2025
  
  754a0e82
16 Mar, 2025 17 commits
- 400 on empty input_ids (#4481) · 799fb5f4
  Yinghai Lu authored Mar 16, 2025
  
  799fb5f4
- fix custom allreduce performance/accuracy problem (#4477) · 25e1816e
  Yi Zhang authored Mar 17, 2025
  
  25e1816e
- Support FlashMLA backend (#4472) · a53fe428
  lukec authored Mar 17, 2025
```
Co-authored-by: yinfan98 <1106310035@qq.com>
```
  a53fe428
- [Eagle] Remove the greedy branch and some redundant code (#4363) · 1b859295
  Ying Sheng authored Mar 16, 2025
```
Co-authored-by: Sehoon Kim <sehoon@x.ai>
```
  1b859295
- Revert "feat: Add FlashMLA submodule (#4449)" (#4470) · 9971dc22
  Yineng Zhang authored Mar 16, 2025
  
  9971dc22
- Release sgl-kernel v0.0.5.post2 (#4469) · 3db35c1a
  Lianmin Zheng authored Mar 16, 2025
  
  3db35c1a
- Add greedy verification kernel (#4383) · 52a34d74
  Ying Sheng authored Mar 16, 2025
  
  52a34d74
- Remove filter for pr-tests (#4468) · 06d12b39
  Lianmin Zheng authored Mar 16, 2025
  
  06d12b39
- Fix finish step for pr tests and notebook tests (#4467) · c30976fb
  Lianmin Zheng authored Mar 16, 2025
  
  c30976fb
- [Fix] use `torch.cat` instead of `torch.concat` to prevent entering the... · 1a3fa75f
  JieXin Liang authored Mar 16, 2025
```
[Fix] use `torch.cat` instead of `torch.concat` to prevent entering the `Autograd` backends. (#4466)
```
  1a3fa75f
- feat: Add FlashMLA submodule (#4449) · 81f431ed
  Shi Shuai authored Mar 16, 2025
```
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
```
  81f431ed
- cleanup deps 2/n (#4464) · 65b7c9b7
  Yineng Zhang authored Mar 15, 2025
  
  65b7c9b7
- Fix minor style (#4460) · 2c4f5cca
  Lianmin Zheng authored Mar 15, 2025
  
  2c4f5cca
- Fix typos (#4368) · 15843047
  Wang Ran (汪然) authored Mar 16, 2025
  
  15843047
- perf: update fused moe config (#4459) · 8ec2ce07
  Mick authored Mar 16, 2025
  
  8ec2ce07
- Update comment in qwen2.py (#4447) · 1fd0cf8a
  Michael Feil authored Mar 15, 2025
  
  1fd0cf8a
- Auto-detect device if not specified in server arguments. (#4423) · bf63ee54
  vikram singh shekhawat authored Mar 16, 2025
  
  bf63ee54
15 Mar, 2025 7 commits
- typos: Update sampling_params.md (#4391) · 22c96f78
  Wang Ran (汪然) authored Mar 16, 2025
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  22c96f78
- bugfix: Update sampling_params.py (#4413) · 2892b9bb
  Wang Ran (汪然) authored Mar 16, 2025
  
  2892b9bb
- Update bench_serving.py (#4454) · 470b4740
  Xu Song authored Mar 16, 2025
  
  470b4740
- docs: Add Llama 3.3 to supported models (#4453) · 26c372c1
  江家瑋 authored Mar 16, 2025
```
Signed-off-by: JiangJiaWei1103 <waynechuang97@gmail.com>
```
  26c372c1
- [Fix] Fix errors when using the device except cuda. (#4455) · 86d9baed
  Chen Shengzhi authored Mar 16, 2025
  
  86d9baed
- Fix test_create_kvindices unit test (#4452) · 21d485f8
  lukec authored Mar 16, 2025
  
  21d485f8
- ci: update transformers==4.48.3 (#4451) · 035ac2ab
  Mick authored Mar 16, 2025
  
  035ac2ab