Commits · 49420741746c8f3e80e0eb17e7d012bfaf25793a · change / sglang · GitLab

19 Mar, 2025 8 commits
- [fix] fix initialization of _ENABLE_TORCH_INFERENCE_MODE (#4549) · 49420741
  JieXin Liang authored Mar 20, 2025
  
  49420741
- Add clang-format to pre-commit config (#4583) · ba52fd18
  Hongbo Xu authored Mar 20, 2025
  
  ba52fd18
- Support FlashMLA backend cuda graph (#4514) · b6944f97
  lukec authored Mar 19, 2025
```
Co-authored-by: yinfan98 <1106310035@qq.com>
Co-authored-by: Hongbosherlock <hongbosherlock@gmail.com>
Co-authored-by: ispobock <ispobaoke@163.com>
```
  b6944f97
- [Feature] Integrate DeepEP into SGLang (#4232) · f44db16c
  Jinyan Chen authored Mar 19, 2025
```
Co-authored-by: Cheng Wan <cwan39@gatech.edu>
Co-authored-by: Xuting Zhou <xutingz@nvidia.com>
```
  f44db16c
- Create col-major and tma-aligned x_scale for deep_gemm.gemm_fp8_fp8_bf16_nt (#4515) · f9c53cbb
  strgrb authored Mar 19, 2025
```
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
```
  f9c53cbb
- [Fix] Fix raw_bs bug when using flashinfer mla and eagle (#4557) · 90532b76
  Baizhou Zhang authored Mar 18, 2025
  
  90532b76
- Optimize Triton decoding kernel for dynamic workload (#4553) · c0e9a36c
  JieXin Liang authored Mar 19, 2025
  
  c0e9a36c
- [Feature] Support Tensor Parallelism and Weight Slicing for Lora (#4274) · 588865f0
  aoshen524 authored Mar 18, 2025
```
Co-authored-by: ShenAo1111 <1377693092@qq.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
```
  588865f0
18 Mar, 2025 12 commits
- Reduce computation and communication in DP attention (#4521) · 3196999f
  Cheng Wan authored Mar 18, 2025
  
  3196999f
- [Feature] Support EAGLE 3 (#4247) · 9e0186f3
  James Liu authored Mar 18, 2025
  
  9e0186f3
- [Fix] Type annotation correction for UpdateWeightsFromTensorReqInput (#4532) · 8baf9a0c
  Wei Wu authored Mar 18, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  8baf9a0c
- use sgl custom all reduce (#4441) · c7872985
  Yineng Zhang authored Mar 18, 2025
  
  c7872985
- Add deepseek v2 torch compile pr test (#4538) · 45212ce1
  Ke Bao authored Mar 18, 2025
  
  45212ce1
- cleanup deps 3/n (#4541) · c16b33cc
  Yineng Zhang authored Mar 18, 2025
  
  c16b33cc
- Fix the incorrect args in benchmark_and_profiling.md (#4542) · 2d004512
  Albert authored Mar 18, 2025
```
Signed-off-by: Tianyu Zhou <albert.zty@antgroup.com>
```
  2d004512
- remove useless backend forward in rotary_embedding (#4500) · 804d250a
  Xiaoyu Zhang authored Mar 18, 2025
  
  804d250a
- [Hotfix] solve fp8 w8a8 ci test fail (#4531) · dd865bef
  Xiaoyu Zhang authored Mar 18, 2025
  
  dd865bef
- fix: second_per_grid_ts should be used to get mrope position (#3682) · d373a48c
  Mick authored Mar 18, 2025
  
  d373a48c
- refactor: rewrite bench-mmmu-sglang (#4458) · 98be3bd3
  Mick authored Mar 18, 2025
  
  98be3bd3
- Unit test for Hierarchical Caching (#4486) · a98290ae
  Zhiqiang Xie authored Mar 17, 2025
  
  a98290ae
17 Mar, 2025 20 commits
- sglang quant module remove vllm dependency (#4507) · 9b81f9bd
  Xiaoyu Zhang authored Mar 18, 2025
  
  9b81f9bd
- upgrade sgl-kernel 0.0.5.post3 (#4522) · f81a27f6
  Yineng Zhang authored Mar 17, 2025
  
  f81a27f6
- bump v0.0.5.post3 (#4520) · 988ab646
  Yineng Zhang authored Mar 17, 2025
  
  988ab646
- Revert "feat: update grouped_topk to support softmax and sigmoid" (#4505) · 3ded4b21
  Ke Bao authored Mar 18, 2025
  
  3ded4b21
- [sgl-router] improvement to avoid hang (#4482) · f4d7ab7a
  Yinghai Lu authored Mar 17, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
```
  f4d7ab7a
- Update readme (#4517) · c38ca4fc
  Lianmin Zheng authored Mar 17, 2025
  
  c38ca4fc
- Remove redundant type conversion (#4513) · 82dec1f7
  Lianmin Zheng authored Mar 17, 2025
  
  82dec1f7
- [ROCm] fix dtype (#4510) · 5f9b2c62
  yiakwy-xpu-ml-framework-team authored Mar 17, 2025
  
  5f9b2c62
- Fix data parallel + tensor parallel (#4499) · 5493c334
  Lianmin Zheng authored Mar 17, 2025
  
  5493c334
- [Doc] add doc for quantization w8a8_fp8 or w8a8_int8 (#4495) · f2ab37e5
  HandH1998 authored Mar 17, 2025
  
  f2ab37e5
- [Fix] Resolve GPU Memory Leak in update_weights_from_tensor (#4446) · 91ba98fe
  Wei Wu authored Mar 17, 2025
  
  91ba98fe
- Nicer standalone engine inferface (#4480) · c614dbdf
  Yinghai Lu authored Mar 17, 2025
  
  c614dbdf
- Constraint Decoding: Tool call with text (#4067) · 927ca935
  Xihuai Wang authored Mar 17, 2025
  
  927ca935
- Support Online Quantization for W8A8 (#4485) · ef3c2dd0
  Stefan He authored Mar 17, 2025
  
  ef3c2dd0
- Support serving DeepSeek-R1-Channel-INT8 with 32 L40S. (#4418) · 75b65648
  Wenbo Yang authored Mar 17, 2025
  
  75b65648
- config: Update fused moe config (#4493) · 0f52fb55
  Mick authored Mar 17, 2025
  
  0f52fb55
- [Feature] Support Deepseek-VL2 (#2798) · d6d21640
  萝卜菜 authored Mar 17, 2025
```
Co-authored-by: Edenzzzz <wtan45@wisc.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yi Zhang <1109276519@qq.com>
```
  d6d21640
- [Fix] use `torch.inference_mode()` instead of `torch.no_grad()` (#4372) · 0212d2e2
  JieXin Liang authored Mar 17, 2025
  
  0212d2e2
- Fix router test (#4483) · 8cc300f5
  Byron Hsu authored Mar 16, 2025
  
  8cc300f5
- Constraint Decoding: Set xgrammar as the default grammar backend (#4386) · 452db508
  mlmz authored Mar 17, 2025
  
  452db508