Commits · 2f79f58873b0bf38d8c64f5b6ac6fbb5ab50d7b4 · change / sglang · GitLab

27 Jan, 2025 13 commits
- feat: use sgl-kernel 0.0.3 in sglang (#3179) · 2f79f588
  Yineng Zhang authored Jan 27, 2025
  
  2f79f588
- chore: bump 0.0.3 for sgl-kernel (#3178) · 8a96f749
  Yineng Zhang authored Jan 27, 2025
```
Co-authored-by: ispobock <ispobaoke@hotmail.com>
Co-authored-by: BBuf <35585791+BBuf@users.noreply.github.com>
Co-authored-by: HandH1998 <007aabbcc411@gmail.com>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: ByronHsu <byronhsu1230@gmail.com>
```
  8a96f749
- cleanup sgl-kernel kernels (#3175) · 827aa873
  Yineng Zhang authored Jan 27, 2025
  
  827aa873
- Update thresholds in test_nightly_gsm8k_eval.py (#3176) · f8ca66fb
  Lianmin Zheng authored Jan 27, 2025
  
  f8ca66fb
- Improve weight loading and code style (#3174) · 53cef815
  Lianmin Zheng authored Jan 27, 2025
  
  53cef815
- add dsv3 mi300 triton config for block scale (#3146) · 351a72d4
  yigex authored Jan 27, 2025
  
  351a72d4
- [kernel] Fix position ids in rope (#3173) · 514f37c3
  Byron Hsu authored Jan 27, 2025
  
  514f37c3
- Add activation parameters to fused_moe (#3170) · 52c03f16
  Lianmin Zheng authored Jan 27, 2025
  
  52c03f16
- Bump sgl kernel to 0.0.2.post19 (#3167) · 741fccd7
  Byron Hsu authored Jan 26, 2025
  
  741fccd7
- add unit test for block wise fp8 (#3156) · 1e3e5215
  yizhang2077 authored Jan 27, 2025
  
  1e3e5215
- [kernel] Integrate flashinfer's rope with higher precision and better perf (#3134) · fb11a439
  Byron Hsu authored Jan 26, 2025
  
  fb11a439
- Add more logprob tests (#3162) · af02f99b
  Lianmin Zheng authored Jan 26, 2025
  
  af02f99b
- Doc: Add Docs about EAGLE speculative decoding (#3144) · 9472e699
  Jhin authored Jan 26, 2025
```
Co-authored-by: Chayenne <zhaochenyang@ucla.edu>
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  9472e699
26 Jan, 2025 19 commits
- [Docs]: Add function calling in index.rst (#3155) · 1acc1f56
  Chayenne authored Jan 26, 2025
  
  1acc1f56
- Feature/function calling update (#2700) · b045841b
  YAMY authored Jan 26, 2025
```
Co-authored-by: Mingyuan Ma <mamingyuan2001@berkeley.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  b045841b
- use self-hosted to build sgl-kernel (#3154) · f265d15b
  Yineng Zhang authored Jan 26, 2025
  
  f265d15b
- fix link in README (#3153) · 02431b9a
  Yineng Zhang authored Jan 26, 2025
  
  02431b9a
- Return more infos for computing average acceptance length (#3152) · 1dda8c5e
  Lianmin Zheng authored Jan 26, 2025
  
  1dda8c5e
- udpate sgl-kernel version for srt (#3150) · 7e097613
  Yineng Zhang authored Jan 26, 2025
  
  7e097613
- Temporarily skip the openai frontend tests (#3151) · f4a92f4b
  Lianmin Zheng authored Jan 26, 2025
  
  f4a92f4b
- chore: bump 0.0.2.post18 for sgl-kernel (#3149) · 318260c0
  Yineng Zhang authored Jan 26, 2025
  
  318260c0
- Do not load OPENAI_KEY from secrets (#3147) · 4a612531
  Lianmin Zheng authored Jan 26, 2025
  
  4a612531
- Add a test case for cached_tokens (#3145) · d1a08632
  Lianmin Zheng authored Jan 26, 2025
  
  d1a08632
- Add CPU affinity setting to latency benchmark (#3085) · f8b28e46
  Hubert Lu authored Jan 25, 2025
  
  f8b28e46
- support w8a8 fp8 kernel with CUTLASS (#3047) · 82392da8
  HandH1998 authored Jan 26, 2025
```
Co-authored-by: yych0745 <1398089567@qq.com>
```
  82392da8
- minor: cleanup sgl-kernel (#3143) · 95f789ad
  Yineng Zhang authored Jan 26, 2025
  
  95f789ad
- Fix repetition penalty (#3139) · 4f118a39
  Lianmin Zheng authored Jan 25, 2025
  
  4f118a39
- [Fix] Not skip NVML Check on AMD Platform (#3135) · 66283dbc
  yigex authored Jan 26, 2025
  
  66283dbc
- feat: cross python wheel for sgl-kernel (#3138) · 822bae8c
  Yineng Zhang authored Jan 26, 2025
  
  822bae8c
- enable kv_scale for Gemma2 (#3113) · 8e48ca8c
  Hui Liu authored Jan 25, 2025
  
  8e48ca8c
- Use torch.compile for scaling penalty (#3133) · 27acf63b
  Lianmin Zheng authored Jan 25, 2025
  
  27acf63b
- Fix CI tests (#3132) · da6f8081
  Lianmin Zheng authored Jan 25, 2025
  
  da6f8081
25 Jan, 2025 8 commits
- feat: refactor sgl-kernel and use TORCH_LIBRARY instead of PYBIND11_MODULE for custom ops (#3130) · 9286740e
  yinfan98 authored Jan 26, 2025
```
Co-authored-by: yinfan.1024 <yinfan.1024@bytedance.com>
Co-authored-by: yinfan98 <1106110035@qq.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  9286740e
- update installation doc for sgl-kernel (#3129) · 896c0744
  Yineng Zhang authored Jan 26, 2025
  
  896c0744
- Update whl index path (#3128) · c23d5706
  Ke Bao authored Jan 25, 2025
  
  c23d5706
- Update tag name for whl release (#3127) · 67ad4338
  Ke Bao authored Jan 25, 2025
  
  67ad4338
- speedup pr test for sgl-kernel (#3126) · 3cab5f71
  Yineng Zhang authored Jan 25, 2025
  
  3cab5f71
- chore: bump v0.0.2.post17 for sgl-kernel (#3125) · 14e754a8
  Yineng Zhang authored Jan 25, 2025
  
  14e754a8
- mirror fix for custom allreduce (#3124) · 98522149
  yizhang2077 authored Jan 25, 2025
  
  98522149
- support fp32 in sampling_scaling_penalties kernel (#3121) · 5d9d15e7
  Xiaoyu Zhang authored Jan 25, 2025
  
  5d9d15e7