Commits · 2c1a695ff111cadc200cf97d4c2cbfe95ebecb70 · change / sglang · GitLab

04 Feb, 2025 3 commits
- ROCm: sgl-kernel enablement starting with sgl_moe_align_block (#3287) · 2c1a695f
  HAI authored Feb 04, 2025
  
  2c1a695f
- upgrade flashinfer v0.2.0.post2 (#3288) · d39899e8
  Yineng Zhang authored Feb 04, 2025
```
Co-authored-by: pankajroark <pankajroark@users.noreply.github.com>
```
  d39899e8
- [Feature] Define backends and add Triton backend for Lora (#3161) · 70817a7e
  Baizhou Zhang authored Feb 03, 2025
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  70817a7e
03 Feb, 2025 2 commits
- adding Triton configs for DeepSeekV3 on Blackwell (#3272) · d54cee14
  kushanam authored Feb 03, 2025
  
  d54cee14
- refactor EAGLE 2 (#3269) · 013021b6
  Yineng Zhang authored Feb 03, 2025
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: merrymercy <lianminzheng@gmail.com>
Co-authored-by: Ying1123 <sqy1415@gmail.com>
```
  013021b6
02 Feb, 2025 3 commits
- Bug: Fix min_p sampling crash when using flashinfer backend (#3207) · 28b0a62b
  zifeitong authored Feb 02, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  28b0a62b
- ROCm: bump 6.3.0 (#3259) · 566d61d9
  HAI authored Feb 02, 2025
  
  566d61d9
- Tune paged attention parameters for AMD GPU. (#3255) · d9eb9358
  Wen-Heng (Jack) Chung authored Feb 01, 2025
  
  d9eb9358
01 Feb, 2025 3 commits
- use srt VocabParallelEmbedding (#3252) · 959dca4f
  Yineng Zhang authored Feb 01, 2025
  
  959dca4f
- support QuickGELU (#3250) · 8db776f0
  Yineng Zhang authored Feb 01, 2025
  
  8db776f0
- update and simplify CustomOp (#3249) · 4eb4b401
  Yineng Zhang authored Feb 01, 2025
  
  4eb4b401
31 Jan, 2025 6 commits
- update sgl-kernel version for sglang (#3238) · 34e405e0
  Yineng Zhang authored Feb 01, 2025
  
  34e405e0
- Optimize MoE topk with torch compile (#3236) · 1ebe1d6d
  Ke Bao authored Feb 01, 2025
  
  1ebe1d6d
- compatible with flashinfer v0.2 (#3235) · 7811bfda
  Yineng Zhang authored Feb 01, 2025
  
  7811bfda
- chore: bump v0.4.2.post1 (#3233) · cf0f7eaf
  Yineng Zhang authored Jan 31, 2025
  
  cf0f7eaf
- Fix block wise fp8 torch compile (#3232) · c02e3139
  Ke Bao authored Jan 31, 2025
  
  c02e3139
- [fix] Clamp logprob with dtype min to prevent `-inf` (#3224) · 734daedd
  Byron Hsu authored Jan 31, 2025
  
  734daedd
28 Jan, 2025 3 commits
- [Fix] Address remaining issues of supporting MiniCPMV (#2977) · 9f635ea5
  Mick authored Jan 28, 2025
  
  9f635ea5
- [kernel] Use sgl_kernel rope (#3169) · 988d0a4b
  Byron Hsu authored Jan 27, 2025
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  988d0a4b
- Docs fix about EAGLE and streaming output (#3166) · 7b9b4f44
  Jhin authored Jan 27, 2025
```
Co-authored-by: Chayenne <zhaochenyang@ucla.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Jhin <jhinpan@umich.edu>
```
  7b9b4f44
27 Jan, 2025 6 commits
- Sanity check to prevent performance regression (#3171) · 08104b56
  Zhiqiang Xie authored Jan 27, 2025
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  08104b56
- chore: bump v0.4.2 (#3180) · 4ab43cfb
  Yineng Zhang authored Jan 27, 2025
  
  4ab43cfb
- feat: use sgl-kernel 0.0.3 in sglang (#3179) · 2f79f588
  Yineng Zhang authored Jan 27, 2025
  
  2f79f588
- Improve weight loading and code style (#3174) · 53cef815
  Lianmin Zheng authored Jan 27, 2025
  
  53cef815
- add dsv3 mi300 triton config for block scale (#3146) · 351a72d4
  yigex authored Jan 27, 2025
  
  351a72d4
- Add activation parameters to fused_moe (#3170) · 52c03f16
  Lianmin Zheng authored Jan 27, 2025
  
  52c03f16
26 Jan, 2025 9 commits
- Feature/function calling update (#2700) · b045841b
  YAMY authored Jan 26, 2025
```
Co-authored-by: Mingyuan Ma <mamingyuan2001@berkeley.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  b045841b
- Return more infos for computing average acceptance length (#3152) · 1dda8c5e
  Lianmin Zheng authored Jan 26, 2025
  
  1dda8c5e
- udpate sgl-kernel version for srt (#3150) · 7e097613
  Yineng Zhang authored Jan 26, 2025
  
  7e097613
- Add a test case for cached_tokens (#3145) · d1a08632
  Lianmin Zheng authored Jan 26, 2025
  
  d1a08632
- Add CPU affinity setting to latency benchmark (#3085) · f8b28e46
  Hubert Lu authored Jan 25, 2025
  
  f8b28e46
- Fix repetition penalty (#3139) · 4f118a39
  Lianmin Zheng authored Jan 25, 2025
  
  4f118a39
- [Fix] Not skip NVML Check on AMD Platform (#3135) · 66283dbc
  yigex authored Jan 26, 2025
  
  66283dbc
- enable kv_scale for Gemma2 (#3113) · 8e48ca8c
  Hui Liu authored Jan 25, 2025
  
  8e48ca8c
- Use torch.compile for scaling penalty (#3133) · 27acf63b
  Lianmin Zheng authored Jan 25, 2025
  
  27acf63b
23 Jan, 2025 3 commits
- Revert "disable custom allreduce on HIP" (#3067) · ea535dc5
  Lianmin Zheng authored Jan 22, 2025
  
  ea535dc5
- Support loading of larger models with on-the-fly quantization (#3061) · 862bcff8
  Ke Wen authored Jan 22, 2025
  
  862bcff8
- Fix tp token sync for dp attention (#3062) · 8b84e69f
  Lianmin Zheng authored Jan 22, 2025
  
  8b84e69f
22 Jan, 2025 2 commits
- Add some flags to allow sync token ids across TP ranks (#3060) · 022614d2
  Lianmin Zheng authored Jan 22, 2025
  
  022614d2
- Fix the FP8 E4M3 parsing offline scales failure bug (#3045) · b8ab989f
  lukec authored Jan 23, 2025
  
  b8ab989f