Commits · 55f5fc68acc84027a275649d2cfb320448ed95d7 · change / sglang · GitLab

02 Feb, 2025 1 commit
- Tune paged attention parameters for AMD GPU. (#3255) · d9eb9358
  Wen-Heng (Jack) Chung authored Feb 01, 2025
  
  d9eb9358
01 Feb, 2025 3 commits
- use srt VocabParallelEmbedding (#3252) · 959dca4f
  Yineng Zhang authored Feb 01, 2025
  
  959dca4f
- support QuickGELU (#3250) · 8db776f0
  Yineng Zhang authored Feb 01, 2025
  
  8db776f0
- update and simplify CustomOp (#3249) · 4eb4b401
  Yineng Zhang authored Feb 01, 2025
  
  4eb4b401
31 Jan, 2025 6 commits
- update sgl-kernel version for sglang (#3238) · 34e405e0
  Yineng Zhang authored Feb 01, 2025
  
  34e405e0
- Optimize MoE topk with torch compile (#3236) · 1ebe1d6d
  Ke Bao authored Feb 01, 2025
  
  1ebe1d6d
- compatible with flashinfer v0.2 (#3235) · 7811bfda
  Yineng Zhang authored Feb 01, 2025
  
  7811bfda
- chore: bump v0.4.2.post1 (#3233) · cf0f7eaf
  Yineng Zhang authored Jan 31, 2025
  
  cf0f7eaf
- Fix block wise fp8 torch compile (#3232) · c02e3139
  Ke Bao authored Jan 31, 2025
  
  c02e3139
- [fix] Clamp logprob with dtype min to prevent `-inf` (#3224) · 734daedd
  Byron Hsu authored Jan 31, 2025
  
  734daedd
28 Jan, 2025 3 commits
- [Fix] Address remaining issues of supporting MiniCPMV (#2977) · 9f635ea5
  Mick authored Jan 28, 2025
  
  9f635ea5
- [kernel] Use sgl_kernel rope (#3169) · 988d0a4b
  Byron Hsu authored Jan 27, 2025
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  988d0a4b
- Docs fix about EAGLE and streaming output (#3166) · 7b9b4f44
  Jhin authored Jan 27, 2025
```
Co-authored-by: Chayenne <zhaochenyang@ucla.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Jhin <jhinpan@umich.edu>
```
  7b9b4f44
27 Jan, 2025 6 commits
- Sanity check to prevent performance regression (#3171) · 08104b56
  Zhiqiang Xie authored Jan 27, 2025
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
```
  08104b56
- chore: bump v0.4.2 (#3180) · 4ab43cfb
  Yineng Zhang authored Jan 27, 2025
  
  4ab43cfb
- feat: use sgl-kernel 0.0.3 in sglang (#3179) · 2f79f588
  Yineng Zhang authored Jan 27, 2025
  
  2f79f588
- Improve weight loading and code style (#3174) · 53cef815
  Lianmin Zheng authored Jan 27, 2025
  
  53cef815
- add dsv3 mi300 triton config for block scale (#3146) · 351a72d4
  yigex authored Jan 27, 2025
  
  351a72d4
- Add activation parameters to fused_moe (#3170) · 52c03f16
  Lianmin Zheng authored Jan 27, 2025
  
  52c03f16
26 Jan, 2025 9 commits
- Feature/function calling update (#2700) · b045841b
  YAMY authored Jan 26, 2025
```
Co-authored-by: Mingyuan Ma <mamingyuan2001@berkeley.edu>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: shuaills <shishuaiuoe@gmail.com>
```
  b045841b
- Return more infos for computing average acceptance length (#3152) · 1dda8c5e
  Lianmin Zheng authored Jan 26, 2025
  
  1dda8c5e
- udpate sgl-kernel version for srt (#3150) · 7e097613
  Yineng Zhang authored Jan 26, 2025
  
  7e097613
- Add a test case for cached_tokens (#3145) · d1a08632
  Lianmin Zheng authored Jan 26, 2025
  
  d1a08632
- Add CPU affinity setting to latency benchmark (#3085) · f8b28e46
  Hubert Lu authored Jan 25, 2025
  
  f8b28e46
- Fix repetition penalty (#3139) · 4f118a39
  Lianmin Zheng authored Jan 25, 2025
  
  4f118a39
- [Fix] Not skip NVML Check on AMD Platform (#3135) · 66283dbc
  yigex authored Jan 26, 2025
  
  66283dbc
- enable kv_scale for Gemma2 (#3113) · 8e48ca8c
  Hui Liu authored Jan 25, 2025
  
  8e48ca8c
- Use torch.compile for scaling penalty (#3133) · 27acf63b
  Lianmin Zheng authored Jan 25, 2025
  
  27acf63b
23 Jan, 2025 3 commits
- Revert "disable custom allreduce on HIP" (#3067) · ea535dc5
  Lianmin Zheng authored Jan 22, 2025
  
  ea535dc5
- Support loading of larger models with on-the-fly quantization (#3061) · 862bcff8
  Ke Wen authored Jan 22, 2025
  
  862bcff8
- Fix tp token sync for dp attention (#3062) · 8b84e69f
  Lianmin Zheng authored Jan 22, 2025
  
  8b84e69f
22 Jan, 2025 5 commits
- Add some flags to allow sync token ids across TP ranks (#3060) · 022614d2
  Lianmin Zheng authored Jan 22, 2025
  
  022614d2
- Fix the FP8 E4M3 parsing offline scales failure bug (#3045) · b8ab989f
  lukec authored Jan 23, 2025
  
  b8ab989f
- disable custom allreduce on HIP (#3058) · ddc2001f
  Hui Liu authored Jan 22, 2025
  
  ddc2001f
- fix rotary_embedding rope_scaling for phi (#3055) · 0d2148ef
  nstream-ai-devx authored Jan 22, 2025
  
  0d2148ef
- Use int64 as indices for set_kv_buffer (#3039) · 3d8f1c9b
  Lianmin Zheng authored Jan 21, 2025
  
  3d8f1c9b
21 Jan, 2025 4 commits
- Add accuracy and latency tests of eagle into CI (#3027) · a4331cd2
  Lianmin Zheng authored Jan 21, 2025
  
  a4331cd2
- Misc fixes for eagle (flush_cache, CPU overhead) (#3014) · 287d07a6
  Lianmin Zheng authored Jan 20, 2025
  
  287d07a6
- Enable Cohere2 Models (#3018) · d2571dd5
  Hui Liu authored Jan 20, 2025
  
  d2571dd5
- [EAGLE] Fix some boundary situation when retract reqs and req's max token = 1 (#2939) · b730aa6b
  996_icu authored Jan 21, 2025
```
Co-authored-by: josephyou <josephyou@tencent.com>
```
  b730aa6b