Commits · 8d8ef8497ebb5b98b7bfd6f6ce4e20baa2bda976 · change / sglang · GitLab

24 Jan, 2025 3 commits
- bump router to 0.1.4 (#3094) · 8d8ef849
  Byron Hsu authored Jan 23, 2025
  
  8d8ef849
- [router] Forward all request headers from router to workers (#3070) · 9a0cc2e9
  Byron Hsu authored Jan 23, 2025
  
  9a0cc2e9
- Add shapes for int8 gemm benchmark (#3093) · 7bad7e75
  Ke Bao authored Jan 24, 2025
  
  7bad7e75
23 Jan, 2025 19 commits
- Docs: Update doc for server arguments (#2742) · 1c4e0d24
  simveit authored Jan 23, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  1c4e0d24
- chore: bump sgl-kernel 0.0.2.post16 (#3087) · 54bac8af
  Yineng Zhang authored Jan 24, 2025
  
  54bac8af
- feat: integrate sampling kernels into sgl-kernel (#3086) · 5de4051b
  Yineng Zhang authored Jan 24, 2025
```
Co-authored-by: Zihao Ye <expye@outlook.com>
```
  5de4051b
- [hotfix] fix test_sampling_scaling_penalties.py ci test (#3084) · e0cd65c2
  Xiaoyu Zhang authored Jan 24, 2025
  
  e0cd65c2
- use flashinfer vec_dtypes in sgl_kernel (#3083) · f1b68618
  Xiaoyu Zhang authored Jan 23, 2025
  
  f1b68618
- sync flashinfer and update sgl-kernel tests (#3081) · 0da0989a
  Yineng Zhang authored Jan 23, 2025
  
  0da0989a
- use env variable to control the build conf on the CPU build node (#3080) · 07a22cbb
  Yineng Zhang authored Jan 23, 2025
  
  07a22cbb
- update version setup for sgl-kernel (#3079) · 3d0bfa3e
  Yineng Zhang authored Jan 23, 2025
  
  3d0bfa3e
- fix build error for sgl-kernel (#3078) · 1f6cf0d4
  Yineng Zhang authored Jan 23, 2025
  
  1f6cf0d4
- Remove torch dependency in sgl-kernel (#3074) · 553f5a3f
  Lianmin Zheng authored Jan 23, 2025
  
  553f5a3f
- support lightning_attention_decode in sgl-kernel for MiniMax-Text-01 (#3030) · ac2dc35d
  Xiaoyu Zhang authored Jan 23, 2025
  
  ac2dc35d
- use v0.6.4.post1 for sgl-kernel ci (#3071) · 3e032c07
  Yineng Zhang authored Jan 23, 2025
  
  3e032c07
- docs: update developer guide for sgl-kernel (#3069) · 44e12ce4
  Yineng Zhang authored Jan 23, 2025
  
  44e12ce4
- docs: add developer guide for sgl-kernel (#3068) · a547aad6
  Yineng Zhang authored Jan 23, 2025
  
  a547aad6
- Revert "disable custom allreduce on HIP" (#3067) · ea535dc5
  Lianmin Zheng authored Jan 22, 2025
  
  ea535dc5
- Support loading of larger models with on-the-fly quantization (#3061) · 862bcff8
  Ke Wen authored Jan 22, 2025
  
  862bcff8
- Fix tp token sync for dp attention (#3062) · 8b84e69f
  Lianmin Zheng authored Jan 22, 2025
  
  8b84e69f
- [router] make error actionable (#3063) · 5de50653
  Byron Hsu authored Jan 22, 2025
  
  5de50653
- [devcontainer] add non-root user (#2989) · c0bf9bf1
  Byron Hsu authored Jan 22, 2025
  
  c0bf9bf1
22 Jan, 2025 14 commits
- Add some flags to allow sync token ids across TP ranks (#3060) · 022614d2
  Lianmin Zheng authored Jan 22, 2025
  
  022614d2
- Fix the FP8 E4M3 parsing offline scales failure bug (#3045) · b8ab989f
  lukec authored Jan 23, 2025
  
  b8ab989f
- [Doc] Update doc of profiling with PyTorch Profiler (#3038) · b3393e94
  Baizhou Zhang authored Jan 22, 2025
  
  b3393e94
- disable custom allreduce on HIP (#3058) · ddc2001f
  Hui Liu authored Jan 22, 2025
  
  ddc2001f
- add notice about flashinfer in sgl-kernel (#3057) · 806a3002
  Yineng Zhang authored Jan 23, 2025
  
  806a3002
- fix rotary_embedding rope_scaling for phi (#3055) · 0d2148ef
  nstream-ai-devx authored Jan 22, 2025
  
  0d2148ef
- feat: integrate bmm_fp8 kernel into sgl-kernel (#3056) · bf669606
  Yineng Zhang authored Jan 23, 2025
  
  bf669606
- minor: update header and use pytest (#3054) · b2bd8f44
  Yineng Zhang authored Jan 22, 2025
  
  b2bd8f44
- feat: integrate activation kernels into sgl-kernel (#3053) · 9d9b482a
  Yineng Zhang authored Jan 22, 2025
  
  9d9b482a
- feat: integrate norm kernels into sgl-kernel (#3052) · 7353fb9b
  Yineng Zhang authored Jan 22, 2025
  
  7353fb9b
- sync the upstream updates of flashinfer (#3051) · bcda0c9e
  Yineng Zhang authored Jan 22, 2025
  
  bcda0c9e
- update norm cu (#3048) · 9f8f2c7f
  Yineng Zhang authored Jan 22, 2025
  
  9f8f2c7f
- Fix sgl-kernel compile for sm80 (#3046) · 6fc37bd8
  Ke Bao authored Jan 22, 2025
  
  6fc37bd8
- Use int64 as indices for set_kv_buffer (#3039) · 3d8f1c9b
  Lianmin Zheng authored Jan 21, 2025
  
  3d8f1c9b
21 Jan, 2025 4 commits
- fix pr-test-sgl-kernel (#3036) · a42213db
  Yineng Zhang authored Jan 22, 2025
  
  a42213db
- Support sm90 Int8 gemm (#3035) · 0ac019f1
  Ke Bao authored Jan 21, 2025
  
  0ac019f1
- feat: add flashinfer as 3rdparty and use rmsnorm as example (#3033) · 5a0d680a
  Yineng Zhang authored Jan 21, 2025
  
  5a0d680a
- Add accuracy and latency tests of eagle into CI (#3027) · a4331cd2
  Lianmin Zheng authored Jan 21, 2025
  
  a4331cd2