Commits · 9d9b482a392598fc342ee449835af5535ccc772f · change / sglang · GitLab

22 Jan, 2025 6 commits
- feat: integrate activation kernels into sgl-kernel (#3053) · 9d9b482a
  Yineng Zhang authored Jan 22, 2025
  
  9d9b482a
- feat: integrate norm kernels into sgl-kernel (#3052) · 7353fb9b
  Yineng Zhang authored Jan 22, 2025
  
  7353fb9b
- sync the upstream updates of flashinfer (#3051) · bcda0c9e
  Yineng Zhang authored Jan 22, 2025
  
  bcda0c9e
- update norm cu (#3048) · 9f8f2c7f
  Yineng Zhang authored Jan 22, 2025
  
  9f8f2c7f
- Fix sgl-kernel compile for sm80 (#3046) · 6fc37bd8
  Ke Bao authored Jan 22, 2025
  
  6fc37bd8
- Use int64 as indices for set_kv_buffer (#3039) · 3d8f1c9b
  Lianmin Zheng authored Jan 21, 2025
  
  3d8f1c9b
21 Jan, 2025 11 commits
- fix pr-test-sgl-kernel (#3036) · a42213db
  Yineng Zhang authored Jan 22, 2025
  
  a42213db
- Support sm90 Int8 gemm (#3035) · 0ac019f1
  Ke Bao authored Jan 21, 2025
  
  0ac019f1
- feat: add flashinfer as 3rdparty and use rmsnorm as example (#3033) · 5a0d680a
  Yineng Zhang authored Jan 21, 2025
  
  5a0d680a
- Add accuracy and latency tests of eagle into CI (#3027) · a4331cd2
  Lianmin Zheng authored Jan 21, 2025
  
  a4331cd2
- upgrade torch version for sgl-kernel (#3026) · ec1c21cd
  Yineng Zhang authored Jan 21, 2025
  
  ec1c21cd
- minor: update Makefile for sgl-kernel (#3025) · 6c856b4f
  Yineng Zhang authored Jan 21, 2025
  
  6c856b4f
- Misc fixes for eagle (flush_cache, CPU overhead) (#3014) · 287d07a6
  Lianmin Zheng authored Jan 20, 2025
  
  287d07a6
- Enable Cohere2 Models (#3018) · d2571dd5
  Hui Liu authored Jan 20, 2025
  
  d2571dd5
- [EAGLE] Fix some boundary situation when retract reqs and req's max token = 1 (#2939) · b730aa6b
  996_icu authored Jan 21, 2025
```
Co-authored-by: josephyou <josephyou@tencent.com>
```
  b730aa6b
- Fix flaky tests in test_programs.py (#3022) · 60b2a44a
  Lianmin Zheng authored Jan 20, 2025
  
  60b2a44a
- [Doc] Update doc of custom logit processor (#3021) · 949b3fbf
  Hongpeng Guo authored Jan 20, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  949b3fbf
20 Jan, 2025 23 commits
- enable kv_scale remap (#3017) · da4e8b38
  Hui Liu authored Jan 20, 2025
  
  da4e8b38
- deepseek v3 and r1 chat template (#3015) · af6c5357
  Enrique Shockwave authored Jan 20, 2025
  
  af6c5357
- bump router to 0.1.3 (#3020) · 3ad4cd49
  Byron Hsu authored Jan 20, 2025
  
  3ad4cd49
- [router] Expose worker startup interval (#3019) · 3a8428ec
  Byron Hsu authored Jan 20, 2025
  
  3a8428ec
- [router] Expose worker startup secs & Return error instead of panic for router init (#3016) · 0311ce8e
  Byron Hsu authored Jan 20, 2025
  
  0311ce8e
- Add compile flags for cutlass 3.x (#3013) · 5dfcacfc
  Ke Bao authored Jan 21, 2025
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  5dfcacfc
- Add clang-format check to sgl-kernel ci (#3012) · 41a0ccd4
  Ke Bao authored Jan 20, 2025
  
  41a0ccd4
- chore: bump v0.4.1.post7 (#3009) · e94fb7cb
  Yineng Zhang authored Jan 20, 2025
  
  e94fb7cb
- [kernel] port rope cuda kernel to sgl-kernel (#2993) · b5caa22d
  Byron Hsu authored Jan 20, 2025
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  b5caa22d
- Sync distributed package from vllm 0.6.4.post1 (#3010) · 73401fd0
  Lianmin Zheng authored Jan 20, 2025
  
  73401fd0
- Roll back to use vllm custom allreduce (#3006) · 89cd9235
  Lianmin Zheng authored Jan 20, 2025
  
  89cd9235
- Fix perf regression on small batch sizes (#3008) · dc188132
  Lianmin Zheng authored Jan 20, 2025
  
  dc188132
- fix moe align blocks benchmark (#3003) · 10bfce71
  yiakwy-xpu-ml-framework-team authored Jan 20, 2025
  
  10bfce71
- [Enhancement] Custom Logit Processor Improvement (#2998) · 583697cd
  Hongpeng Guo authored Jan 20, 2025
```
Signed-off-by: Hongpeng Guo <hpguo@anyscale.com>
```
  583697cd
- Docs: Add Performance Demonstaration for DPA (#3005) · 2584f6d9
  Chayenne authored Jan 20, 2025
  
  2584f6d9
- Skip flaky custom_logit_processor tests (#3004) · 51e87f6f
  Lianmin Zheng authored Jan 20, 2025
  
  51e87f6f
- Update TypeBasedDispatcher and balance CI tests (#3001) · 09bcbe01
  Lianmin Zheng authored Jan 19, 2025
  
  09bcbe01
- Separate two entry points: Engine and HTTP server (#2996) · 03464890
  Lianmin Zheng authored Jan 19, 2025
```
Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com>
```
  03464890
- keep rotary_embedding only (#2997) · 44a96697
  Yineng Zhang authored Jan 20, 2025
  
  44a96697
- Remove dependency of pynvml on ROCm (#2995) · 1a820e38
  Chaitanya Sri Krishna Lolla authored Jan 20, 2025
  
  1a820e38
- Docs: Only use X-Grammar in structed output (#2991) · 0ffcfdf4
  Chayenne authored Jan 19, 2025
  
  0ffcfdf4
- Improve metrics, logging, and importing orders (#2992) · cd493b5a
  Lianmin Zheng authored Jan 19, 2025
  
  cd493b5a
- Move sgl.Runtime under sglang/lang (#2990) · 61f42b57
  Lianmin Zheng authored Jan 19, 2025
  
  61f42b57