Commits · ffa1b3e318c9d1342a5e430eb04df609e22a3775 · change / sglang · GitLab

07 Mar, 2025 1 commit
- Add an example of using deepseekv3 int8 sglang. (#4177) · ffa1b3e3
  lukec authored Mar 07, 2025
```
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
```
  ffa1b3e3
06 Mar, 2025 1 commit
- Online serving benchmarks of real datasets for hierarchical KV caching (#3211) · 25482edb
  Yueyang Pan authored Mar 06, 2025
```
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
```
  25482edb
05 Mar, 2025 1 commit
- bench: add dataset param for bench_multiturn (#3990) · e5760bc4
  Lu Changqi authored Mar 05, 2025
  
  e5760bc4
03 Mar, 2025 4 commits
- Improve code styles (#4021) · 66301e12
  Lianmin Zheng authored Mar 03, 2025
  
  66301e12
- Support penalty in overlap mode; return logprob with chunked prefill; improve... · ac238727
  Lianmin Zheng authored Mar 03, 2025
```
Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>
```
  ac238727
- Optimize Triton Kernel of Group GEMM in DeepGEMM Benchmark (#4014) · 0194948f
  Stefan He authored Mar 02, 2025
  
  0194948f
- Add Benchmark for DeepGEMM Group GEMM (#3993) · b7e274f2
  Stefan He authored Mar 02, 2025
  
  b7e274f2
02 Mar, 2025 2 commits
- fix typo in deep gemm benchmarking(#3991) · 50f28f65
  Xiaoyu Zhang authored Mar 02, 2025
  
  50f28f65
- add deepgemm and sglang fp8 block-wise gemm benchmark (#3893) · 90a55e25
  Xiaoyu Zhang authored Mar 02, 2025
  
  90a55e25
01 Mar, 2025 1 commit
- Revert "[MOE] enable efficient moe_alignment multi-blocks execution (3x~6x)" (#3982) · 18bb216c
  Chayenne authored Feb 28, 2025
  
  18bb216c
28 Feb, 2025 1 commit
- [MOE] enable efficient moe_alignment multi-blocks execution (3x~6x) (#3613) · 1c96fa86
  yiakwy-xpu-ml-framework-team authored Feb 28, 2025
  
  1c96fa86
27 Feb, 2025 2 commits
- revert "Docs: Reorngaize dpsk links #3900" (#3933) · 5d860168
  Yineng Zhang authored Feb 27, 2025
  
  5d860168
- Tuning Script for Feature DeepSeek V3/R1 INT8 Quantization (block-wise) (#3922) · b0df5d24
  laixin authored Feb 27, 2025
```
Co-authored-by: sleepcoo <sleepcoo@gmail.com>
```
  b0df5d24
26 Feb, 2025 1 commit
- Docs: Reorngaize dpsk links (#3900) · 7c1692aa
  Chayenne authored Feb 26, 2025
  
  7c1692aa
25 Feb, 2025 1 commit
- [BugFix] Fix crash when receive a req with structed output in DP attention mode. (#3841) · 10771026
  IAN authored Feb 26, 2025
  
  10771026
24 Feb, 2025 1 commit

Hierarchical Caching for SGLang (#2693) · 6c7a152c

Zhiqiang Xie authored Feb 23, 2025


Co-authored-by: Wenxuan Tan <wenxuan.tan@wisc.edu>
Co-authored-by: Yineng Zhang <me@zhyncs.com>

6c7a152c

22 Feb, 2025 1 commit
- bench: Add MMMU benchmark for vLM (#3562) · 45205d88
  Mick authored Feb 23, 2025
  
  45205d88
19 Feb, 2025 3 commits
- Variance measure for reasoning benchmark (#3677) · bb121214
  simveit authored Feb 19, 2025
  
  bb121214
- [Docs] Add SkyPilot DeepSeek example (#3706) · f93e9158
  Zhanghao Wu authored Feb 19, 2025
  
  f93e9158
- set NCCL_IB_GID_INDEX=3 for multi node NVIDIA InfiniBand if needed (#3698) · fe0673f1
  Yineng Zhang authored Feb 19, 2025
  
  fe0673f1
18 Feb, 2025 1 commit
- [ROCm] Optimal MOE Tuning for AMD Radeon Graphics (#3567) · ddf39d3f
  yigex authored Feb 18, 2025
  
  ddf39d3f
17 Feb, 2025 2 commits
- support multi-gpu block-gemm tuning (#3639) · c38f3aed
  Xiaoyu Zhang authored Feb 18, 2025
  
  c38f3aed
- [docker] added rdma support (#3619) · c9565e49
  Shenggui Li authored Feb 17, 2025
  
  c9565e49
16 Feb, 2025 1 commit
- Benchmark for reasoning models (#3532) · 3d4a8f9b
  simveit authored Feb 16, 2025
```
Co-authored-by: Chayenne <zhaochen20@outlook.com>
```
  3d4a8f9b
14 Feb, 2025 2 commits
- update flashinfer-python (#3557) · ac963be2
  Yineng Zhang authored Feb 14, 2025
  
  ac963be2
- chore: bump v0.4.3 (#3556) · e0b9a423
  Yineng Zhang authored Feb 14, 2025
  
  e0b9a423
13 Feb, 2025 2 commits
- update README (#3543) · 20de05a7
  Yineng Zhang authored Feb 13, 2025
  
  20de05a7
- Update DeepSeek V3 Doc (#3541) · bf2a7087
  Jhin authored Feb 13, 2025
  
  bf2a7087
12 Feb, 2025 1 commit
- refine deepseek_v3 launch server doc (#3522) · 693c2600
  Xiaoyu Zhang authored Feb 12, 2025
  
  693c2600
11 Feb, 2025 1 commit
- [ROCm] Add ROCm tuning config to block gemm and Re-tune for AMD Radeon Graphics (#3418) · fdf04a14
  yigex authored Feb 11, 2025
```
Co-authored-by: Bruce Xue <yigex@xilinx.com>
Co-authored-by: HAI <hixiao@gmail.com>
```
  fdf04a14
10 Feb, 2025 2 commits
- refine some typo (#3473) · 2f47d710
  Xiaoyu Zhang authored Feb 10, 2025
  
  2f47d710
- chore: bump v0.4.2.post4 (#3459) · cddb1cdf
  Yineng Zhang authored Feb 10, 2025
  
  cddb1cdf
08 Feb, 2025 5 commits
- fix EAGLE 2 non greedy case (#3407) · fad315cb
  Yineng Zhang authored Feb 09, 2025
```
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
```
  fad315cb
- fix typo · f90db8bc
  Yineng Zhang authored Feb 08, 2025
  
  f90db8bc
- Add deepseek-v3 a100 serving example (#3404) · d8ad5970
  Ke Bao authored Feb 08, 2025
  
  d8ad5970
- Update fused_moe's benchmark (#3346) · 849f58d6
  GaoYuYang authored Feb 08, 2025
  
  849f58d6
- [BUG] fix moe benchmark when bs*seq is small (#3382) · 64480df4
  yiakwy-xpu-ml-framework-team authored Feb 08, 2025
  
  64480df4
07 Feb, 2025 1 commit
- chore: bump v0.4.2.post3 (#3369) · c1f5f99f
  Yineng Zhang authored Feb 08, 2025
  
  c1f5f99f
06 Feb, 2025 2 commits
- optimize moe_align_kernel cuda (#3347) · cdae77b0
  Xiaoyu Zhang authored Feb 07, 2025
  
  cdae77b0
- [Doc] Add optimization option guide for deepseek v3 (#3349) · 6792411e
  Ke Bao authored Feb 06, 2025
  
  6792411e