Commits · 72c77763559317b2c8bddfd67e173b67aa1facb0 · change / sglang

13 Jan, 2025 2 commits
- Fix linear.py and improve weight loading (#2851) · 72c77763
  Lianmin Zheng authored Jan 13, 2025
```
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
```
  72c77763
- Add int8 quant kernel (#2848) · 85b2e057
  Ke Bao authored Jan 13, 2025
  
  85b2e057
11 Jan, 2025 2 commits
- docs: update README (#2841) · 197cbf9b
  Yineng Zhang authored Jan 11, 2025
  
  197cbf9b
- chore: bump v0.4.1.post5 (#2840) · f624901c
  Yineng Zhang authored Jan 11, 2025
  
  f624901c
09 Jan, 2025 1 commit
- minor: support specifying local dataset path for gsm8k and hellaswag (#2816) · 4f077c01
  sleepcoo authored Jan 09, 2025
  
  4f077c01
07 Jan, 2025 2 commits
- Misc fix for min_p_sampling, --cuda-graph-bs (#2761) · bdc1acf6
  Lianmin Zheng authored Jan 07, 2025
  
  bdc1acf6
- add benchmark_moe_align_blocks (#2767) · 380930a9
  Xiaoyu Zhang authored Jan 07, 2025
  
  380930a9
02 Jan, 2025 2 commits
- Included multi-node DeepSeekv3 example (#2707) · a990daff
  Rodrigo Garcia authored Jan 02, 2025
  
  a990daff
- Eagle speculative decoding part 3: small modifications to the general scheduler (#2709) · ad20b795
  Lianmin Zheng authored Jan 02, 2025
```
Co-authored-by: kavioyu <kavioyu@tencent.com>
```
  ad20b795
30 Dec, 2024 3 commits
- [Docs] clean up structured outputs docs (#2654) · 8c3b420e
  Lianmin Zheng authored Dec 29, 2024
  
  8c3b420e
- docs: update README (#2651) · 098d659c
  Yineng Zhang authored Dec 30, 2024
  
  098d659c
- add 2*h20 node serving example for deepseek v3 (#2650) · 76d14f8c
  Lzhang-hub authored Dec 30, 2024
```
Co-authored-by: Yineng Zhang <me@zhyncs.com>
```
  76d14f8c
29 Dec, 2024 3 commits
- Release 0.4.1.post3 - upload the config.json to PyPI (#2647) · 03d5fbfd
  Lianmin Zheng authored Dec 29, 2024
  
  03d5fbfd
- docs: update README (#2644) · 763dd55d
  Yineng Zhang authored Dec 30, 2024
  
  763dd55d
- Update Triton configs for block fp8 kernels (#2641) · afa0341e
  HandH1998 authored Dec 29, 2024
  
  afa0341e
28 Dec, 2024 2 commits
- add configs for block fp8 related kernels (#2628) · 7863e436
  Yineng Zhang authored Dec 28, 2024
```
Co-authored-by: HandH1998 <1335248067@qq.com>
```
  7863e436
- Update readme (#2625) · 8a2681e2
  Ke Bao authored Dec 28, 2024
  
  8a2681e2
27 Dec, 2024 1 commit
- docs: update README (#2618) · d9e6ee38
  Yineng Zhang authored Dec 28, 2024
  
  d9e6ee38
26 Dec, 2024 6 commits
- Update README.md (#2605) · f46f394f
  Lianmin Zheng authored Dec 26, 2024
  
  f46f394f
- Fix logprob_start_len for multi modal models (#2597) · 77395154
  Lianmin Zheng authored Dec 26, 2024
```
Co-authored-by: libra <lihu723@gmail.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: Wang, Haoyu <haoyu.wang@intel.com>
```
  77395154
- update readme of DeepSeek V3 (#2596) · 637de9e8
  fsygd authored Dec 26, 2024
  
  637de9e8
- h100 tuning fused_moe_triton for qwen2 moe (#2560) · 9a23c484
  Xiaoyu Zhang authored Dec 26, 2024
  
  9a23c484
- docs: update deepseek v3 example (#2592) · 635a0426
  Yineng Zhang authored Dec 26, 2024
  
  635a0426
- docs: add deepseek v3 launch instructions (#2589) · 75ad0a14
  Yineng Zhang authored Dec 26, 2024
  
  75ad0a14
23 Dec, 2024 1 commit
- Reorg moe code (#2563) · e835a500
  Ke Bao authored Dec 24, 2024
  
  e835a500
22 Dec, 2024 1 commit
- [kernel optimize] benchmark write_req_to_token_pool_triton and optimize kernel (#2509) · 7d672d27
  Xiaoyu Zhang authored Dec 22, 2024
  
  7d672d27
17 Dec, 2024 2 commits
- benchmark decoding attention kernel with cudnn (#2467) · e2102669
  bjmsong authored Dec 17, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  e2102669
- Add a benchmark script for in-batch prefix caching (#2494) · 56198b45
  Lianmin Zheng authored Dec 16, 2024
  
  56198b45
15 Dec, 2024 1 commit
- [Benchmark] add a benchmark for hf/vllm/sglang rmsnorm (#2486) · a0592c05
  Xiaoyu Zhang authored Dec 15, 2024
  
  a0592c05
11 Dec, 2024 1 commit
- decoding attention kernel benchmark (#2425) · f6772394
  bjmsong authored Dec 11, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  f6772394
09 Dec, 2024 1 commit
- Add a unittest for fused_moe (#2416) · 3844feb9
  Xiaoyu Zhang authored Dec 09, 2024
  
  3844feb9
03 Dec, 2024 1 commit
- Improve torch compile for fused moe (#2327) · 07ec07ad
  Lianmin Zheng authored Dec 03, 2024
  
  07ec07ad
02 Dec, 2024 1 commit
- Add more fused moe benchmark utilities (#2314) · 33deca81
  Lianmin Zheng authored Dec 02, 2024
  
  33deca81
29 Nov, 2024 1 commit
- [benchmark] Add fused_moe_triton benchmark and tuning tools (#2225) · 262e370f
  Xiaoyu Zhang authored Nov 30, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
```
  262e370f
24 Nov, 2024 1 commit
- Merged three native APIs into one: get_server_info (#2152) · dbe17293
  Henry Hyeonmok Ko authored Nov 24, 2024
  
  dbe17293
23 Nov, 2024 1 commit
- [router] cache-aware load-balancing router v1 (#2114) · cbedd1db
  Byron Hsu authored Nov 23, 2024
  
  cbedd1db
22 Nov, 2024 1 commit
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
15 Nov, 2024 2 commits
- Fix json benchmark (#2043) · c29b98e0
  Lianmin Zheng authored Nov 15, 2024
  
  c29b98e0
- benchmark json schema (#2030) · 954f4e6b
  DarkSharpness authored Nov 15, 2024
  
  954f4e6b
11 Nov, 2024 1 commit
- [rust] cache-aware DP - approx tree (#1934) · f9633fa9
  Byron Hsu authored Nov 10, 2024
  
  f9633fa9