Commits · f46f394f4d4dbe4aae85403dec006199b34d2840 · change / sglang

26 Dec, 2024 6 commits
- Update README.md (#2605) · f46f394f
  Lianmin Zheng authored Dec 26, 2024
  
  f46f394f
- Fix logprob_start_len for multi modal models (#2597) · 77395154
  Lianmin Zheng authored Dec 26, 2024
```
Co-authored-by: libra <lihu723@gmail.com>
Co-authored-by: fzyzcjy <ch271828n@outlook.com>
Co-authored-by: Wang, Haoyu <haoyu.wang@intel.com>
```
  77395154
- update readme of DeepSeek V3 (#2596) · 637de9e8
  fsygd authored Dec 26, 2024
  
  637de9e8
- h100 tuning fused_moe_triton for qwen2 moe (#2560) · 9a23c484
  Xiaoyu Zhang authored Dec 26, 2024
  
  9a23c484
- docs: update deepseek v3 example (#2592) · 635a0426
  Yineng Zhang authored Dec 26, 2024
  
  635a0426
- docs: add deepseek v3 launch instructions (#2589) · 75ad0a14
  Yineng Zhang authored Dec 26, 2024
  
  75ad0a14
23 Dec, 2024 1 commit
- Reorg moe code (#2563) · e835a500
  Ke Bao authored Dec 24, 2024
  
  e835a500
22 Dec, 2024 1 commit
- [kernel optimize] benchmark write_req_to_token_pool_triton and optimize kernel (#2509) · 7d672d27
  Xiaoyu Zhang authored Dec 22, 2024
  
  7d672d27
17 Dec, 2024 2 commits
- benchmark decoding attention kernel with cudnn (#2467) · e2102669
  bjmsong authored Dec 17, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  e2102669
- Add a benchmark script for in-batch prefix caching (#2494) · 56198b45
  Lianmin Zheng authored Dec 16, 2024
  
  56198b45
15 Dec, 2024 1 commit
- [Benchmark] add a benchmark for hf/vllm/sglang rmsnorm (#2486) · a0592c05
  Xiaoyu Zhang authored Dec 15, 2024
  
  a0592c05
11 Dec, 2024 1 commit
- decoding attention kernel benchmark (#2425) · f6772394
  bjmsong authored Dec 11, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  f6772394
09 Dec, 2024 1 commit
- Add a unittest for fused_moe (#2416) · 3844feb9
  Xiaoyu Zhang authored Dec 09, 2024
  
  3844feb9
03 Dec, 2024 1 commit
- Improve torch compile for fused moe (#2327) · 07ec07ad
  Lianmin Zheng authored Dec 03, 2024
  
  07ec07ad
02 Dec, 2024 1 commit
- Add more fused moe benchmark utilities (#2314) · 33deca81
  Lianmin Zheng authored Dec 02, 2024
  
  33deca81
29 Nov, 2024 1 commit
- [benchmark] Add fused_moe_triton benchmark and tuning tools (#2225) · 262e370f
  Xiaoyu Zhang authored Nov 30, 2024
```
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
```
  262e370f
24 Nov, 2024 1 commit
- Merged three native APIs into one: get_server_info (#2152) · dbe17293
  Henry Hyeonmok Ko authored Nov 24, 2024
  
  dbe17293
23 Nov, 2024 1 commit
- [router] cache-aware load-balancing router v1 (#2114) · cbedd1db
  Byron Hsu authored Nov 23, 2024
  
  cbedd1db
22 Nov, 2024 1 commit
- docs: fix module docstrings and copyright headers (#2077) · 62a4a339
  Xuehai Pan authored Nov 22, 2024
  
  62a4a339
15 Nov, 2024 2 commits
- Fix json benchmark (#2043) · c29b98e0
  Lianmin Zheng authored Nov 15, 2024
  
  c29b98e0
- benchmark json schema (#2030) · 954f4e6b
  DarkSharpness authored Nov 15, 2024
  
  954f4e6b
11 Nov, 2024 1 commit
- [rust] cache-aware DP - approx tree (#1934) · f9633fa9
  Byron Hsu authored Nov 10, 2024
  
  f9633fa9
06 Nov, 2024 1 commit
- minor: Add basic editorconfig and pre-commit hooks to enforce style for whitespaces (#1926) · a5e0defb
  Xuehai Pan authored Nov 06, 2024
  
  a5e0defb
17 Oct, 2024 1 commit
- Fix engine unit test (#1701) · dd3809fa
  Lianmin Zheng authored Oct 17, 2024
  
  dd3809fa
06 Oct, 2024 1 commit
- [LoRA, Performance] Speedup multi-LoRA serving - Step 1 (#1587) · 9c064bf7
  Ying Sheng authored Oct 06, 2024
  
  9c064bf7
02 Oct, 2024 1 commit
- [Fix] Fix all the Huggingface paths (#1553) · 2c7d0a5b
  Theresa Barton authored Oct 02, 2024
  
  2c7d0a5b
15 Sep, 2024 1 commit
- [Feature] Support LoRA path renaming and add LoRA serving benchmarks (#1433) · 37963394
  Ying Sheng authored Sep 15, 2024
  
  37963394
09 Sep, 2024 2 commits
- [Minor] Many cleanup (#1357) · e4d68afc
  Lianmin Zheng authored Sep 09, 2024
  
  e4d68afc
- [server] Passing `model_override_args` to `launch_server` via the CLI. (#1298) · c9b75917
  Kai-Hsun Chen authored Sep 09, 2024
```
Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
```
  c9b75917
05 Sep, 2024 3 commits
- docs: add conclusion (#1340) · 62f15eea
  Yineng Zhang authored Sep 06, 2024
  
  62f15eea
- docs: highlight ttft itl and throughput (#1337) · 79794af5
  Yineng Zhang authored Sep 06, 2024
  
  79794af5
- docs: update README (#1336) · 3494b32c
  Yineng Zhang authored Sep 05, 2024
  
  3494b32c
18 Aug, 2024 1 commit
- Improve benchmark (#1140) · 57d0bd91
  Lianmin Zheng authored Aug 17, 2024
  
  57d0bd91
16 Aug, 2024 1 commit
- Fix the deadlock in multi-node tp (#1122) · 5a261bd0
  Lianmin Zheng authored Aug 16, 2024
  
  5a261bd0
15 Aug, 2024 1 commit
- Use a single workspace for flashinfer (#1077) · 326df4ba
  Lianmin Zheng authored Aug 14, 2024
  
  326df4ba
14 Aug, 2024 3 commits
- docs: update nsys usage (#1103) · 1c2b5f52
  Yineng Zhang authored Aug 15, 2024
  
  1c2b5f52
- Use `dtype` to control generate (#1082) · a34dd86a
  Liangsheng Yin authored Aug 14, 2024
```
Co-authored-by: zhyncs <me@zhyncs.com>
```
  a34dd86a
- Update grok 1 model (#1095) · a59636bb
  Lianmin Zheng authored Aug 14, 2024
  
  a59636bb
07 Aug, 2024 1 commit
- fix nsys cannot profile cuda kernel (#957) · 41bb1ab1
  Meng, Peng authored Aug 07, 2024
  
  41bb1ab1
04 Aug, 2024 1 commit
- Support MLA for DeepSeek-V2 with Triton - step 1 (#905) · e1eae1fd
  Ke Bao authored Aug 05, 2024
  
  e1eae1fd