Commits · 9924bbe153772570744f7cbbcb8b23d7064f0f76 · change / sglang

20 Apr, 2025 1 commit
- Fix bench_serving fail when zero warmup requests (#5574) · 9924bbe1
  fzyzcjy authored Apr 21, 2025
  
  9924bbe1
16 Apr, 2025 1 commit
- Clean up imports (#5467) · 177320a5
  Lianmin Zheng authored Apr 16, 2025
  
  177320a5
15 Apr, 2025 1 commit
- Fix bench_serving with random-ids (#5214) · 3dfc6023
  Yuhong Guo authored Apr 15, 2025
  
  3dfc6023
04 Apr, 2025 1 commit
- Add DeepSeek V3/R1 shared experts fusion (#4918) · 924ca7c9
  Xiaoyu Zhang authored Apr 04, 2025
  
  924ca7c9
01 Apr, 2025 1 commit
- update bench_serving (#4958) · fda6bb78
  Yineng Zhang authored Apr 01, 2025
  
  fda6bb78
28 Mar, 2025 1 commit
- [Feature] add multi-rank support for Lora (#4492) · ef9a378a
  chaobo jia authored Mar 29, 2025
```
Co-authored-by: rudy152 <czh1137892874@gmail.com>
```
  ef9a378a
24 Mar, 2025 1 commit
- Support FA3 as Attention backend by using `--attention-backend fa3` (#4680) · 5d7edc8e
  Stefan He authored Mar 23, 2025
```
Co-authored-by: qsong <qsong@linkedin.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
```
  5d7edc8e
15 Mar, 2025 1 commit
- Update bench_serving.py (#4454) · 470b4740
  Xu Song authored Mar 16, 2025
  
  470b4740
08 Mar, 2025 1 commit
- Fix bench_serving flush cache not recognizing OPENAI_API_KEY (#4181) · 0fe7c13b
  Mingshan authored Mar 08, 2025
```
Signed-off-by: Mingshan <git@brighill.com>
```
  0fe7c13b
06 Mar, 2025 1 commit
- fix bench serving bug (#4135) · 3a391812
  Lzhang-hub authored Mar 06, 2025
  
  3a391812
05 Mar, 2025 1 commit
- [Minor] more code cleanup (#4077) · e074d84e
  Lianmin Zheng authored Mar 04, 2025
  
  e074d84e
03 Mar, 2025 2 commits

Improve code styles (#4021) · 66301e12
Lianmin Zheng authored Mar 03, 2025

66301e12

Support penalty in overlap mode; return logprob with chunked prefill; improve... · ac238727

Lianmin Zheng authored Mar 03, 2025


Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: dhou-xai <dhou@x.ai>
Co-authored-by: Hanming Lu <hanming_lu@berkeley.edu>

ac238727

28 Feb, 2025 1 commit
- Fix bench_serving not recognizing OPENAI_API_KEY (#3870) · ec0a72c2
  Kebe authored Feb 28, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  ec0a72c2
21 Jan, 2025 1 commit
- Misc fixes for eagle (flush_cache, CPU overhead) (#3014) · 287d07a6
  Lianmin Zheng authored Jan 20, 2025
  
  287d07a6
16 Jan, 2025 1 commit
- Improve benchmark scripts and error message printing (#2922) · 8f2c522a
  Lianmin Zheng authored Jan 16, 2025
  
  8f2c522a
10 Jan, 2025 1 commit
- Add more metrics to serving benchmark. (#2819) · 8f157893
  Pratyush Patel authored Jan 10, 2025
  
  8f157893
07 Jan, 2025 1 commit
- Misc fix for min_p_sampling, --cuda-graph-bs (#2761) · bdc1acf6
  Lianmin Zheng authored Jan 07, 2025
  
  bdc1acf6
26 Dec, 2024 1 commit
- Fix test and benchmark scripts (#2598) · dc3bee48
  Lianmin Zheng authored Dec 26, 2024
  
  dc3bee48
24 Dec, 2024 1 commit
- [Bench] Flush cache before benchmarking (#2566) · 8a56b431
  Ying Sheng authored Dec 23, 2024
  
  8a56b431
08 Dec, 2024 1 commit
- Fix a bug with logprob streaming + chunked prefill (#2403) · a2486eb5
  Lianmin Zheng authored Dec 08, 2024
  
  a2486eb5
03 Dec, 2024 1 commit
- [Minor] Fix logger and style (#2325) · 3ddb1c46
  Lianmin Zheng authored Dec 02, 2024
  
  3ddb1c46
30 Nov, 2024 1 commit
- Support LoRA in Completion API (#2243) · 01017d4c
  bjmsong authored Nov 30, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  01017d4c
25 Nov, 2024 2 commits
- Replace prob based with threshold based load balancing (#2170) · 4b0a1c93
  Byron Hsu authored Nov 24, 2024
  
  4b0a1c93
- Allow overwrite flashinfer use_tensorcore (#2169) · 8e1adb84
  Lianmin Zheng authored Nov 24, 2024
  
  8e1adb84
23 Nov, 2024 3 commits
- [router] cache-aware load-balancing router v1 (#2114) · cbedd1db
  Byron Hsu authored Nov 23, 2024
  
  cbedd1db
- fix: resolve bench_serving args (#2139) · ad47749b
  Yineng Zhang authored Nov 23, 2024
  
  ad47749b
- Add concurrency option for benchmark (#2136) · 60769be1
  Yunmeng authored Nov 23, 2024
  
  60769be1
22 Nov, 2024 1 commit
- Benchmark with Pytorch Profiler easily (#2110) · ad30d5cf
  bjmsong authored Nov 22, 2024
```
Co-authored-by: root <bjmsong@126.com>
```
  ad30d5cf
19 Nov, 2024 1 commit
- minor: add dataset dump and questions shuffle (#2093) · 55bd97f3
  Yineng Zhang authored Nov 20, 2024
  
  55bd97f3
15 Nov, 2024 2 commits
- Fix the default arguments of bench_offline_throughput.py & simplify detokenizer manager (#2042) · 2558d6a6
  Lianmin Zheng authored Nov 15, 2024
  
  2558d6a6
- Offline LLM Engine Benchmark Throughput (#1968) · f6dd6486
  zolinthecow authored Nov 14, 2024
```
Co-authored-by: ByronHsu <byronhsu1230@gmail.com>
```
  f6dd6486
12 Nov, 2024 1 commit
- Filter empty prompt in random bench serving (#2011) · b808a383
  Ke Bao authored Nov 12, 2024
  
  b808a383
11 Nov, 2024 1 commit
- Add gen-shared-prefix dataset in bench_serving (#1990) · 8169c6f4
  Byron Hsu authored Nov 10, 2024
  
  8169c6f4
03 Nov, 2024 1 commit
- feat: support truss endpoint for benchmark serving (#1906) · 793b79db
  Yineng Zhang authored Nov 03, 2024
  
  793b79db
15 Oct, 2024 2 commits
- Improve benchmark scripts (#1672) · 175afed3
  Lianmin Zheng authored Oct 14, 2024
  
  175afed3
- [Minor] Add some utility functions (#1671) · 4a292f67
  Lianmin Zheng authored Oct 14, 2024
  
  4a292f67
04 Oct, 2024 1 commit
- [Fix] Fix major performance bug in certain cases (#1563) · 04b262cd
  Ying Sheng authored Oct 04, 2024
```
Co-authored-by: hnyls2002 <hnyls2002@gmail.com>
```
  04b262cd
18 Sep, 2024 1 commit
- Add bench_server_latency.py (#1452) · 5e62a6b7
  Lianmin Zheng authored Sep 18, 2024
  
  5e62a6b7
09 Sep, 2024 1 commit
- [Minor] Many cleanup (#1357) · e4d68afc
  Lianmin Zheng authored Sep 09, 2024
  
  e4d68afc