Commits · 4851c202bbfb5dadc5a3b3b4409d3f083f4e8e90 · OpenDAS / vllm_cscc

07 Sep, 2024 1 commit
- Enable Random Prefix Caching in Serving Profiling Tool (benchmark_serving.py) (#8241) · 795b662c
  Wei-Sheng Chin authored Sep 06, 2024
  
  795b662c
06 Sep, 2024 1 commit
- [Frontend] Add --logprobs argument to `benchmark_serving.py` (#8191) · e5cab715
  afeldman-nm authored Sep 06, 2024
  
  e5cab715
04 Sep, 2024 1 commit
- [MISC] Replace input token throughput with total token throughput (#8164) · 77d9e514
  Cody Yu authored Sep 04, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  77d9e514
29 Aug, 2024 1 commit
- Add more percentiles and latencies (#7759) · 0c785d34
  Wei-Sheng Chin authored Aug 29, 2024
  
  0c785d34
21 Aug, 2024 1 commit
- [misc] Add Torch profiler support (#7451) · dd53c4b0
  William Lin authored Aug 21, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  dd53c4b0
15 Jul, 2024 1 commit
- [Bugfix] Benchmark serving script used global parameter 'args' in function... · ccb20db8
  Fish authored Jul 15, 2024
```
[Bugfix] Benchmark serving script used global parameter 'args' in function 'sample_random_requests' (#6428)
```
  ccb20db8
14 Jul, 2024 1 commit
- [Feature] vLLM CLI (#5090) · dbfe254e
  Ethan Xu authored Jul 14, 2024
```
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  dbfe254e
11 Jul, 2024 1 commit
- [CI/Build] Add nightly benchmarking for tgi, tensorrt-llm and lmdeploy (#5362) · a4feba92
  Kuntai Du authored Jul 11, 2024
  
  a4feba92
08 Jul, 2024 1 commit
- Feature/add benchmark testing (#5947) · 717f4bce
  Haichuan authored Jul 08, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  717f4bce
07 Jul, 2024 1 commit
- add benchmark for fix length input and output (#5857) · 333306a2
  Haichuan authored Jul 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  333306a2
20 Jun, 2024 1 commit
- [Frontend] Add FlexibleArgumentParser to support both underscore and dash in names (#5718) · 8065a7e2
  Michael Goin authored Jun 20, 2024
  
  8065a7e2
17 Jun, 2024 1 commit
- [Misc] use AutoTokenizer for benchmark serving when vLLM not installed (#5588) · 1f12122b
  zhyncs authored Jun 18, 2024
  
  1f12122b
15 Jun, 2024 1 commit
- [mypy] Enable type checking for test directory (#5017) · 0e9164b4
  Cyrus Leung authored Jun 15, 2024
  
  0e9164b4
14 Jun, 2024 1 commit

[CI/Build][Misc] Add CI that benchmarks vllm performance on those PRs with... · 319ad7f1

Kuntai Du authored Jun 13, 2024


[CI/Build][Misc] Add CI that benchmarks vllm performance on those PRs with `perf-benchmarks` label (#5073)
Co-authored-by: simon-mo <simon.mo@hey.com>

319ad7f1

05 Jun, 2024 1 commit
- [misc] benchmark_serving.py -- add ITL results and tweak TPOT results (#5263) · 02cc3b51
  Tyler Michael Smith authored Jun 05, 2024
  
  02cc3b51
25 May, 2024 1 commit
- [Misc] Make Serving Benchmark More User-friendly (#5044) · f17a1a8f
  Roger Wang authored May 25, 2024
  
  f17a1a8f
20 May, 2024 1 commit
- [Doc]Add documentation to benchmarking script when running TGI (#4920) · c3af4472
  Kuntai Du authored May 20, 2024
  
  c3af4472
24 Apr, 2024 1 commit
- [Misc] Update ShareGPT Dataset Sampling in Serving Benchmark (#4279) · 7923dcad
  Roger Wang authored Apr 24, 2024
  
  7923dcad
04 Apr, 2024 1 commit
- [Bugfix] Fix args in benchmark_serving (#3836) · 819a309c
  Chang Su authored Apr 04, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  819a309c
27 Mar, 2024 1 commit
- feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark (#3277) · 45b6ef65
  Roger Wang authored Mar 27, 2024
  
  45b6ef65
25 Mar, 2024 1 commit
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
16 Mar, 2024 1 commit
- [Misc] fix line length for entire codebase (#3444) · 8e67598a
  Simon Mo authored Mar 16, 2024
  
  8e67598a
08 Mar, 2024 1 commit
- [Minor Fix] Fix comments in benchmark_serving (#3252) · 1ece1ae8
  TianYu GUO authored Mar 08, 2024
  
  1ece1ae8
22 Feb, 2024 2 commits
- chore(vllm): codespell for spell checking (#2820) · 93dc5a28
  Massimiliano Pronesti authored Feb 22, 2024
  
  93dc5a28
- Update comment (#2934) · d7f39648
  Ronen Schaffer authored Feb 22, 2024
  
  d7f39648
13 Feb, 2024 1 commit
- Serving Benchmark Refactoring (#2433) · a4211a4d
  Roger Wang authored Feb 12, 2024
  
  a4211a4d
23 Jan, 2024 1 commit
- lint: format all python file instead of just source code (#2567) · 1e4277d2
  Simon Mo authored Jan 23, 2024
  
  1e4277d2
22 Jan, 2024 1 commit
- Fix progress bar and allow HTTPS in `benchmark_serving.py` (#2552) · 63e835cb
  Harry Mellor authored Jan 22, 2024
  
  63e835cb
19 Jan, 2024 1 commit
- Support OpenAI API server in `benchmark_serving.py` (#2172) · 2709c000
  Harry Mellor authored Jan 19, 2024
  
  2709c000
03 Oct, 2023 1 commit
- Use monotonic time where appropriate (#1249) · acbed3ef
  Antoni Baum authored Oct 02, 2023
  
  acbed3ef
20 Jul, 2023 1 commit
- fix: enable trust-remote-code in api server & benchmark. (#509) · 8c4b2592
  Ricardo Lu authored Jul 20, 2023
  
  8c4b2592
28 Jun, 2023 1 commit
- [Tokenizer] Add an option to specify tokenizer (#284) · 4338cc47
  Woosuk Kwon authored Jun 28, 2023
  
  4338cc47
26 Jun, 2023 1 commit
- [Fix] Fix default port number in benchmark scripts (#265) · 43710e8d
  Zhuohan Li authored Jun 26, 2023
  
  43710e8d
18 Jun, 2023 1 commit
- Add comments on swap space (#154) · 3f92038b
  Woosuk Kwon authored Jun 18, 2023
  
  3f92038b
17 Jun, 2023 2 commits
- Change the name to vLLM (#150) · 0b98ba15
  Woosuk Kwon authored Jun 17, 2023
  
  0b98ba15
- Rename servers to engines (#152) · e5464ee4
  Zhuohan Li authored Jun 17, 2023
  
  e5464ee4
15 Jun, 2023 1 commit
- Add script for benchmarking serving throughput (#145) · 311490a7
  Woosuk Kwon authored Jun 14, 2023
  
  311490a7