Commits · 222a9dc350d99e111ba09b29576113c9278e6a3e · OpenDAS / vllm_cscc

22 Jan, 2025 1 commit
- [Benchmark] More accurate TPOT calc in `benchmark_serving.py` (#12288) · 222a9dc3
  Nick Hill authored Jan 21, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  222a9dc3
19 Jan, 2025 1 commit

benchmark_serving support --served-model-name param (#12109) · 936db119

gujing authored Jan 19, 2025


Signed-off-by: zibai <zibai.gj@alibaba-inc.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

936db119

13 Jan, 2025 1 commit
- bugfix: Fix signature mismatch in benchmark's `get_tokenizer` function (#11982) · c6db2131
  elijah authored Jan 13, 2025
```
Signed-off-by: elijah <f1renze.142857@gmail.com>
```
  c6db2131
04 Dec, 2024 1 commit

Benchmark serving structured output (#10880) · 82eb5ea8

Chendi.Xue authored Dec 04, 2024


Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

82eb5ea8

21 Nov, 2024 1 commit
- fix the issue that len(tokenizer(prompt)["input_ids"]) > prompt_len (#10524) · 8a93a598
  Wang, Yi authored Nov 21, 2024
```
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
```
  8a93a598
08 Nov, 2024 2 commits
- [Misc] Fix typo in #5895 (#10145) · f4c2187e
  Cyrus Leung authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f4c2187e
- [Bugfix] Add error handling when server cannot respond any valid tokens (#5895) · ad39bd64
  DearPlanet authored Nov 08, 2024
  
  ad39bd64
06 Nov, 2024 1 commit
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
31 Oct, 2024 1 commit
- [Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens... · abbfb613
  Guillaume Calmettes authored Oct 31, 2024
```
[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)
```
  abbfb613
07 Oct, 2024 1 commit
- [core] remove beam search from the core (#9105) · 18b296fd
  youkaichao authored Oct 06, 2024
  
  18b296fd
04 Oct, 2024 1 commit
- [CI] Update performance benchmark: upgrade trt-llm to r24.07, and add SGLang (#7412) · fbb74420
  Kuntai Du authored Oct 04, 2024
  
  fbb74420
17 Sep, 2024 1 commit
- [Benchmark] Support sample from HF datasets and image input for benchmark_serving (#8495) · 1b6de835
  Isotr0py authored Sep 17, 2024
  
  1b6de835
06 Sep, 2024 1 commit
- [Frontend] Add --logprobs argument to `benchmark_serving.py` (#8191) · e5cab715
  afeldman-nm authored Sep 06, 2024
  
  e5cab715
21 Aug, 2024 1 commit
- [misc] Add Torch profiler support (#7451) · dd53c4b0
  William Lin authored Aug 21, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  dd53c4b0
09 Aug, 2024 1 commit
- [Bugfix] Fix ITL recording in serving benchmark (#7372) · 70d268a3
  Roger Wang authored Aug 09, 2024
  
  70d268a3
11 Jul, 2024 1 commit
- [Bugfix] Fix snapshot download in serving benchmark (#6318) · c4774eb8
  Roger Wang authored Jul 11, 2024
  
  c4774eb8
01 Jul, 2024 2 commits
- [Frontend] Relax api url assertion for openai benchmarking (#6046) · e373853e
  James Whedbee authored Jul 01, 2024
  
  e373853e
- [Misc] update benchmark backend for scalellm (#6018) · bb603268
  zhyncs authored Jul 02, 2024
  
  bb603268
29 Jun, 2024 1 commit
- [Bugfix] fix missing last itl in openai completions benchmark (#5926) · c4bca740
  mcalman authored Jun 28, 2024
  
  c4bca740
17 Jun, 2024 1 commit
- [Misc] use AutoTokenizer for benchmark serving when vLLM not installed (#5588) · 1f12122b
  zhyncs authored Jun 18, 2024
  
  1f12122b
13 Jun, 2024 1 commit
- [Bugfix]if the content is started with ":"(response of ping), client should i… (#5303) · 88407532
  Wang, Yi authored Jun 13, 2024
```
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  88407532
25 May, 2024 1 commit
- [Misc] Make Serving Benchmark More User-friendly (#5044) · f17a1a8f
  Roger Wang authored May 25, 2024
  
  f17a1a8f
17 Apr, 2024 1 commit
- [Bugfix] fix output parsing error for trtllm backend (#4137) · fe3b5bbc
  Elinx authored Apr 17, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  fe3b5bbc
12 Apr, 2024 1 commit
- [Doc] Add typing hints / mypy types cleanup (#3816) · c2b4a1bc
  Michael Feil authored Apr 11, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  c2b4a1bc
01 Apr, 2024 1 commit
- [Misc] Fix Benchmark TTFT Calculation for Chat Completions (#3768) · ccb58b23
  Roger Wang authored Apr 01, 2024
  
  ccb58b23
27 Mar, 2024 1 commit
- feat(benchmarks): Add Prefix Caching Benchmark to Serving Benchmark (#3277) · 45b6ef65
  Roger Wang authored Mar 27, 2024
  
  45b6ef65
16 Mar, 2024 2 commits
- [Misc] fix line length for entire codebase (#3444) · 8e67598a
  Simon Mo authored Mar 16, 2024
  
  8e67598a
- Replace `lstrip()` with `removeprefix()` to fix Ruff linter warning (#2958) · 14e3f9a1
  Ronen Schaffer authored Mar 16, 2024
  
  14e3f9a1
04 Mar, 2024 1 commit
- Fix the openai benchmarking requests to work with latest OpenAI apis (#2992) · 9a4548ba
  Chen Wang authored Mar 04, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9a4548ba
13 Feb, 2024 1 commit
- Serving Benchmark Refactoring (#2433) · a4211a4d
  Roger Wang authored Feb 12, 2024
  
  a4211a4d