Commits · d085a4408244502dd496cae0ccbc524352e87ef8 · OpenDAS / vllm_cscc

11 Apr, 2025 2 commits
- update benchmark_serving_structured_output to include auto backend (#16438) · d544d141
  Chenyaaang authored Apr 10, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  d544d141
- check input length of sonnet samples (#16423) · 3e397a94
  Alexey Belyakov authored Apr 11, 2025
```
Signed-off-by: alexey-belyakov <alexey.belyakov@intel.com>
```
  3e397a94
10 Apr, 2025 4 commits
- Fix range_ratio Bug in RandomDataset (#16126) · 268c3250
  WWW authored Apr 11, 2025
```
Signed-off-by: jadewang21 <jadewangcn@outlook.com>
```
  268c3250
- [Bugfix] Fix output token length check logic (#16419) · 7cd0bd72
  look authored Apr 11, 2025
```
Signed-off-by: look <eeslook@163.com>
```
  7cd0bd72
- [Bugfix] Fix bug when dataset is json (#15899) · 5fbab20e
  Chenyaaang authored Apr 10, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  5fbab20e
- fix sonnet dataset sample when prefix len is very small (#16379) · 417bcefb
  Chenyaaang authored Apr 09, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  417bcefb
09 Apr, 2025 2 commits
- Fix `benchmark_throughput.py --backend=hf` (#16352) · b2ce859b
  Michael Goin authored Apr 09, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b2ce859b
- [BugFix] fix some typos found by typos. (#16314) · 04149cce
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  04149cce
07 Apr, 2025 1 commit

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

55dcce91

06 Apr, 2025 1 commit
- [Benchmark] Add sampling parameters to benchmark_serving. (#16022) · ba108019
  Hyesoo Yang authored Apr 05, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  ba108019
04 Apr, 2025 1 commit

[Benchmark][Doc] Update throughput benchmark and README (#15998) · 95862f7b

Ziji Shi (Steven) authored Apr 04, 2025


Signed-off-by: StevenShi-23 <shi.ziji.sm@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

95862f7b

03 Apr, 2025 1 commit

[Benchmark] Add AIMO Dataset to Benchmark (#15955) · 06f21ce7

Ziji Shi (Steven) authored Apr 02, 2025


Signed-off-by: Ziji Shi <shi.ziji.sm@gmail.com>
Signed-off-by: StevenShi-23 <shi.ziji.sm@gmail.com>

06f21ce7

02 Apr, 2025 2 commits
- [Bugfix][Benchmarks] Ensure `async_request_deepspeed_mii` uses the OpenAI choices key (#15926) · 25293780
  Brayden Zhong authored Apr 02, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  25293780
- [Benchmark]Fix error message (#15866) · aa557e64
  Li Wang authored Apr 02, 2025
```
Signed-off-by: wangli <wangli858794774@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  aa557e64
01 Apr, 2025 1 commit
- Add option to use DeepGemm contiguous grouped gemm kernel for fused MoE operations. (#13932) · e59ca942
  bnellnm authored Apr 01, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  e59ca942
31 Mar, 2025 1 commit
- [Benchmark] Update Vision Arena Dataset and HuggingFaceDataset Setup (#15748) · effc5d24
  Jennifer Zhao authored Mar 31, 2025
```
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  effc5d24
28 Mar, 2025 2 commits
- [Minor] Remove TGI launching script (#15646) · 70e13224
  Woosuk Kwon authored Mar 28, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  70e13224
- [Misc]add coding benchmark for speculative decoding (#15303) · e7f720ea
  Chen Xia authored Mar 27, 2025
```
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
```
  e7f720ea
27 Mar, 2025 1 commit

[Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) · 9239bf71

ElizaWszola authored Mar 27, 2025


Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>

9239bf71

26 Mar, 2025 1 commit
- [Misc] Warn about v0 in benchmark_paged_attn.py (#15495) · 23114d33
  Tyler Michael Smith authored Mar 25, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  23114d33
23 Mar, 2025 1 commit
- [Misc] Add tuned R1 w8a8 and MoE configs for NVIDIA L20 (#15322) · f90d34b4
  DefTruth authored Mar 23, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  f90d34b4
20 Mar, 2025 3 commits
- [Core][V0] Add guidance backend for structured output (#14589) · 1f16b7fe
  Russell Bryant authored Mar 20, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <lohuynh@microsoft.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
```
  1f16b7fe
- [Benchmark] Allow oversample request in benchmark dataset (#15170) · b88be221
  Jennifer Zhao authored Mar 19, 2025
```
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  b88be221
- fix "Total generated tokens:" is 0 if using --backend tgi and --endpo… (#14673) · 40828ce5
  Wang, Yi authored Mar 20, 2025
```
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
```
  40828ce5
19 Mar, 2025 1 commit

[Misc][Benchmark] Add support for different `tokenizer_mode` (#15040) · 6c5a3195

Aaron Pham authored Mar 19, 2025


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

6c5a3195

18 Mar, 2025 1 commit

[Kernels] LoRA - Retire SGMV and BGMV Kernels (#14685) · 400d483e

Varun Sundar Rabindranath authored Mar 18, 2025


Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

400d483e

17 Mar, 2025 1 commit
- [Benchmark] Do not save detailed info to json by default (#14879) · 583a9778
  Simon Mo authored Mar 16, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  583a9778
16 Mar, 2025 1 commit
- [Misc][Doc] Minor benchmark README update (#14874) · 3453b964
  Roger Wang authored Mar 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  3453b964
14 Mar, 2025 2 commits

[BugFix]Fix performance serving benchmark when enable profiling (#14737) · 09269b31
Li Wang authored Mar 14, 2025
```
Signed-off-by: wangli <wangli858794774@gmail.com>
```
09269b31

[Feature] Add visionarena offline support for benchmark_throughput (#14654) · a6e0d096

Jennifer Zhao authored Mar 13, 2025

Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Jennifer Zhao <JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

a6e0d096

13 Mar, 2025 1 commit
- [Bugfix] fix benchmark moe (#14653) · a73122de
  Jee Jee Li authored Mar 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a73122de
12 Mar, 2025 1 commit

[Doc] Update benchmarks README (#14646) · 4a42b9f5

Jennifer Zhao authored Mar 11, 2025

Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

4a42b9f5

11 Mar, 2025 2 commits
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
- benchmarks: simplify test jsonschema (#14567) · 08a1a112
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  08a1a112
10 Mar, 2025 4 commits

Fix typo in benchmark_serving_structured_output.py (#14566) · 432d6dad
Russell Bryant authored Mar 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
432d6dad

[V1] LoRA - Add triton kernels for V1 (#13096) · 5ff0d325

Varun Sundar Rabindranath authored Mar 10, 2025


Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

5ff0d325

Correct capitalisation: `VLLM` -> `vLLM` (#14562) · 3b352a2f
Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
3b352a2f

[Feature] Consolidate performance benchmark datasets (#14036) · 1253b157

Jennifer Zhao authored Mar 10, 2025


Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

1253b157

08 Mar, 2025 1 commit
- [benchmarks] Add option to use unique jsonschema for each request (#14457) · 9085aabd
  Russell Bryant authored Mar 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  9085aabd
07 Mar, 2025 1 commit
- [Benchmarks] Make detokenization optional in benchmark scripts (#11697) · 58abe354
  Jeremy Arnold authored Mar 07, 2025
```
Signed-off-by: Jeremy Arnold <Jeremy.Arnold@amd.com>
```
  58abe354