Commits · ffa443afedd3ffefb8dbe20607692950b78c1496 · OpenDAS / vllm_cscc

19 Mar, 2025 1 commit

[Misc][Benchmark] Add support for different `tokenizer_mode` (#15040) · 6c5a3195

Aaron Pham authored Mar 19, 2025


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

6c5a3195

18 Mar, 2025 1 commit

[Kernels] LoRA - Retire SGMV and BGMV Kernels (#14685) · 400d483e

Varun Sundar Rabindranath authored Mar 18, 2025


Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

400d483e

17 Mar, 2025 1 commit
- [Benchmark] Do not save detailed info to json by default (#14879) · 583a9778
  Simon Mo authored Mar 16, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  583a9778
16 Mar, 2025 1 commit
- [Misc][Doc] Minor benchmark README update (#14874) · 3453b964
  Roger Wang authored Mar 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  3453b964
14 Mar, 2025 2 commits

[BugFix]Fix performance serving benchmark when enable profiling (#14737) · 09269b31
Li Wang authored Mar 14, 2025
```
Signed-off-by: wangli <wangli858794774@gmail.com>
```
09269b31

[Feature] Add visionarena offline support for benchmark_throughput (#14654) · a6e0d096

Jennifer Zhao authored Mar 13, 2025

Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Jennifer Zhao <JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

a6e0d096

13 Mar, 2025 1 commit
- [Bugfix] fix benchmark moe (#14653) · a73122de
  Jee Jee Li authored Mar 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a73122de
12 Mar, 2025 1 commit

[Doc] Update benchmarks README (#14646) · 4a42b9f5

Jennifer Zhao authored Mar 11, 2025

Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

4a42b9f5

11 Mar, 2025 2 commits
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
- benchmarks: simplify test jsonschema (#14567) · 08a1a112
  Russell Bryant authored Mar 11, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  08a1a112
10 Mar, 2025 4 commits

Fix typo in benchmark_serving_structured_output.py (#14566) · 432d6dad
Russell Bryant authored Mar 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
432d6dad

[V1] LoRA - Add triton kernels for V1 (#13096) · 5ff0d325

Varun Sundar Rabindranath authored Mar 10, 2025


Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

5ff0d325

Correct capitalisation: `VLLM` -> `vLLM` (#14562) · 3b352a2f
Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
3b352a2f

[Feature] Consolidate performance benchmark datasets (#14036) · 1253b157

Jennifer Zhao authored Mar 10, 2025


Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

1253b157

08 Mar, 2025 1 commit
- [benchmarks] Add option to use unique jsonschema for each request (#14457) · 9085aabd
  Russell Bryant authored Mar 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  9085aabd
07 Mar, 2025 5 commits

[Benchmarks] Make detokenization optional in benchmark scripts (#11697) · 58abe354
Jeremy Arnold authored Mar 07, 2025
```
Signed-off-by: Jeremy Arnold <Jeremy.Arnold@amd.com>
```
58abe354

[V1][Core] Support for Structured Outputs (#12388) · 80e9afb5

Aaron Pham authored Mar 07, 2025


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

80e9afb5

[BUGFIX] Skip tokenization support for throughput benchmark (#12712) · 0ca3b8e0

Aleksandr Malyshev authored Mar 07, 2025


Signed-off-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>

0ca3b8e0

[Bugfix] Correctly call `cudaProfilerStop` in benchmarks script (#14183) · c34eeec5
Brayden Zhong authored Mar 06, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
c34eeec5
[Doc] Fix a typo (#14385) · ad60bbb2
Daniel Li authored Mar 06, 2025

ad60bbb2

06 Mar, 2025 1 commit
- Add benchmark for DeepGEMM and vLLM Block FP8 Dense GEMM (#13917) · ca100c90
  Michael Goin authored Mar 05, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  ca100c90
05 Mar, 2025 2 commits

Deprecate `best_of` Sampling Parameter in anticipation for vLLM V1 (#13997) · a4f1ee35

Vincent authored Mar 05, 2025

Signed-off-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

a4f1ee35

[Misc] Add Qwen2MoeForCausalLM moe tuning support (#14276) · 7bab4bb0
Jee Jee Li authored Mar 05, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
7bab4bb0

04 Mar, 2025 2 commits
- Fix benchmark_moe.py tuning for CUDA devices (#14164) · f78c0be8
  Michael Goin authored Mar 04, 2025
  
  f78c0be8
- [core] moe fp8 block quant tuning support (#14068) · bb5b6403
  Divakar Verma authored Mar 03, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  bb5b6403
03 Mar, 2025 2 commits
- [ROCm] Faster Custom Paged Attention kernels (#12348) · 848a6438
  TJian authored Mar 04, 2025
  
  848a6438
- Update deprecated Python 3.8 typing (#13971) · cf069aa8
  Harry Mellor authored Mar 03, 2025
  
  cf069aa8
01 Mar, 2025 2 commits
- [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) · 6a92ff93
  YajieWang authored Mar 01, 2025
  
  6a92ff93
- [Bugfix] Add file lock for ModelScope download (#14060) · 6a84164a
  Jee Jee Li authored Mar 01, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  6a84164a
26 Feb, 2025 2 commits
- [Misc]: Add support for goodput on guided benchmarking + TPOT calculation refactor (#13736) · ec8a5e53
  Brayden Zhong authored Feb 26, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  ec8a5e53
- [Misc] Improve LoRA spelling (#13831) · 5157338e
  Jee Jee Li authored Feb 26, 2025
  
  5157338e
24 Feb, 2025 2 commits
- Expert Parallelism (EP) Support for DeepSeek V2 (#12583) · 781096e3
  Jongseok Park authored Feb 24, 2025
  
  781096e3
- Fix some issues with benchmark data output (#13641) · e7ef74e2
  Huy Do authored Feb 23, 2025
```
Signed-off-by: Huy Do <huydhn@gmail.com>
```
  e7ef74e2
23 Feb, 2025 1 commit
- [Misc] Deprecate `--dataset` from `benchmark_serving.py` (#13708) · 9bebc951
  Roger Wang authored Feb 23, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  9bebc951
22 Feb, 2025 2 commits
- [CI/Build] Fix pre-commit errors (#13696) · 7f6bae56
  Cyrus Leung authored Feb 22, 2025
  
  7f6bae56
- [Bugfix] Fix benchmark script bug: inaccurate stats for vllm backend when... · 8aca27fa
  Robin authored Feb 22, 2025
```
[Bugfix] Fix benchmark script bug: inaccurate stats for vllm backend when max_model_len < input_len + output_len (#13691)
Signed-off-by: WangErXiao <863579016@qq.com>
```
  8aca27fa
17 Feb, 2025 1 commit
- Run v1 benchmark and integrate with PyTorch OSS benchmark database (#13068) · 45186834
  Huy Do authored Feb 17, 2025
```
Signed-off-by: Huy Do <huydhn@gmail.com>
```
  45186834
12 Feb, 2025 1 commit
- [RFC][vllm-API] Support tokenizer registry for customized tokenizer in vLLM (#12518) · 3ee696a6
  Keyun Tong authored Feb 11, 2025
```
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
```
  3ee696a6
11 Feb, 2025 1 commit

[Benchmark] Add BurstGPT to benchmark_serving (#13063) · 58047c6f

Woosuk Kwon authored Feb 10, 2025


Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

58047c6f

08 Feb, 2025 1 commit
- [CI/Build] Auto-fix Markdown files (#12941) · 8a69e0e2
  Cyrus Leung authored Feb 08, 2025
  
  8a69e0e2