Commits · f256ebe4df6757d76f1f1642d7e110268a2f8190 · OpenDAS / vllm_cscc · GitLab

01 Feb, 2025 1 commit

[Bugfix/CI] Fixup benchmark_moe.py (#12562) · cfa134d2

Tyler Michael Smith authored Feb 01, 2025

Fixes `is_marlin` not being passed into `get_default_config`

Also allow `--tensor-parallel-size` in addition to `-tp` and `--tp-size`
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

cfa134d2

31 Jan, 2025 1 commit
- [Kernel] Update `cutlass_scaled_mm` to support 2d group (blockwise) scaling (#11868) · 9798b2fb
  Lucas Wilkinson authored Jan 30, 2025
  
  9798b2fb
30 Jan, 2025 1 commit
- [Misc][MoE] add Deepseek-V3 moe tuning support (#12558) · 1c1bb0bb
  Divakar Verma authored Jan 29, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  1c1bb0bb
28 Jan, 2025 1 commit
- Update `pre-commit` hooks (#12475) · 823ab796
  Harry Mellor authored Jan 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  823ab796
24 Jan, 2025 2 commits
- [Misc] Enable proxy support in benchmark script (#12356) · 3bb8e2c9
  Junichi Sato authored Jan 24, 2025
```
Signed-off-by: Junichi Sato <junichi.sato@sbintuitions.co.jp>
```
  3bb8e2c9
- [Misc] Use VisionArena Dataset for VLM Benchmarking (#12389) · 3c818bdb
  Roger Wang authored Jan 24, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  3c818bdb
23 Jan, 2025 2 commits
- [Misc] Fix OpenAI API Compatibility Issues in Benchmark Script (#12357) · 9726ad67
  Junichi Sato authored Jan 24, 2025
```
Signed-off-by: Junichi Sato <junichi.sato@sbintuitions.co.jp>
```
  9726ad67
- [FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b
  Gregory Shtrasberg authored Jan 23, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
```
  e97f802b
22 Jan, 2025 1 commit
- [Benchmark] More accurate TPOT calc in `benchmark_serving.py` (#12288) · 222a9dc3
  Nick Hill authored Jan 21, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  222a9dc3
21 Jan, 2025 1 commit
- [bugfix] moe tuning. rm is_navi() (#12273) · 2acba47d
  Divakar Verma authored Jan 21, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  2acba47d
19 Jan, 2025 1 commit

benchmark_serving support --served-model-name param (#12109) · 936db119

gujing authored Jan 19, 2025


Signed-off-by: zibai <zibai.gj@alibaba-inc.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

936db119

17 Jan, 2025 1 commit
- [ROCm][MoE] moe tuning support for rocm (#12049) · 8027a724
  Divakar Verma authored Jan 17, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  8027a724
16 Jan, 2025 1 commit
- [misc] Add LoRA kernel micro benchmarks (#11579) · 5fd24ec0
  Varun Sundar Rabindranath authored Jan 16, 2025
  
  5fd24ec0
13 Jan, 2025 1 commit
- bugfix: Fix signature mismatch in benchmark's `get_tokenizer` function (#11982) · c6db2131
  elijah authored Jan 13, 2025
```
Signed-off-by: elijah <f1renze.142857@gmail.com>
```
  c6db2131
10 Jan, 2025 2 commits
- [Misc] Update benchmark_prefix_caching.py fixed example usage (#11920) · 8a579408
  minmin authored Jan 11, 2025
```
Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn>
Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>
```
  8a579408
- Doc fix in `benchmark_long_document_qa_throughput.py` (#11933) · 5959564f
  Kuntai Du authored Jan 10, 2025
```
Signed-off-by: Kuntai Du <kuntai@uchicago.edu>
```
  5959564f
09 Jan, 2025 1 commit
- [Bugfix] fix beam search input errors and latency benchmark script (#11875) · 1d967acb
  Ye (Charlotte) Qi authored Jan 09, 2025
```
Signed-off-by: Ye Qi <yeq@meta.com>
Co-authored-by: yeq <yeq@devgpu004.lla3.facebook.com>
```
  1d967acb
08 Jan, 2025 1 commit
- [Misc] sort torch profiler table by kernel timing (#11813) · 4d29e91b
  Divakar Verma authored Jan 07, 2025
  
  4d29e91b
01 Jan, 2025 1 commit
- [Benchmark] Add benchmark script for CPU offloading (#11533) · 0c6f9985
  Yihua Cheng authored Dec 31, 2024
```
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: KuntaiDu <kuntai@uchicago.edu>
```
  0c6f9985
25 Dec, 2024 1 commit
- [Misc] Update disaggregation benchmark scripts and test logs (#11456) · fc601665
  Jiaxin Shan authored Dec 24, 2024
```
Signed-off-by: Jiaxin Shan <seedjeffwan@gmail.com>
```
  fc601665
19 Dec, 2024 1 commit

[misc] benchmark_throughput : Add LoRA (#11267) · 98356735

Varun Sundar Rabindranath authored Dec 19, 2024


Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

98356735

18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

17 Dec, 2024 1 commit

[Misc] Kernel Benchmark for `RMSNorm` (#11241) · 02222a02

Roger Wang authored Dec 16, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiaoyu Zhang <BBuf@users.noreply.github.com>

02222a02

13 Dec, 2024 2 commits
- [Misc] Add tokenizer_mode param to benchmark_serving.py (#11174) · 238c0d93
  Alexander Matveev authored Dec 13, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  238c0d93
- [torch.compile] Dynamic fp8 + rms_norm fusion (#10906) · 30870b4f
  Luka Govedič authored Dec 12, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  30870b4f
04 Dec, 2024 2 commits

Benchmark serving structured output (#10880) · 82eb5ea8

Chendi.Xue authored Dec 04, 2024


Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

82eb5ea8

[Benchmark] Benchmark structured output with datasets (#10557) · 381ac93b

Chendi.Xue authored Dec 03, 2024


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>

381ac93b

03 Dec, 2024 1 commit
- [Bugfix] Prevent benchmark_throughput.py from using duplicated random prompts (#10753) · 4433195a
  Michael Goin authored Dec 02, 2024
  
  4433195a
02 Dec, 2024 1 commit

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

01 Dec, 2024 1 commit

[Misc] Adding `MMMU-Pro` vision dataset to serving benchmark (#10804) · c11f1721

Roger Wang authored Dec 01, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

c11f1721

21 Nov, 2024 1 commit
- fix the issue that len(tokenizer(prompt)["input_ids"]) > prompt_len (#10524) · 8a93a598
  Wang, Yi authored Nov 21, 2024
```
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
```
  8a93a598
19 Nov, 2024 1 commit
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
18 Nov, 2024 2 commits
- [misc] partial prefix & random input generation benchmark (#9929) · 90a6c759
  Ricky Xu authored Nov 18, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  90a6c759
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
16 Nov, 2024 1 commit
- [Misc] Update benchmark to support image_url file or http (#10287) · 8b6725b0
  Jaehyun An authored Nov 16, 2024
```
Signed-off-by: rbbang <anjaehyun87@gmail.com>
```
  8b6725b0
08 Nov, 2024 3 commits
- [Misc] Fix typo in #5895 (#10145) · f4c2187e
  Cyrus Leung authored Nov 08, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f4c2187e
- [Bugfix] Add error handling when server cannot respond any valid tokens (#5895) · ad39bd64
  DearPlanet authored Nov 08, 2024
  
  ad39bd64
- [V1] Prefix caching (take 2) (#9972) · 201fc077
  Cody Yu authored Nov 07, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  201fc077
07 Nov, 2024 2 commits
- [CI/Build] Add shell script linting using shellcheck (#7925) · 3be5b26a
  Russell Bryant authored Nov 07, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  3be5b26a
- [Misc] Add Gamma-Distribution Request Generation Support for Serving Benchmark. (#10105) · a62bc010
  Atlas authored Nov 07, 2024
```
Signed-off-by: Mozhou <spli161006@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  a62bc010