Commits · 9b70e2b4c147ea650f9b943e6aecd977377fbbfd · OpenDAS / vllm_cscc

01 May, 2025 2 commits
- [Misc][Tools][Benchmark] Publish script to auto tune server parameters (#17207) · 9b70e2b4
  Chenyaaang authored May 01, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  9b70e2b4
- [Bugfix][Benchmarks] Allow benchmark of deepspeed-mii backend to select a model (#17285) · 86a1f67a
  Teruaki Ishizaki authored May 01, 2025
```
Signed-off-by: Teruaki Ishizaki <teruaki.ishizaki@ntt.com>
```
  86a1f67a
30 Apr, 2025 1 commit
- [V1][Feature] Enable Speculative Decoding with Structured Outputs (#14702) · 34120f5a
  Benjamin Chislett authored Apr 29, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
```
  34120f5a
28 Apr, 2025 2 commits
- [Benchmark] Add single turn MTBench to Serving Bench (#17202) · cfe45320
  Ekagra Ranjan authored Apr 28, 2025
  
  cfe45320
- [Model] Add tuned triton fused_moe configs for Qwen3Moe (#17328) · 8fc88d63
  Michael Goin authored Apr 28, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  8fc88d63
27 Apr, 2025 1 commit
- [Misc] Make cached tokenizer pickle-compatible (#17048) · 93a126fb
  Cyrus Leung authored Apr 27, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  93a126fb
25 Apr, 2025 3 commits
- Use Transformers helper `get_text_config()` instead of checking for `text_config` (#17105) · 423e9f1c
  Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  423e9f1c
- [Misc] Benchmark Serving Script Support Appending Results (#17028) · 881f7358
  Lucas Wilkinson authored Apr 25, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  881f7358
- [Bugfix][Misc] Use TritonPlaceholderModule to defensively import triton (#15099) · 2f540455
  Mengqing Cao authored Apr 25, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  2f540455
24 Apr, 2025 1 commit

[V1] Update structured output (#16812) · db2f8d91

Reid authored Apr 24, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

db2f8d91

23 Apr, 2025 1 commit
- [Core][V1][TPU] Enable structured decoding on TPU V1 (#16499) · 83d93371
  Chenyaaang authored Apr 22, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  83d93371
22 Apr, 2025 1 commit
- [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
  Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
  8d32dc60
21 Apr, 2025 1 commit
- Raise error for data-parallel with benchmark_throughput (#16737) · 3b34fd52
  Kartik Ramesh authored Apr 21, 2025
```
Signed-off-by: Kartik Ramesh <kartikx2000@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  3b34fd52
19 Apr, 2025 1 commit
- [Misc] Benchmarks for audio models (#16505) · 9d4ca19d
  Nicolò Lucchesi authored Apr 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  9d4ca19d
14 Apr, 2025 1 commit
- [Benchmark][Bugfix] Fix SonnetDataset default values in benchmark_throughput.py (#16556) · 63d2705e
  Jennifer Zhao authored Apr 13, 2025
  
  63d2705e
11 Apr, 2025 2 commits
- update benchmark_serving_structured_output to include auto backend (#16438) · d544d141
  Chenyaaang authored Apr 10, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  d544d141
- check input length of sonnet samples (#16423) · 3e397a94
  Alexey Belyakov authored Apr 11, 2025
```
Signed-off-by: alexey-belyakov <alexey.belyakov@intel.com>
```
  3e397a94
10 Apr, 2025 4 commits
- Fix range_ratio Bug in RandomDataset (#16126) · 268c3250
  WWW authored Apr 11, 2025
```
Signed-off-by: jadewang21 <jadewangcn@outlook.com>
```
  268c3250
- [Bugfix] Fix output token length check logic (#16419) · 7cd0bd72
  look authored Apr 11, 2025
```
Signed-off-by: look <eeslook@163.com>
```
  7cd0bd72
- [Bugfix] Fix bug when dataset is json (#15899) · 5fbab20e
  Chenyaaang authored Apr 10, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  5fbab20e
- fix sonnet dataset sample when prefix len is very small (#16379) · 417bcefb
  Chenyaaang authored Apr 09, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  417bcefb
09 Apr, 2025 2 commits
- Fix `benchmark_throughput.py --backend=hf` (#16352) · b2ce859b
  Michael Goin authored Apr 09, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b2ce859b
- [BugFix] fix some typos found by typos. (#16314) · 04149cce
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  04149cce
07 Apr, 2025 1 commit

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

55dcce91

06 Apr, 2025 1 commit
- [Benchmark] Add sampling parameters to benchmark_serving. (#16022) · ba108019
  Hyesoo Yang authored Apr 05, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  ba108019
04 Apr, 2025 1 commit

[Benchmark][Doc] Update throughput benchmark and README (#15998) · 95862f7b

Ziji Shi (Steven) authored Apr 04, 2025


Signed-off-by: StevenShi-23 <shi.ziji.sm@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

95862f7b

03 Apr, 2025 1 commit

[Benchmark] Add AIMO Dataset to Benchmark (#15955) · 06f21ce7

Ziji Shi (Steven) authored Apr 02, 2025


Signed-off-by: Ziji Shi <shi.ziji.sm@gmail.com>
Signed-off-by: StevenShi-23 <shi.ziji.sm@gmail.com>

06f21ce7

02 Apr, 2025 2 commits
- [Bugfix][Benchmarks] Ensure `async_request_deepspeed_mii` uses the OpenAI choices key (#15926) · 25293780
  Brayden Zhong authored Apr 02, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  25293780
- [Benchmark]Fix error message (#15866) · aa557e64
  Li Wang authored Apr 02, 2025
```
Signed-off-by: wangli <wangli858794774@gmail.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  aa557e64
01 Apr, 2025 1 commit
- Add option to use DeepGemm contiguous grouped gemm kernel for fused MoE operations. (#13932) · e59ca942
  bnellnm authored Apr 01, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  e59ca942
31 Mar, 2025 1 commit
- [Benchmark] Update Vision Arena Dataset and HuggingFaceDataset Setup (#15748) · effc5d24
  Jennifer Zhao authored Mar 31, 2025
```
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  effc5d24
28 Mar, 2025 2 commits
- [Minor] Remove TGI launching script (#15646) · 70e13224
  Woosuk Kwon authored Mar 28, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  70e13224
- [Misc]add coding benchmark for speculative decoding (#15303) · e7f720ea
  Chen Xia authored Mar 27, 2025
```
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
```
  e7f720ea
27 Mar, 2025 1 commit

[Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) · 9239bf71

ElizaWszola authored Mar 27, 2025


Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>

9239bf71

26 Mar, 2025 1 commit
- [Misc] Warn about v0 in benchmark_paged_attn.py (#15495) · 23114d33
  Tyler Michael Smith authored Mar 25, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  23114d33
23 Mar, 2025 1 commit
- [Misc] Add tuned R1 w8a8 and MoE configs for NVIDIA L20 (#15322) · f90d34b4
  DefTruth authored Mar 23, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  f90d34b4
20 Mar, 2025 3 commits
- [Core][V0] Add guidance backend for structured output (#14589) · 1f16b7fe
  Russell Bryant authored Mar 20, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <lohuynh@microsoft.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
```
  1f16b7fe
- [Benchmark] Allow oversample request in benchmark dataset (#15170) · b88be221
  Jennifer Zhao authored Mar 19, 2025
```
Signed-off-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>
```
  b88be221
- fix "Total generated tokens:" is 0 if using --backend tgi and --endpo… (#14673) · 40828ce5
  Wang, Yi authored Mar 20, 2025
```
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
```
  40828ce5
19 Mar, 2025 1 commit

[Misc][Benchmark] Add support for different `tokenizer_mode` (#15040) · 6c5a3195

Aaron Pham authored Mar 19, 2025


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

6c5a3195