Commits · d54af615d5cf63e111e564b3a2c25405fc16b4fd · OpenDAS / vllm_cscc · GitLab

29 May, 2025 1 commit
- [Misc][Tools][Benchmark] Add benchmark_serving supports for llama.cpp. (#18692) · b169d5f7
  Duyi-Wang authored May 29, 2025
```
Signed-off-by: Duyi-Wang <duyi.wang@intel.com>
```
  b169d5f7
28 May, 2025 1 commit
- [V1] fix torch profiling for V1 offline scenarios (#18445) · 774c5fde
  Divakar Verma authored May 27, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  774c5fde
27 May, 2025 2 commits
- Disable prefix cache by default for benchmark (#18639) · aaa4ac1c
  cascade authored May 27, 2025
```
Signed-off-by: cascade812 <cascade812@outlook.com>
```
  aaa4ac1c
- [Doc] cleanup deprecated flag for doc (#18715) · 4693a343
  Calvin Chen authored May 27, 2025
```
Signed-off-by: calvin chen <120380290@qq.com>
```
  4693a343
26 May, 2025 1 commit
- [Doc] Move examples and further reorganize user guide (#18666) · 82e2339b
  Cyrus Leung authored May 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  82e2339b
23 May, 2025 3 commits
- [Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454) · 4fc1bf81
  Feng XiaoLong authored May 24, 2025
```
Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>
```
  4fc1bf81
- [Bugfix][Benchmarks] Fix a benchmark of deepspeed-mii backend to use api_key (#17291) · 4be2255c
  Teruaki Ishizaki authored May 23, 2025
```
Signed-off-by: Teruaki Ishizaki <teruaki.ishizaki@ntt.com>
```
  4be2255c
- Re-submit: Fix: Proper RGBA -> RGB conversion for PIL images. (#18569) · 04eb88dc
  Chenheli Hua authored May 22, 2025
```
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
```
  04eb88dc
21 May, 2025 1 commit
- [ROCm][Kernel][V1] Enable AMD Radeon GPU Custom Paged Attention on v1 (#17004) · dd5fa7e0
  Hosang authored May 21, 2025
```
Signed-off-by: Hosang Yoon <hosang.yoon@amd.com>
```
  dd5fa7e0
16 May, 2025 1 commit
- use ceil_div in cutlass block scaling shape check (#17918) · e23564cb
  Lain authored May 16, 2025
  
  e23564cb
13 May, 2025 2 commits
- Convert `benchmarks` to `ruff format` (#18068) · 009d9e75
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  009d9e75
- [Benchmarks] Refactor run_structured_output_benchmarks.sh (#17722) · 23b3134e
  Russell Bryant authored May 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  23b3134e
12 May, 2025 1 commit
- [Fix] Benchmark `"EngineClient" has no attribute "model_config"` (#17976) · 891b9d33
  Brayden Zhong authored May 12, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  891b9d33
09 May, 2025 1 commit
- [Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362) · 0c0fdae8
  Pavani Majety authored May 09, 2025
  
  0c0fdae8
08 May, 2025 1 commit
- [Misc] support model prefix & add deepseek vl2 tiny fused moe config (#17763) · 0a9bbaa1
  xsank authored May 08, 2025
```
Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
```
  0a9bbaa1
06 May, 2025 3 commits
- [Misc] Add Next Edit Prediction (NEP) datasets support in `benchmark_serving.py` (#16839) · d456aea7
  d.transposed authored May 06, 2025
```
Signed-off-by: dtransposed <damian@damian-ml-machine.europe-west3-b.c.jetbrains-grazie.internal>
Signed-off-by: dtransposed <>
Co-authored-by: dtransposed <damian@damian-ml-machine.europe-west3-b.c.jetbrains-grazie.internal>
```
  d456aea7
- [Bugfix] Fix triton import with local TritonPlaceholder (#17446) · f9bc5a06
  Mengqing Cao authored May 06, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  f9bc5a06
- [Bugfix] Fixed prompt length for random dataset (#17408) · dc47ba32
  Mikhail Podvitskii authored May 06, 2025
```
Signed-off-by: Mikhail Podvitskii <podvitskiymichael@gmail.com>
```
  dc47ba32
05 May, 2025 1 commit
- [Benchmarks] Remove invalid option under V1 engine (#17651) · d3efde81
  Russell Bryant authored May 05, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  d3efde81
02 May, 2025 2 commits
- [Hardware][AMD] Improve OAM device ID + llama4 Maverick MOE tuning (#16263) · 9352cdb5
  Xiaodong Wang authored May 02, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
Co-authored-by: Lu Fang <lufang@fb.com>
```
  9352cdb5
- permute/unpermute kernel for moe optimization (#14568) · 3e887d2e
  Caleb_Du authored May 03, 2025
```
Signed-off-by: Caleb_Du <Caleb_Du@zju.edu.cn>
```
  3e887d2e
01 May, 2025 2 commits
- [Misc][Tools][Benchmark] Publish script to auto tune server parameters (#17207) · 9b70e2b4
  Chenyaaang authored May 01, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  9b70e2b4
- [Bugfix][Benchmarks] Allow benchmark of deepspeed-mii backend to select a model (#17285) · 86a1f67a
  Teruaki Ishizaki authored May 01, 2025
```
Signed-off-by: Teruaki Ishizaki <teruaki.ishizaki@ntt.com>
```
  86a1f67a
30 Apr, 2025 1 commit
- [V1][Feature] Enable Speculative Decoding with Structured Outputs (#14702) · 34120f5a
  Benjamin Chislett authored Apr 29, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
```
  34120f5a
28 Apr, 2025 2 commits
- [Benchmark] Add single turn MTBench to Serving Bench (#17202) · cfe45320
  Ekagra Ranjan authored Apr 28, 2025
  
  cfe45320
- [Model] Add tuned triton fused_moe configs for Qwen3Moe (#17328) · 8fc88d63
  Michael Goin authored Apr 28, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  8fc88d63
27 Apr, 2025 1 commit
- [Misc] Make cached tokenizer pickle-compatible (#17048) · 93a126fb
  Cyrus Leung authored Apr 27, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  93a126fb
25 Apr, 2025 3 commits
- Use Transformers helper `get_text_config()` instead of checking for `text_config` (#17105) · 423e9f1c
  Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  423e9f1c
- [Misc] Benchmark Serving Script Support Appending Results (#17028) · 881f7358
  Lucas Wilkinson authored Apr 25, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  881f7358
- [Bugfix][Misc] Use TritonPlaceholderModule to defensively import triton (#15099) · 2f540455
  Mengqing Cao authored Apr 25, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  2f540455
24 Apr, 2025 1 commit

[V1] Update structured output (#16812) · db2f8d91

Reid authored Apr 24, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

db2f8d91

23 Apr, 2025 1 commit
- [Core][V1][TPU] Enable structured decoding on TPU V1 (#16499) · 83d93371
  Chenyaaang authored Apr 22, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  83d93371
22 Apr, 2025 1 commit
- [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
  Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
  8d32dc60
21 Apr, 2025 1 commit
- Raise error for data-parallel with benchmark_throughput (#16737) · 3b34fd52
  Kartik Ramesh authored Apr 21, 2025
```
Signed-off-by: Kartik Ramesh <kartikx2000@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  3b34fd52
19 Apr, 2025 1 commit
- [Misc] Benchmarks for audio models (#16505) · 9d4ca19d
  Nicolò Lucchesi authored Apr 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  9d4ca19d
14 Apr, 2025 1 commit
- [Benchmark][Bugfix] Fix SonnetDataset default values in benchmark_throughput.py (#16556) · 63d2705e
  Jennifer Zhao authored Apr 13, 2025
  
  63d2705e
11 Apr, 2025 2 commits
- update benchmark_serving_structured_output to include auto backend (#16438) · d544d141
  Chenyaaang authored Apr 10, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  d544d141
- check input length of sonnet samples (#16423) · 3e397a94
  Alexey Belyakov authored Apr 11, 2025
```
Signed-off-by: alexey-belyakov <alexey.belyakov@intel.com>
```
  3e397a94
10 Apr, 2025 2 commits
- Fix range_ratio Bug in RandomDataset (#16126) · 268c3250
  WWW authored Apr 11, 2025
```
Signed-off-by: jadewang21 <jadewangcn@outlook.com>
```
  268c3250
- [Bugfix] Fix output token length check logic (#16419) · 7cd0bd72
  look authored Apr 11, 2025
```
Signed-off-by: look <eeslook@163.com>
```
  7cd0bd72