Commits · eeb135eb87277b388e1a2c97ff9c13fcd94fc6ae · OpenDAS / vllm_cscc

17 Sep, 2025 1 commit

[FP8] Extend per-token-group quantization support to QuantFP8 (#24342) · cef32104

Tahsin Tunan authored Sep 17, 2025

Signed-off-by: Tahsin Tunan <tahsintunan@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

cef32104

16 Sep, 2025 2 commits
- [Docs] move benchmarks README to contributing guides (#24820) · 85e0df13
  Ye (Charlotte) Qi authored Sep 16, 2025
  
  85e0df13
- [benchmark] Add triton version in the moe tuned config (#24769) · 04ad0dc2
  Jee Jee Li authored Sep 16, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  04ad0dc2
13 Sep, 2025 1 commit
- [Kernels][DP/EP] Optimize Silu Kernel for R1 (#24054) · 98229db2
  Elvir Crnčević authored Sep 13, 2025
```
Signed-off-by: elvircrn <elvircrn@gmail.com>
```
  98229db2
12 Sep, 2025 1 commit
- [Doc]: fix typos in various files (#24726) · bcb06d7b
  Didier Durand authored Sep 12, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  bcb06d7b
11 Sep, 2025 4 commits

[Perf] Use upstream CUTLASS for SM90 Block FP8 kernel (#23280) · c3aea10d

Michael Goin authored Sep 11, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

c3aea10d

[Kernels] Enable Torch Symmetric Memory All-Reduce By Default (#24111) · 1fdd5c42
Ilya Markov authored Sep 11, 2025
```
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
1fdd5c42
[Bench] Add qwen-next in benchmark_moe.py (#24661) · d11ec124
Jee Jee Li authored Sep 11, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
d11ec124

[Model] New model support for Motif-1-Tiny (#23414) · 9bd831f5

TaehyunKim authored Sep 11, 2025


Signed-off-by: ca1207 <ca1207zzz@gmail.com>
Signed-off-by: TaehyunKim <73943231+ca1207@users.noreply.github.com>
Co-authored-by: WyldeCat <skan1543@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

9bd831f5

09 Sep, 2025 3 commits
- [Benchmark] Update bench doc with mtbench, blazedit, spec bench (#24450) · 0dc9cbb5
  Ekagra Ranjan authored Sep 09, 2025
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
```
  0dc9cbb5
- [CI/Build][Doc] Fully deprecate old bench scripts for serving / throughput / latency (#24411) · 6fb27881
  Ye (Charlotte) Qi authored Sep 09, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  6fb27881
- [Flashinfer] Support Flashinfer TRTLLM FP8-qkv BF16/FP16-out Attention Kernel (#23647) · bba1042c
  elvischenv authored Sep 09, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  bba1042c
07 Sep, 2025 2 commits

[Bugfix] Fix Qwen3-coder moe tuned config (#24072) · 62f66be1
Jee Jee Li authored Sep 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
62f66be1

[Benchmark] add benchmark for custom activation op (#23908) · 77aec83b

Jiangyun Zhu authored Sep 07, 2025


Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

77aec83b

04 Sep, 2025 3 commits

[Doc]: fix typos in Python comments (#24173) · 83609ca9

Didier Durand authored Sep 04, 2025


Signed-off-by: Didier Durand <durand.didier@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

83609ca9

Improve flexibility of auto_tune.sh execution. (#23766) · 04f3c35c

anthonsu authored Sep 04, 2025

Signed-off-by: Anthony Su <50185138+anthonsu@users.noreply.github.com>
Signed-off-by: anthonsu <50185138+anthonsu@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

04f3c35c

[Misc] Enhance output readability of helper script (#24214) · 12e1e63c
Weida Hong authored Sep 04, 2025
```
Signed-off-by: Weida Hong <wdhongtw@google.com>
```
12e1e63c

03 Sep, 2025 3 commits
- Remove deprecated `PyNcclConnector` (#24151) · b5ee1e32
  Peter Pan authored Sep 04, 2025
```
Signed-off-by: Peter Pan <Peter.Pan@daocloud.io>
```
  b5ee1e32
- [Doc]: fix typos in Python comments (#24115) · 02d411fd
  Didier Durand authored Sep 03, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  02d411fd
- fix some typos (#24071) · 1bd007f2
  co63oc authored Sep 03, 2025
```
Signed-off-by: co63oc <co63oc@users.noreply.github.com>
```
  1bd007f2
02 Sep, 2025 1 commit
- [Benchmark] Add support for local hf dataset path in benchmark (#23999) · c83c4ff8
  Jiangyun Zhu authored Sep 03, 2025
```
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
```
  c83c4ff8
29 Aug, 2025 1 commit
- Tuned H100/H200 triton fp8 block configs for fused_qkv_a_proj (#23939) · b7adf94c
  Michael Goin authored Aug 29, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b7adf94c
28 Aug, 2025 2 commits
- [Bugfix] Fix benchmark_moe.py for blockwise fp8. (#23823) · 66548f66
  YUQI.CHENG authored Aug 28, 2025
```
Signed-off-by: crischeng <420985011@qq.com>
Co-authored-by: cris <grace@guisenbindeMacBook-Pro.local>
```
  66548f66
- [Perf] Tune configs for triton block fp8 gemm H100/H200 (#23748) · a781e84e
  Michael Goin authored Aug 27, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a781e84e
27 Aug, 2025 1 commit
- [Frontend] Optimize beam search performance by limiting concurrency (#23599) · 142ac080
  Chen Zhang authored Aug 26, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  142ac080
26 Aug, 2025 1 commit
- [Doc]: fix various spelling issues in multiple files (#23636) · 7c04779a
  Didier Durand authored Aug 26, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  7c04779a
25 Aug, 2025 2 commits
- Feature/benchmark/random mm data/images (#23119) · 0cb7b065
  Breno Baldas Skuk authored Aug 25, 2025
```
Signed-off-by: breno.skuk <breno.skuk@hcompany.ai>
```
  0cb7b065
- [Perf] Add Triton config for DeepSeek V3 FP8 EP32 H200 (#23504) · 504d9143
  Ming Yang authored Aug 24, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  504d9143
24 Aug, 2025 1 commit
- [kernel] Support W4A8 on Hopper (#23198) · e76e2335
  czhu-cohere authored Aug 24, 2025
```
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
```
  e76e2335
22 Aug, 2025 1 commit

[NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) · 24d0c9e6

elvischenv authored Aug 23, 2025


Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

24d0c9e6

21 Aug, 2025 3 commits
- [Perf] Small optimizations for silu_mul_fp8_quant_deep_gemm (#23265) · 3bbe11cc
  Michael Goin authored Aug 21, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  3bbe11cc
- [Core] Always use tensor cores for Flashinfer Decode Wrapper (#23214) · 1d353b63
  Pavani Majety authored Aug 21, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  1d353b63
- [Bugfix] Fix extra whitespace in strings caused by newline (#23272) · 0c31e28e
  Cyrus Leung authored Aug 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0c31e28e
20 Aug, 2025 3 commits
- [Kernel/Quant] Remove the original marlin format and qqq (#23204) · 0cdbf5e6
  Michael Goin authored Aug 20, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  0cdbf5e6
- [Kernel] CUTLASS MoE FP8: Integrate cuda moe permute/unpermute (#23045) · b17109be
  shixianc authored Aug 20, 2025
```
Signed-off-by: Shixian Cui <shixian@amazon.com>
```
  b17109be
- [CI/Build] Also check DP in benchmarks throughput script (#23038) · f7290232
  Zhewen Li authored Aug 19, 2025
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  f7290232
19 Aug, 2025 4 commits

[Benchmarks] Add video inputs to ShareGPTDataset. (#23199) · 1630cc8d
Chenheli Hua authored Aug 19, 2025
```
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
```
1630cc8d

[Misc] Fix the benchmark's README and improve the error messages for the... · 03d4235f

Ruixiang Tan authored Aug 20, 2025


[Misc] Fix the benchmark's README and improve the error messages for the benchmark's argument checks (#22654)
Signed-off-by: tanruixiang <tanruixiang0104@gmail.com>

03d4235f

[Bugfix] Fix benchmark_moe.py (#23177) · 4d9c6199
Jee Jee Li authored Aug 19, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
4d9c6199

[NVIDIA] Support Flashinfer TRTLLM FP8-q/kv/out Attention Kernel (#21716) · 03752dba

elvischenv authored Aug 19, 2025

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

03752dba