Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
27208be6
Unverified
Commit
27208be6
authored
Jun 04, 2024
by
Woosuk Kwon
Committed by
GitHub
Jun 04, 2024
Browse files
[Kernel] Add back batch size 1536 and 3072 to MoE tuning (#5242)
parent
87d5abef
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
1 deletion
+3
-1
benchmarks/kernels/benchmark_moe.py
benchmarks/kernels/benchmark_moe.py
+3
-1
No files found.
benchmarks/kernels/benchmark_moe.py
View file @
27208be6
...
@@ -254,7 +254,9 @@ def main(args: argparse.Namespace):
...
@@ -254,7 +254,9 @@ def main(args: argparse.Namespace):
use_fp8
=
args
.
dtype
==
"fp8"
use_fp8
=
args
.
dtype
==
"fp8"
if
args
.
batch_size
is
None
:
if
args
.
batch_size
is
None
:
batch_sizes
=
[
1
,
2
,
4
,
8
,
16
,
32
,
64
,
128
,
256
,
512
,
1024
,
2048
,
4096
]
batch_sizes
=
[
1
,
2
,
4
,
8
,
16
,
32
,
64
,
128
,
256
,
512
,
1024
,
1536
,
2048
,
3072
,
4096
]
else
:
else
:
batch_sizes
=
[
args
.
batch_size
]
batch_sizes
=
[
args
.
batch_size
]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment