Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
23114d33
Unverified
Commit
23114d33
authored
Mar 25, 2025
by
Tyler Michael Smith
Committed by
GitHub
Mar 25, 2025
Browse files
[Misc] Warn about v0 in benchmark_paged_attn.py (#15495)
Signed-off-by:
Tyler Michael Smith
<
tyler@neuralmagic.com
>
parent
997c8811
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
0 deletions
+6
-0
benchmarks/kernels/benchmark_paged_attention.py
benchmarks/kernels/benchmark_paged_attention.py
+6
-0
No files found.
benchmarks/kernels/benchmark_paged_attention.py
View file @
23114d33
...
@@ -7,10 +7,13 @@ from typing import Optional
...
@@ -7,10 +7,13 @@ from typing import Optional
import
torch
import
torch
from
vllm
import
_custom_ops
as
ops
from
vllm
import
_custom_ops
as
ops
from
vllm.logger
import
init_logger
from
vllm.platforms
import
current_platform
from
vllm.platforms
import
current_platform
from
vllm.utils
import
(
STR_DTYPE_TO_TORCH_DTYPE
,
FlexibleArgumentParser
,
from
vllm.utils
import
(
STR_DTYPE_TO_TORCH_DTYPE
,
FlexibleArgumentParser
,
create_kv_caches_with_random
)
create_kv_caches_with_random
)
logger
=
init_logger
(
__name__
)
NUM_BLOCKS
=
128
*
1024
NUM_BLOCKS
=
128
*
1024
PARTITION_SIZE
=
512
PARTITION_SIZE
=
512
PARTITION_SIZE_ROCM
=
256
PARTITION_SIZE_ROCM
=
256
...
@@ -193,6 +196,9 @@ def main(
...
@@ -193,6 +196,9 @@ def main(
if
__name__
==
'__main__'
:
if
__name__
==
'__main__'
:
logger
.
warning
(
"This script benchmarks the paged attention kernel. "
"By default this is no longer used in vLLM inference."
)
parser
=
FlexibleArgumentParser
(
parser
=
FlexibleArgumentParser
(
description
=
"Benchmark the paged attention kernel."
)
description
=
"Benchmark the paged attention kernel."
)
parser
.
add_argument
(
"--version"
,
parser
.
add_argument
(
"--version"
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment