Skip to content

GitLab

Unverified Commit 51e98e4f authored by cascade's avatar cascade Committed by GitHub
Browse files

[Bugfix] Disable prefix caching by default for benchmark (#18771) 


Signed-off-by: default avatarcascade812 <cascade812@outlook.com>
parent e56f44d9
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/benchmarks/latency.py
View file @ 51e98e4f
...@@ -82,7 +82,7 @@ def add_cli_args(parser: argparse.ArgumentParser): ...@@ -82,7 +82,7 @@ def add_cli_args(parser: argparse.ArgumentParser):
parser = EngineArgs.add_cli_args(parser) parser = EngineArgs.add_cli_args(parser)
# V1 enables prefix caching by default which skews the latency # V1 enables prefix caching by default which skews the latency
# numbers. We need to disable prefix caching by default. # numbers. We need to disable prefix caching by default.
parser.set_defaults(enable_prefix_caching=True) parser.set_defaults(enable_prefix_caching=False)
def main(args: argparse.Namespace): def main(args: argparse.Namespace):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment