Skip to content

GitLab

Unverified Commit 7151f922 authored by Woosuk Kwon's avatar Woosuk Kwon Committed by GitHub
Browse files

[Misc] Fix spec decode example (#20296) 


Signed-off-by: default avatarWoosuk Kwon <woosuk.kwon@berkeley.edu>
parent e28533a1
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 2 deletions
examples/offline_inference/spec_decode.py
View file @ 7151f922
...@@ -79,9 +79,7 @@ def main(): ...@@ -79,9 +79,7 @@ def main():
trust_remote_code=True, trust_remote_code=True,
tensor_parallel_size=args.tp, tensor_parallel_size=args.tp,
enable_chunked_prefill=args.enable_chunked_prefill, enable_chunked_prefill=args.enable_chunked_prefill,
max_num_batched_tokens=args.max_num_batched_tokens,
enforce_eager=args.enforce_eager, enforce_eager=args.enforce_eager,
max_num_seqs=args.max_num_seqs,
gpu_memory_utilization=0.8, gpu_memory_utilization=0.8,
speculative_config=speculative_config, speculative_config=speculative_config,
disable_log_stats=False, disable_log_stats=False,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment