Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "c8ea982d9b86e145a16092017528d068a7f94630"
Unverified Commit c0935c96 authored by Woosuk Kwon's avatar Woosuk Kwon Committed by GitHub
Browse files

[Bugfix] Set enable_prefix_caching=True in prefix caching example (#3703)

parent cb40b3ab
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
examples/offline_inference_with_prefix.py
View file @ c0935c96
...@@ -22,7 +22,7 @@ prompts = [ ...@@ -22,7 +22,7 @@ prompts = [
sampling_params = SamplingParams(temperature=0.0) sampling_params = SamplingParams(temperature=0.0)
# Create an LLM. # Create an LLM.
llm = LLM(model="facebook/opt-125m") llm = LLM(model="facebook/opt-125m", enable_prefix_caching=True)
generating_prompts = [prefix + prompt for prompt in prompts] generating_prompts = [prefix + prompt for prompt in prompts]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment