Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
89ec06c3
Unverified
Commit
89ec06c3
authored
Jun 11, 2024
by
Cade Daniel
Committed by
GitHub
Jun 11, 2024
Browse files
[Docs] [Spec decode] Fix docs error in code example (#5427)
parent
9fde251b
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
0 deletions
+2
-0
docs/source/models/spec_decode.rst
docs/source/models/spec_decode.rst
+2
-0
No files found.
docs/source/models/spec_decode.rst
View file @
89ec06c3
...
...
@@ -17,6 +17,7 @@ Speculating with a draft model
The following code configures vLLM to use speculative decoding with a draft model, speculating 5 tokens at a time.
.. code-block:: python
from vllm import LLM, SamplingParams
prompts = [
...
...
@@ -45,6 +46,7 @@ The following code configures vLLM to use speculative decoding where proposals a
matching n-grams in the prompt. For more information read `this thread. <https://x.com/joao_gante/status/1747322413006643259>`_
.. code-block:: python
from vllm import LLM, SamplingParams
prompts = [
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment