Unverified Commit 89ec06c3 authored by Cade Daniel's avatar Cade Daniel Committed by GitHub
Browse files

[Docs] [Spec decode] Fix docs error in code example (#5427)

parent 9fde251b
...@@ -17,6 +17,7 @@ Speculating with a draft model ...@@ -17,6 +17,7 @@ Speculating with a draft model
The following code configures vLLM to use speculative decoding with a draft model, speculating 5 tokens at a time. The following code configures vLLM to use speculative decoding with a draft model, speculating 5 tokens at a time.
.. code-block:: python .. code-block:: python
from vllm import LLM, SamplingParams from vllm import LLM, SamplingParams
prompts = [ prompts = [
...@@ -45,6 +46,7 @@ The following code configures vLLM to use speculative decoding where proposals a ...@@ -45,6 +46,7 @@ The following code configures vLLM to use speculative decoding where proposals a
matching n-grams in the prompt. For more information read `this thread. <https://x.com/joao_gante/status/1747322413006643259>`_ matching n-grams in the prompt. For more information read `this thread. <https://x.com/joao_gante/status/1747322413006643259>`_
.. code-block:: python .. code-block:: python
from vllm import LLM, SamplingParams from vllm import LLM, SamplingParams
prompts = [ prompts = [
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment