Unverified Commit 4f27044a authored by Irina Yuryeva's avatar Irina Yuryeva Committed by GitHub
Browse files

[Doc] Correct beam_search using in generative_models.md (#14363)

parent 0ddc991f
......@@ -54,14 +54,16 @@ The {class}`~vllm.LLM.beam_search` method implements [beam search](https://huggi
For example, to search using 5 beams and output at most 50 tokens:
```python
from vllm import LLM
from vllm.sampling_params import BeamSearchParams
llm = LLM(model="facebook/opt-125m")
params = BeamSearchParams(beam_width=5, max_tokens=50)
outputs = llm.generate("Hello, my name is", params)
outputs = llm.beam_search([{"prompt": "Hello, my name is "}], params)
for output in outputs:
prompt = output.prompt
generated_text = output.outputs[0].text
print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
generated_text = output.sequences[0].text
print(f"Generated text: {generated_text!r}")
```
### `LLM.chat`
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment