Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
4f27044a
Unverified
Commit
4f27044a
authored
Mar 06, 2025
by
Irina Yuryeva
Committed by
GitHub
Mar 06, 2025
Browse files
[Doc] Correct beam_search using in generative_models.md (#14363)
parent
0ddc991f
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
4 deletions
+6
-4
docs/source/models/generative_models.md
docs/source/models/generative_models.md
+6
-4
No files found.
docs/source/models/generative_models.md
View file @
4f27044a
...
@@ -54,14 +54,16 @@ The {class}`~vllm.LLM.beam_search` method implements [beam search](https://huggi
...
@@ -54,14 +54,16 @@ The {class}`~vllm.LLM.beam_search` method implements [beam search](https://huggi
For example, to search using 5 beams and output at most 50 tokens:
For example, to search using 5 beams and output at most 50 tokens:
```
python
```
python
from
vllm
import
LLM
from
vllm.sampling_params
import
BeamSearchParams
llm
=
LLM
(
model
=
"facebook/opt-125m"
)
llm
=
LLM
(
model
=
"facebook/opt-125m"
)
params
=
BeamSearchParams
(
beam_width
=
5
,
max_tokens
=
50
)
params
=
BeamSearchParams
(
beam_width
=
5
,
max_tokens
=
50
)
outputs
=
llm
.
generate
(
"Hello, my name is
"
,
params
)
outputs
=
llm
.
beam_search
([{
"prompt"
:
"Hello, my name is
"
}]
,
params
)
for
output
in
outputs
:
for
output
in
outputs
:
prompt
=
output
.
prompt
generated_text
=
output
.
sequences
[
0
].
text
generated_text
=
output
.
outputs
[
0
].
text
print
(
f
"Generated text:
{
generated_text
!
r
}
"
)
print
(
f
"Prompt:
{
prompt
!
r
}
, Generated text:
{
generated_text
!
r
}
"
)
```
```
### `LLM.chat`
### `LLM.chat`
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment