Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
d3c002ea
Unverified
Commit
d3c002ea
authored
Aug 22, 2024
by
Brian Li
Committed by
GitHub
Aug 21, 2024
Browse files
[Bugfix] chat method add_generation_prompt param (#7734)
parent
9b73a2f4
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
3 deletions
+3
-3
vllm/entrypoints/llm.py
vllm/entrypoints/llm.py
+3
-3
No files found.
vllm/entrypoints/llm.py
View file @
d3c002ea
...
...
@@ -353,7 +353,7 @@ class LLM:
use_tqdm
:
bool
=
True
,
lora_request
:
Optional
[
LoRARequest
]
=
None
,
chat_template
:
Optional
[
str
]
=
None
,
add_generation_
template
:
bool
=
True
,
add_generation_
prompt
:
bool
=
True
,
)
->
List
[
RequestOutput
]:
"""
Generates responses for chat messages.
...
...
@@ -374,7 +374,7 @@ class LLM:
lora_request: LoRA request to use for generation, if any.
chat_template: The template to use for structuring the chat.
If not provided, the model's default chat template will be used.
add_generation_
template
: If True, adds a generation template
add_generation_
prompt
: If True, adds a generation template
to each message.
Returns:
...
...
@@ -392,7 +392,7 @@ class LLM:
tokenizer
,
conversations
,
chat_template
=
chat_template
,
add_generation_
template
=
add_generation_
template
)
add_generation_
prompt
=
add_generation_
prompt
)
return
self
.
generate
(
prompts
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment