Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
d3c002ea
Unverified
Commit
d3c002ea
authored
Aug 22, 2024
by
Brian Li
Committed by
GitHub
Aug 21, 2024
Browse files
[Bugfix] chat method add_generation_prompt param (#7734)
parent
9b73a2f4
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
3 deletions
+3
-3
vllm/entrypoints/llm.py
vllm/entrypoints/llm.py
+3
-3
No files found.
vllm/entrypoints/llm.py
View file @
d3c002ea
...
@@ -353,7 +353,7 @@ class LLM:
...
@@ -353,7 +353,7 @@ class LLM:
use_tqdm
:
bool
=
True
,
use_tqdm
:
bool
=
True
,
lora_request
:
Optional
[
LoRARequest
]
=
None
,
lora_request
:
Optional
[
LoRARequest
]
=
None
,
chat_template
:
Optional
[
str
]
=
None
,
chat_template
:
Optional
[
str
]
=
None
,
add_generation_
template
:
bool
=
True
,
add_generation_
prompt
:
bool
=
True
,
)
->
List
[
RequestOutput
]:
)
->
List
[
RequestOutput
]:
"""
"""
Generates responses for chat messages.
Generates responses for chat messages.
...
@@ -374,7 +374,7 @@ class LLM:
...
@@ -374,7 +374,7 @@ class LLM:
lora_request: LoRA request to use for generation, if any.
lora_request: LoRA request to use for generation, if any.
chat_template: The template to use for structuring the chat.
chat_template: The template to use for structuring the chat.
If not provided, the model's default chat template will be used.
If not provided, the model's default chat template will be used.
add_generation_
template
: If True, adds a generation template
add_generation_
prompt
: If True, adds a generation template
to each message.
to each message.
Returns:
Returns:
...
@@ -392,7 +392,7 @@ class LLM:
...
@@ -392,7 +392,7 @@ class LLM:
tokenizer
,
tokenizer
,
conversations
,
conversations
,
chat_template
=
chat_template
,
chat_template
=
chat_template
,
add_generation_
template
=
add_generation_
template
)
add_generation_
prompt
=
add_generation_
prompt
)
return
self
.
generate
(
return
self
.
generate
(
prompts
,
prompts
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment