Skip to content

GitLab

Unverified Commit 3da2313d authored by Xu Song's avatar Xu Song Committed by GitHub
Browse files

Support allowed_token_ids in ChatCompletionRequest (#19143) 


Signed-off-by: default avatarXu Song <xusong.vip@gmail.com>
parent b61dc5f9
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
vllm/entrypoints/openai/protocol.py
View file @ 3da2313d
...@@ -271,6 +271,7 @@ class ChatCompletionRequest(OpenAIBaseModel): ...@@ -271,6 +271,7 @@ class ChatCompletionRequest(OpenAIBaseModel):
spaces_between_special_tokens: bool = True spaces_between_special_tokens: bool = True
truncate_prompt_tokens: Optional[Annotated[int, Field(ge=1)]] = None truncate_prompt_tokens: Optional[Annotated[int, Field(ge=1)]] = None
prompt_logprobs: Optional[int] = None prompt_logprobs: Optional[int] = None
allowed_token_ids: Optional[list[int]] = None
# --8<-- [end:chat-completion-sampling-params] # --8<-- [end:chat-completion-sampling-params]
# --8<-- [start:chat-completion-extra-params] # --8<-- [start:chat-completion-extra-params]
...@@ -549,6 +550,7 @@ class ChatCompletionRequest(OpenAIBaseModel): ...@@ -549,6 +550,7 @@ class ChatCompletionRequest(OpenAIBaseModel):
else RequestOutputKind.FINAL_ONLY, else RequestOutputKind.FINAL_ONLY,
guided_decoding=guided_decoding, guided_decoding=guided_decoding,
logit_bias=self.logit_bias, logit_bias=self.logit_bias,
allowed_token_ids=self.allowed_token_ids,
extra_args=({"kv_transfer_params": self.kv_transfer_params} extra_args=({"kv_transfer_params": self.kv_transfer_params}
if self.kv_transfer_params else None)) if self.kv_transfer_params else None))
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment