Unverified Commit a79151c8 authored by ryan-lempka's avatar ryan-lempka Committed by GitHub
Browse files

fix: ignore stop key in vllm sampling params (#3879)


Signed-off-by: default avatarRyan Lempka <rlempka@nvidia.com>
parent 77cecf4e
......@@ -44,6 +44,9 @@ def build_sampling_params(
# Apply stop_conditions
for key, value in request["stop_conditions"].items():
if value is not None and hasattr(sampling_params, key):
# Do not add stop key to sampling params - dynamo handles stop conditions directly
if key == "stop":
continue
setattr(sampling_params, key, value)
return sampling_params
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment