Unverified Commit b1340f9d authored by Robert Shaw's avatar Robert Shaw Committed by GitHub
Browse files

[V1] Bugfix: Validate Model Input Length (#12600)

SUMMARY:
* avoid crashing the engine when we get an input longer than
max_model_len

FIX #12567(*link existing issues this PR will resolve*)
parent 44bbca78
......@@ -206,6 +206,11 @@ class Processor:
if prompt_ids is None or len(prompt_ids) == 0:
raise ValueError("Prompt cannot be empty")
if len(prompt_ids) >= self.model_config.max_model_len:
raise ValueError(
f"Prompt length of {len(prompt_ids)} is longer than the "
f"maximum model length of {self.model_config.max_model_len}.")
if self.model_config.is_multimodal_model:
max_prompt_len = self.model_config.max_model_len
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment