Unverified Commit 3d764fe8 authored by Taras Tsugrii's avatar Taras Tsugrii Committed by GitHub
Browse files

Return early once stop token is found. (#23421)

Previously even after finding a stop token, other stop tokens were considered, which is unnecessary and slows down processing.

Currently, this unnecessary overhead is negligible since there are usually 2 stop tokens considered and they are fairly short, but in future it may become more expensive.
parent 3d3c7d42
...@@ -490,5 +490,5 @@ class HfAgent(Agent): ...@@ -490,5 +490,5 @@ class HfAgent(Agent):
# Inference API returns the stop sequence # Inference API returns the stop sequence
for stop_seq in stop: for stop_seq in stop:
if result.endswith(stop_seq): if result.endswith(stop_seq):
result = result[: -len(stop_seq)] return result[: -len(stop_seq)]
return result return result
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment