Skip to content

GitLab

Unverified Commit 08289eaa authored by Chuyue Sun's avatar Chuyue Sun Committed by GitHub
Browse files

Support o1 model on Azure (#4980) 


Co-authored-by: default avatarShan Yu <shanyu1@g.ucla.edu>
parent 3b6d539f
Hide whitespace changes
Inline Side-by-side
Showing with 5 additions and 1 deletion
python/sglang/lang/backend/openai.py
View file @ 08289eaa
......@@ -161,7 +161,11 @@ class OpenAI(BaseBackend):
prompt = s.text_
kwargs = sampling_params.to_openai_kwargs()
if self.model_name.startswith("o1") or self.model_name.startswith("o3"):
if (
self.model_name.startswith("o1")
or self.model_name.startswith("o3")
or "o1" in self.model_name
):
kwargs.pop("max_tokens", None)
else:
kwargs.pop("max_completion_tokens", None)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment