Unverified Commit 8bc46207 authored by NourFahmy's avatar NourFahmy Committed by GitHub
Browse files

Fix Anthropic API compatibility issues in chat completions (#3054)



* Fix Anthropic API compatibility issues in chat completions

solves two important compatibility issues between the LM Eval Harness and Anthropic's API:

1) The type field issue - Anthropic's Messages API doesn't accept the type field that other APIs might expect, that was previously included
2) The stop sequences issue - Anthropic requires stop sequences to contain non-whitespace characters

tested with most recent models from anthopic; claude-sonnet-4-0, claude-opus-4-0, resolved my local api errors

* pacufy pre-commit

* add type

---------
Co-authored-by: default avatarBaber <baber@hey.com>
parent 68c3a811
......@@ -323,14 +323,29 @@ class AnthropicChat(LocalCompletionsAPI):
)
if system:
messages = messages[1:]
cleaned_messages = []
for msg in messages:
cleaned_msg = {
"role": msg["role"],
"content": [
{"type": msg["type"], msg["type"]: msg["content"]},
],
}
cleaned_messages.append(cleaned_msg)
gen_kwargs.pop("do_sample", False)
max_tokens = gen_kwargs.pop("max_gen_toks", self._max_gen_toks)
temperature = gen_kwargs.pop("temperature", 0)
stop = handle_stop_sequences(gen_kwargs.pop("until", ["\n\nHuman:"]), eos=eos)
if not isinstance(stop, list):
stop = [stop]
# Filter out empty or whitespace-only stop sequences for Anthropic API
stop = [s for s in stop if s and s.strip()]
out = {
"messages": messages,
"messages": cleaned_messages,
"model": self.model,
"max_tokens": max_tokens,
"temperature": temperature,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment