Unverified Commit fd1c7196 authored by Hailey Schoelkopf's avatar Hailey Schoelkopf Committed by GitHub
Browse files

Merge pull request #765 from EleutherAI/haileyschoelkopf-patch-1

Update crowspairs.py higher_is_better
parents b952a206 9ab8538f
......@@ -132,7 +132,7 @@ class CrowsPairsMutilingual(Task):
def higher_is_better(self):
# For all metrics lower is better
return {"likelihood_difference": False, "pct_stereotype": True}
return {"likelihood_difference": False, "pct_stereotype": False}
def aggregation(self):
return {"likelihood_difference": mean, "pct_stereotype": mean}
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment