Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
fd1c7196
Unverified
Commit
fd1c7196
authored
Aug 13, 2023
by
Hailey Schoelkopf
Committed by
GitHub
Aug 13, 2023
Browse files
Merge pull request #765 from EleutherAI/haileyschoelkopf-patch-1
Update crowspairs.py higher_is_better
parents
b952a206
9ab8538f
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
lm_eval/tasks/crowspairs.py
lm_eval/tasks/crowspairs.py
+1
-1
No files found.
lm_eval/tasks/crowspairs.py
View file @
fd1c7196
...
...
@@ -132,7 +132,7 @@ class CrowsPairsMutilingual(Task):
def
higher_is_better
(
self
):
# For all metrics lower is better
return
{
"likelihood_difference"
:
False
,
"pct_stereotype"
:
Tru
e
}
return
{
"likelihood_difference"
:
False
,
"pct_stereotype"
:
Fals
e
}
def
aggregation
(
self
):
return
{
"likelihood_difference"
:
mean
,
"pct_stereotype"
:
mean
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment