Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
f9eca2c8
Unverified
Commit
f9eca2c8
authored
Jan 18, 2023
by
Hailey Schoelkopf
Committed by
GitHub
Jan 18, 2023
Browse files
Add accuracy metric to crows-pairs (#380)
* add accuracy metric to crows-pairs
parent
ea3df930
Changes
23
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
3 additions
and
3 deletions
+3
-3
tests/testdata/crows_pairs_french_religion-v0-res.json
tests/testdata/crows_pairs_french_religion-v0-res.json
+1
-1
tests/testdata/crows_pairs_french_sexual_orientation-v0-res.json
...estdata/crows_pairs_french_sexual_orientation-v0-res.json
+1
-1
tests/testdata/crows_pairs_french_socioeconomic-v0-res.json
tests/testdata/crows_pairs_french_socioeconomic-v0-res.json
+1
-1
No files found.
tests/testdata/crows_pairs_french_religion-v0-res.json
View file @
f9eca2c8
{
"results"
:
{
"crows_pairs_french_religion"
:
{
"likelihood_difference"
:
0.32691651640972225
,
"likelihood_difference_stderr"
:
0.021833493193249474
}},
"versions"
:
{
"crows_pairs_french_religion"
:
0
}}
{
"results"
:
{
"crows_pairs_french_religion"
:
{
"likelihood_difference"
:
0.32691651640972225
,
"likelihood_difference_stderr"
:
0.021833493193249474
,
"pct_stereotype"
:
0.45217391304347826
,
"pct_stereotype_stderr"
:
0.046614569799583463
}},
"versions"
:
{
"crows_pairs_french_religion"
:
0
}}
\ No newline at end of file
\ No newline at end of file
tests/testdata/crows_pairs_french_sexual_orientation-v0-res.json
View file @
f9eca2c8
{
"results"
:
{
"crows_pairs_french_sexual_orientation"
:
{
"likelihood_difference"
:
0.3160680928470684
,
"likelihood_difference_stderr"
:
0.02397758321605678
}},
"versions"
:
{
"crows_pairs_french_sexual_orientation"
:
0
}}
{
"results"
:
{
"crows_pairs_french_sexual_orientation"
:
{
"likelihood_difference"
:
0.3160680928470684
,
"likelihood_difference_stderr"
:
0.02397758321605678
,
"pct_stereotype"
:
0.43956043956043955
,
"pct_stereotype_stderr"
:
0.05231815698566189
}},
"versions"
:
{
"crows_pairs_french_sexual_orientation"
:
0
}}
\ No newline at end of file
\ No newline at end of file
tests/testdata/crows_pairs_french_socioeconomic-v0-res.json
View file @
f9eca2c8
{
"results"
:
{
"crows_pairs_french_socioeconomic"
:
{
"likelihood_difference"
:
0.3394681494647815
,
"likelihood_difference_stderr"
:
0.01702488895584347
}},
"versions"
:
{
"crows_pairs_french_socioeconomic"
:
0
}}
{
"results"
:
{
"crows_pairs_french_socioeconomic"
:
{
"likelihood_difference"
:
0.3394681494647815
,
"likelihood_difference_stderr"
:
0.01702488895584347
,
"pct_stereotype"
:
0.4642857142857143
,
"pct_stereotype_stderr"
:
0.035714285714285705
}},
"versions"
:
{
"crows_pairs_french_socioeconomic"
:
0
}}
\ No newline at end of file
\ No newline at end of file
Prev
1
2
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment