Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
59aff21d
Commit
59aff21d
authored
May 10, 2021
by
Leo Gao
Browse files
Fix tests
parent
02249855
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
2 additions
and
10 deletions
+2
-10
lm_eval/metrics.py
lm_eval/metrics.py
+1
-1
tests/test_misc.py
tests/test_misc.py
+1
-9
No files found.
lm_eval/metrics.py
View file @
59aff21d
...
...
@@ -174,7 +174,7 @@ def _sacreformat(refs, preds):
## stderr stuff
def
bootstrap_stderr
(
f
,
xs
,
iters
=
10000
):
def
bootstrap_stderr
(
f
,
xs
,
iters
=
10000
0
):
rnd
=
random
.
Random
()
rnd
.
seed
(
42
)
res
=
[]
...
...
tests/test_misc.py
View file @
59aff21d
...
...
@@ -5,16 +5,8 @@ import random
def
test_bootstrapping
():
random
.
seed
(
42
)
arr
=
[
random
.
random
()
for
_
in
range
(
100
)]
arr
=
[
random
.
random
()
for
_
in
range
(
100
0
)]
expected
=
metrics
.
mean_stderr
(
arr
)
bootstrapped
=
metrics
.
bootstrap_stderr
(
metrics
.
mean
,
arr
,
iters
=
100000
)
assert
bootstrapped
==
pytest
.
approx
(
expected
,
abs
=
1e-4
)
def
test_bootstrapping_stella
():
arr
=
[
0.1
,
0.3
,
0.2
,
0.25
,
0.3
,
0.1
,
0.22
]
expected
=
metrics
.
mean_stderr
(
arr
)
bootstrapped
=
metrics
.
bootstrap_stderr
(
metrics
.
mean
,
arr
,
iters
=
100000
)
assert
bootstrapped
==
pytest
.
approx
(
expected
,
abs
=
1e-5
)
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment