Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
8eada53b
Commit
8eada53b
authored
Feb 03, 2021
by
Leo Gao
Browse files
Add tests for GPT2
parent
a75a9481
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
13 additions
and
0 deletions
+13
-0
tests/test_models.py
tests/test_models.py
+13
-0
No files found.
tests/test_models.py
0 → 100644
View file @
8eada53b
import
lm_eval.models
as
models
import
lm_eval.base
as
base
def
test_gpt2
():
gpt2
=
models
.
get_model
(
'gpt2'
)(
device
=
"cpu"
)
(
ll_dog
,
ig_dog
),
(
ll_cat
,
ig_cat
)
=
gpt2
.
loglikelihood
([
(
'The quick brown fox jumps over the lazy'
,
' dog'
),
(
'The quick brown fox jumps over the lazy'
,
' cat'
),
])
assert
ll_dog
>
ll_cat
assert
not
ig_cat
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment