Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
032820dd
Commit
032820dd
authored
Nov 17, 2023
by
haileyschoelkopf
Browse files
initialize tasks in test files
parent
c0000e03
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
4 additions
and
1 deletion
+4
-1
tests/models/test_huggingface.py
tests/models/test_huggingface.py
+2
-0
tests/test_evaluator.py
tests/test_evaluator.py
+1
-0
tests/test_tasks.py
tests/test_tasks.py
+1
-1
No files found.
tests/models/test_huggingface.py
View file @
032820dd
...
@@ -8,6 +8,8 @@ import lm_eval.tasks as tasks
...
@@ -8,6 +8,8 @@ import lm_eval.tasks as tasks
import
sys
import
sys
import
torch
import
torch
tasks
.
initialize_tasks
()
class
Test_HFLM
:
class
Test_HFLM
:
torch
.
use_deterministic_algorithms
(
True
)
torch
.
use_deterministic_algorithms
(
True
)
...
...
tests/test_evaluator.py
View file @
032820dd
...
@@ -11,6 +11,7 @@ from typing import List
...
@@ -11,6 +11,7 @@ from typing import List
import
random
import
random
import
pytest
import
pytest
tasks
.
initialize_tasks
()
# TODO: more fine grained unit tests rather than this big honking integration
# TODO: more fine grained unit tests rather than this big honking integration
# test once we break evaluator into smaller, more manageable pieces
# test once we break evaluator into smaller, more manageable pieces
...
...
tests/test_tasks.py
View file @
032820dd
...
@@ -4,7 +4,7 @@ from .utils import new_tasks
...
@@ -4,7 +4,7 @@ from .utils import new_tasks
import
lm_eval.tasks
as
tasks
import
lm_eval.tasks
as
tasks
from
lm_eval.api.task
import
ConfigurableTask
from
lm_eval.api.task
import
ConfigurableTask
tasks
.
initialize_tasks
()
# Default Task
# Default Task
TASKS
=
[
"arc_easy"
]
TASKS
=
[
"arc_easy"
]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment