Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
6f42ab7c
Commit
6f42ab7c
authored
Aug 31, 2023
by
lintangsutawika
Browse files
detect if a task was not found
parent
ee59182d
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
3 deletions
+6
-3
main.py
main.py
+6
-3
No files found.
main.py
View file @
6f42ab7c
...
...
@@ -127,15 +127,18 @@ def main():
else
:
tasks_list
=
args
.
tasks
.
split
(
","
)
task_names
=
utils
.
pattern_match
(
tasks_list
,
ALL_TASKS
)
task_missing
=
[]
for
task
in
[
task
for
task
in
tasks_list
if
task
not
in
task_names
]:
if
os
.
path
.
isfile
(
task
):
config
=
utils
.
load_yaml_config
(
task
)
task_names
.
append
(
config
)
else
:
task_missing
.
append
(
task
)
if
task_
names
=
=
[]:
if
task_
missing
!
=
[]:
eval_logger
.
error
(
"
No t
asks were
detected,
\n
"
f
"
{
SPACING
}
Try `lm-eval -h` for list of available tasks"
,
"
T
asks were
not found: {}
\n
"
"{}Try `lm-eval -h` for list of available tasks"
.
format
(
", "
.
join
(
task_missing
),
SPACING
)
)
if
args
.
output_path
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment