Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
59c8cf02
Commit
59c8cf02
authored
May 27, 2025
by
Baber
Browse files
Add option to list all tasks in list_all_tasks method
parent
51b67365
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
12 additions
and
1 deletion
+12
-1
lm_eval/tasks/__init__.py
lm_eval/tasks/__init__.py
+12
-1
No files found.
lm_eval/tasks/__init__.py
View file @
59c8cf02
...
...
@@ -106,7 +106,7 @@ class TaskManager:
return
self
.
_task_index
def
list_all_tasks
(
self
,
list_groups
=
True
,
list_tags
=
True
,
list_subtasks
=
True
self
,
list_groups
=
True
,
list_tags
=
True
,
list_subtasks
=
True
,
listall
=
False
)
->
str
:
from
pytablewriter
import
MarkdownTableWriter
...
...
@@ -162,6 +162,13 @@ class TaskManager:
st_values
.
append
([
t
,
path
,
output_type
])
subtask_table
.
value_matrix
=
st_values
if
listall
:
return
(
group_table
.
value_matrix
,
tag_table
.
value_matrix
,
subtask_table
.
value_matrix
,
)
result
=
"
\n
"
if
list_groups
:
result
+=
group_table
.
dumps
()
+
"
\n\n
"
...
...
@@ -171,6 +178,10 @@ class TaskManager:
result
+=
subtask_table
.
dumps
()
+
"
\n\n
"
return
result
@
property
def
get_task_lists
(
self
):
return
self
.
list_all_tasks
(
listall
=
True
)
def
match_tasks
(
self
,
task_list
:
list
[
str
])
->
list
[
str
]:
return
utils
.
pattern_match
(
task_list
,
self
.
all_tasks
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment