Skip to content

GitLab

Unverified Commit 00dc9960 authored by Hailey Schoelkopf's avatar Hailey Schoelkopf Committed by GitHub
Browse files

Log which subtasks were called with which groups (#1456) 

* log group membership

* no stray prints

* Update evaluator.py
parent ba5cdf0f
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
lm_eval/evaluator.py
View file @ 00dc9960
...@@ -636,6 +636,7 @@ def evaluate( ...@@ -636,6 +636,7 @@ def evaluate(
results_dict = { results_dict = {
"results": dict(results_agg.items()), "results": dict(results_agg.items()),
**({"groups": dict(groups_agg.items())} if bool(groups_agg) else {}), **({"groups": dict(groups_agg.items())} if bool(groups_agg) else {}),
"group_subtasks": {k: v for k, v in reversed(task_hierarchy.items())},
"configs": dict(sorted(configs.items())), "configs": dict(sorted(configs.items())),
"versions": dict(sorted(versions.items())), "versions": dict(sorted(versions.items())),
"n-shot": dict(sorted(num_fewshot.items())), "n-shot": dict(sorted(num_fewshot.items())),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment