fixed args input in aggregate_subtask_metrics

9551bbf2 · lintangsutawika · 4dd69062 · 9551bbf2
Commit 9551bbf2 authored Apr 25, 2024 by lintangsutawika
Show whitespace changes
Inline Side-by-side

Showing with 2 additions and 1 deletion

lm_eval/evaluator.py lm_eval/evaluator.py +2 -1

No files found.
--- a/lm_eval/evaluator.py
+++ b/lm_eval/evaluator.py
@@ -535,7 +535,8 @@ def evaluate(
                        metric
                    ] = lm_eval.api.metrics.aggregate_subtask_metrics(
                        metrics,
-                        sizes if group_config["weight_by_size"] else [1] * len(sizes),
+                        sizes,
+                        group_config["weight_by_size"],
                    )
                    # TODO: calculate grouped metric using aggregation fn
                    if "N/A" in stderrs: