Skip to content

GitLab

Unverified Commit 1f21b8ef authored by Lintang Sutawika's avatar Lintang Sutawika Committed by GitHub
Browse files

Update evaluator.py

parent 777f57ce
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
lm_eval/evaluator.py
View file @ 1f21b8ef
...@@ -195,7 +195,7 @@ def evaluate( ...@@ -195,7 +195,7 @@ def evaluate(
samples = collections.defaultdict(list) samples = collections.defaultdict(list)
# tracks all Instances/requests a model must generate output on. # tracks all Instances/requests a model must generate output on.
requests = collections.defaultdict(list) requests = collections.defaultdict(list)
# ? # Stores task scores based on task grouping.
aggregate = collections.defaultdict(dict) aggregate = collections.defaultdict(dict)
# tracks if a task was chosen via user selecting a group containing it # tracks if a task was chosen via user selecting a group containing it
task_groups = collections.defaultdict(dict) task_groups = collections.defaultdict(dict)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment