Skip to content

GitLab

Commit 241aa871 authored by lintangsutawika's avatar lintangsutawika
Browse files

add where to put aggregation

parent 89c4ee5c
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
lm_eval/evaluator.py
View file @ 241aa871
...@@ -353,6 +353,8 @@ def evaluate( ...@@ -353,6 +353,8 @@ def evaluate(
vals = vals_torch vals = vals_torch
# Add Aggregation Here
if lm.rank == 0: if lm.rank == 0:
### Aggregate results over all datapoints ### ### Aggregate results over all datapoints ###
# aggregate results ; run bootstrap CIs # aggregate results ; run bootstrap CIs
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment