Skip to content

GitLab

Unverified Commit cee785e0 authored by KonradSzafer's avatar KonradSzafer Committed by GitHub
Browse files

limit fix (#1785)

parent 18f4eb57
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 1 deletion
lm_eval/evaluator.py
View file @ cee785e0
...@@ -592,7 +592,10 @@ def evaluate( ...@@ -592,7 +592,10 @@ def evaluate(
"n-samples": { "n-samples": {
task_output.task_name: { task_output.task_name: {
"original": len(task_output.task.eval_docs), "original": len(task_output.task.eval_docs),
"effective": min(limit, len(task_output.task.eval_docs)), "effective": min(
limit if limit else len(task_output.task.eval_docs),
len(task_output.task.eval_docs),
),
} }
for task_output in eval_tasks for task_output in eval_tasks
}, },
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment