Skip to content

GitLab

Commit 3d4c4cd6 authored by Herbie Bradley's avatar Herbie Bradley
Browse files

Add evaluator temp code for debug

parent be95d945
Show whitespace changes
Inline Side-by-side
Showing with 24 additions and 23 deletions
lm_eval/evaluator.py
View file @ 3d4c4cd6
...@@ -428,6 +428,7 @@ def evaluate( ...@@ -428,6 +428,7 @@ def evaluate(
torch_device_tensor = lm.accelerator.pad_across_processes( torch_device_tensor = lm.accelerator.pad_across_processes(
metrics_tensor.to(torch.float32), pad_index=pad_value metrics_tensor.to(torch.float32), pad_index=pad_value
) )
gathered_item = lm.accelerator.gather(torch_device_tensor) gathered_item = lm.accelerator.gather(torch_device_tensor)
if numitem > 0: if numitem > 0:
gathered_filtered = gathered_item[gathered_item[:, 0] != pad_value] gathered_filtered = gathered_item[gathered_item[:, 0] != pad_value]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment