lm_eval/tasks/drop.py · 35d809c30ca41670d457f24b9c1a2b11bcc47fa4 · gaoqiong / lm-evaluation-harness

silentv0x authored Aug 11, 2021

Following the official implementation, the following changes were made
 - Validated answers are considered gold answers
 - EM/F1 are the max over all gold answers

35d809c3

drop.py 9.33 KB

Replace drop.py