Commits · 9822b06e869b849fa45850b9ea2dcd3b0365e738 · gaoqiong / lm-evaluation-harness

26 Feb, 2024 1 commit
- Apply code autoformatting with Ruff to tasks/*.py an *__init__.py (#1469) · d27c0c08
  LSinev authored Feb 26, 2024
  
  d27c0c08
20 Feb, 2024 1 commit

Add a new task GPQA (the part without CoT) (#1434) · 5ab295c8

Uanu authored Feb 21, 2024



* add new task GPQA_n_shot

* add new task GPQA_zeroshot

* correct GPQA_zeroshot filename

* Add randomly shuffle choices

* Correct missing parentheses

* delete wrong tasks

* Add README

* Update lm_eval/tasks/gpqa/zeroshot/_gpqa_zeroshot_yaml

* Update lm_eval/tasks/gpqa/n_shot/utils.py

* Update lm_eval/tasks/gpqa/n_shot/utils.py

* Update lm_eval/tasks/gpqa/README.md

* placate linter

* linter

---------
Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>

5ab295c8