Commits · 487b53134a15dc0a4f3335df4e2c76d8b8e81118 · gaoqiong / lm-evaluation-harness

"README_zh-CN.md.bak" did not exist on "13d30a4fcc5175264b9cd5609f633f481cb674a2"

13 Feb, 2021 4 commits
- Refactor `HeadQA` as a `MultipleChoiceTask` · 97174266
  Jonathan Tow authored Feb 13, 2021
  
  97174266
- Update lambada data location · e5a9de80
  Leo Gao authored Feb 12, 2021
  
  e5a9de80
- small change to regex · b07ecab4
  Anthony DiPofi authored Feb 12, 2021
  
  b07ecab4
- change mathqa to use numeric answer strings instead of a,b,c,d,e as choices · 81e42932
  Anthony DiPofi authored Feb 12, 2021
  
  81e42932
12 Feb, 2021 9 commits
- ... · e56381f4
  & authored Feb 12, 2021
  
  e56381f4
- adds missing task error · 1eb91d36
  & authored Feb 12, 2021
  
  1eb91d36
- Fixed metrics. Translation works · 61aa411b
  & authored Feb 12, 2021
  
  61aa411b
- add tasks to registry · 1297e342
  & authored Feb 12, 2021
  
  1297e342
- refactor MathQA as MultipleChoiceTask, uses full words for Q. and A. · c6c67272
  Anthony DiPofi authored Feb 12, 2021
  
  c6c67272
- translation task draft 1. Not hooked up, no metrics · bd3fcce0
  & authored Feb 12, 2021
  
  bd3fcce0
- metrics file · 1fb90b91
  & authored Feb 12, 2021
  
  1fb90b91
- Refactor `ARC` as a `MultipleChoiceTask` · 24ac76df
  Jon Tow authored Feb 11, 2021
  
  24ac76df
- add headqa and mathqa datasets · 742b5df2
  Anthony DiPofi authored Feb 11, 2021
  
  742b5df2
11 Feb, 2021 2 commits
- Fixes to make greedy_until work · 432bd44c
  Leo Gao authored Feb 10, 2021
```
# Conflicts:
#	lm_eval/models/gpt2.py
#	lm_eval/tasks/squad.py
```
  432bd44c
- Remove test set from `HellaSwag` · a2b108b9
  Jon Tow authored Feb 10, 2021
  
  a2b108b9
10 Feb, 2021 12 commits
- Refactor `HellaSwag` as a `MultipleChoiceTask` · 43ffe351
  Jon Tow authored Feb 10, 2021
  
  43ffe351
- Rename list of loglikelihoods for consistency · 155aeee9
  Jon Tow authored Feb 10, 2021
  
  155aeee9
- Fix context orderings to match label positioning · 43e274f8
  Jon Tow authored Feb 10, 2021
  
  43e274f8
- Update spacing · 8b038c2a
  Leo Gao authored Feb 09, 2021
  
  8b038c2a
- Remove incorrect context from `fewshot_context` · 3a90e246
  Jonathan Tow authored Feb 10, 2021
  
  3a90e246
- Update qa4mre.py · 5960874b
  Leo Gao authored Feb 09, 2021
  
  5960874b
- Add missing space in MultiRC · e9e5d0a2
  Leo Gao authored Feb 09, 2021
  
  e9e5d0a2
- Fix arithmetic · 42df113c
  Leo Gao authored Feb 09, 2021
  
  42df113c
- removed truncation of source · a264f449
  jeffhsu3 authored Feb 10, 2021
  
  a264f449
- Fix few-shot contexts for `Winograd Schema Challenge`-based tasks · 74f3307b
  Jonathan Tow authored Feb 09, 2021
  
  74f3307b
- Fix PiQA · 4d8ed7d5
  Leo Gao authored Feb 09, 2021
  
  4d8ed7d5
- Update RACE format · 62714855
  Leo Gao authored Feb 09, 2021
  
  62714855
09 Feb, 2021 5 commits
- Added qa4mre and changed to best_download for sciq · c05b9940
  jeffhsu3 authored Feb 09, 2021
  
  c05b9940
- More task changes · 9b14235d
  Leo Gao authored Feb 08, 2021
  
  9b14235d
- Fix · 96157fc7
  Leo Gao authored Feb 08, 2021
  
  96157fc7
- Change glue and superglue prompts · c3f724cf
  Leo Gao authored Feb 08, 2021
  
  c3f724cf
- Implement `OpenBookQA` evaluation · aa1d7293
  Jon Tow authored Feb 08, 2021
  
  aa1d7293
08 Feb, 2021 1 commit
- LAMBADA metric consistency change · dcd44d22
  Leo Gao authored Feb 07, 2021
  
  dcd44d22
06 Feb, 2021 5 commits
- Remove merge conflict · 79878d13
  Leo Gao authored Feb 06, 2021
  
  79878d13
- Fix `PIQA` prompt · c8e77519
  Jonathan Tow authored Feb 06, 2021
  
  c8e77519
- separated sciq and pubmedqa to different files · 0971cffe
  jeffhsu3 authored Feb 06, 2021
  
  0971cffe
- Fix `doc_to_text` passage bug · 715dc759
  Jonathan Tow authored Feb 05, 2021
  
  715dc759
- Remove extra space · 30c2fe23
  Leo Gao authored Feb 05, 2021
```
There shouldn't be any spaces in the beginning of `text`, only `target`
```
  30c2fe23
05 Feb, 2021 2 commits
- sciq · 620c0d26
  jeffhsu3 authored Feb 05, 2021
  
  620c0d26
- Added SciQ · c71dcb91
  jeffhsu3 authored Feb 05, 2021
  
  c71dcb91