Commits · 155aeee93256a1109529c45bdec504dba36cf3d3 · gaoqiong / lm-evaluation-harness

10 Feb, 2021 16 commits
- Rename list of loglikelihoods for consistency · 155aeee9
  Jon Tow authored Feb 10, 2021
  
  155aeee9
- Fix context orderings to match label positioning · 43e274f8
  Jon Tow authored Feb 10, 2021
  
  43e274f8
- Merge pull request #138 from jon-tow/winograd-fixes · c6b094e0
  Leo Gao authored Feb 09, 2021
```
Fix few-shot contexts for `Winograd Schema Challenge`-based tasks
```
  c6b094e0
- Merge branch 'master' of github.com:EleutherAI/lm_evaluation_harness into winograd-fixes · 1a159d6b
  Leo Gao authored Feb 09, 2021
  
  1a159d6b
- Update spacing · 8b038c2a
  Leo Gao authored Feb 09, 2021
  
  8b038c2a
- Remove incorrect context from `fewshot_context` · 3a90e246
  Jonathan Tow authored Feb 10, 2021
  
  3a90e246
- Merge pull request #139 from jeffhsu3/pubmedqa · 7614a8f3
  Leo Gao authored Feb 09, 2021
```
QA4MRE Task
```
  7614a8f3
- Update qa4mre.py · 5960874b
  Leo Gao authored Feb 09, 2021
  
  5960874b
- Add missing space in MultiRC · e9e5d0a2
  Leo Gao authored Feb 09, 2021
  
  e9e5d0a2
- Fix arithmetic · 42df113c
  Leo Gao authored Feb 09, 2021
  
  42df113c
- Cache by default · 84f14daf
  Leo Gao authored Feb 09, 2021
  
  84f14daf
- Update gpt2 for efficiency and allow specifying model size · aab91285
  Leo Gao authored Feb 09, 2021
  
  aab91285
- removed truncation of source · a264f449
  jeffhsu3 authored Feb 10, 2021
  
  a264f449
- Fix few-shot contexts for `Winograd Schema Challenge`-based tasks · 74f3307b
  Jonathan Tow authored Feb 09, 2021
  
  74f3307b
- Fix PiQA · 4d8ed7d5
  Leo Gao authored Feb 09, 2021
  
  4d8ed7d5
- Update RACE format · 62714855
  Leo Gao authored Feb 09, 2021
  
  62714855
09 Feb, 2021 7 commits
- space in pubmedqa for merge · 18e08d29
  jeffhsu3 authored Feb 09, 2021
  
  18e08d29
- Added qa4mre and changed to best_download for sciq · c05b9940
  jeffhsu3 authored Feb 09, 2021
  
  c05b9940
- More task changes · 9b14235d
  Leo Gao authored Feb 08, 2021
  
  9b14235d
- Fix · 96157fc7
  Leo Gao authored Feb 08, 2021
  
  96157fc7
- Change glue and superglue prompts · c3f724cf
  Leo Gao authored Feb 08, 2021
  
  c3f724cf
- Merge pull request #136 from jon-tow/openbookqa-evaluation · 1050109b
  Leo Gao authored Feb 08, 2021
```
Implement `OpenBookQA` evaluation
```
  1050109b
- Implement `OpenBookQA` evaluation · aa1d7293
  Jon Tow authored Feb 08, 2021
  
  aa1d7293
08 Feb, 2021 5 commits
- LM: handle empty context · 359114fd
  Leo Gao authored Feb 07, 2021
  
  359114fd
- Add table making · 5d56a47d
  Leo Gao authored Feb 07, 2021
  
  5d56a47d
- LAMBADA metric consistency change · dcd44d22
  Leo Gao authored Feb 07, 2021
  
  dcd44d22
- Add tests for {val,test}_docs determinism · 442f843c
  Leo Gao authored Feb 07, 2021
  
  442f843c
- Fix caching · 77d4b087
  Leo Gao authored Feb 07, 2021
  
  77d4b087
06 Feb, 2021 12 commits
- Merge pull request #112 from jeffhsu3/pubmedqa · 1b467c57
  Leo Gao authored Feb 06, 2021
```
pubmedqa and sciq
```
  1b467c57
- Remove merge conflict · 79878d13
  Leo Gao authored Feb 06, 2021
  
  79878d13
- Merge pull request #130 from jon-tow/piqa-prompt-fix · 5dfbba05
  Leo Gao authored Feb 06, 2021
```
Fix `PIQA` prompt
```
  5dfbba05
- Fix `PIQA` prompt · c8e77519
  Jonathan Tow authored Feb 06, 2021
  
  c8e77519
- merge · ab41f089
  jeffhsu3 authored Feb 06, 2021
  
  ab41f089
- Merge branch 'master' of https://github.com/EleutherAI/lm-evaluation-harness into pubmedqa · 695d633d
  jeffhsu3 authored Feb 06, 2021
  
  695d633d
- separated sciq and pubmedqa to different files · 0971cffe
  jeffhsu3 authored Feb 06, 2021
  
  0971cffe
- Merge pull request #128 from jon-tow/superglue-wsc-fix · 01630657
  Leo Gao authored Feb 05, 2021
```
Fix `doc_to_text` passage bug
```
  01630657
- Merge pull request #129 from jon-tow/write-out-fix · f5f60934
  Leo Gao authored Feb 05, 2021
```
Fix `write_out.py` training set bug
```
  f5f60934
- Fix `write_out.py` training set bug · 990d5e31
  Jonathan Tow authored Feb 05, 2021
  
  990d5e31
- Fix `doc_to_text` passage bug · 715dc759
  Jonathan Tow authored Feb 05, 2021
  
  715dc759
- Remove extra space · 30c2fe23
  Leo Gao authored Feb 05, 2021
```
There shouldn't be any spaces in the beginning of `text`, only `target`
```
  30c2fe23