Commits · 12880f1c8060bcdffd617e415a9f9cb0819b42db · gaoqiong / lm-evaluation-harness

31 Jan, 2021 1 commit
- Initial skeleton refactoring · 12880f1c
  thefazzer authored Jan 31, 2021
  
  12880f1c
30 Jan, 2021 6 commits
- Merge pull request #107 from EleutherAI/triviaqa_evaluation · 66558b35
  Leo Gao authored Jan 30, 2021
```
add evaluation for TriviaQA dataset based on loglikelihood method
```
  66558b35
- Merge branch 'master' into triviaqa_evaluation · c8032a1a
  Leo Gao authored Jan 30, 2021
  
  c8032a1a
- Handle aliases correctly in TriviaQA · b2b5a122
  Leo Gao authored Jan 30, 2021
  
  b2b5a122
- Merge pull request #104 from jon-tow/race-evaluation · 0f30237a
  Leo Gao authored Jan 30, 2021
```
Implement `RACE` evaluation
```
  0f30237a
- add evaluation for TriviaQA dataset based on loglikelihood · 0601d0bb
  Anthony DiPofi authored Jan 30, 2021
  
  0601d0bb
- Merge pull request #105 from jon-tow/dependency-updates · e48b6082
  Leo Gao authored Jan 30, 2021
```
Update dependencies
```
  e48b6082
29 Jan, 2021 12 commits
- Merge branch 'master' into race-evaluation · 48358352
  Jonathan Tow authored Jan 29, 2021
  
  48358352
- Fix versioning · 33ab4a29
  Jon Tow authored Jan 29, 2021
  
  33ab4a29
- Update dependencies · f37dcab6
  Jon Tow authored Jan 29, 2021
  
  f37dcab6
- Implement `RACE` evaluation · 6e1a76f0
  Jon Tow authored Jan 29, 2021
  
  6e1a76f0
- Merge pull request #99 from jon-tow/hellaswag-evaluation · 0f536808
  Stella Biderman authored Jan 29, 2021
```
Add `HellaSwag` evaluation implementation
```
  0f536808
- Merge pull request #103 from EleutherAI/piqa · 2d61b3ce
  Stella Biderman authored Jan 29, 2021
```
Implement PiQA
```
  2d61b3ce
- Implement PiQA · 63854c10
  Leo Gao authored Jan 29, 2021
  
  63854c10
- Remove brackets from documents · 97b88570
  Jonathan Tow authored Jan 29, 2021
  
  97b88570
- Merge pull request #101 from EleutherAI/lambada · a2f5b74b
  Stella Biderman authored Jan 29, 2021
```
Implement LAMBADA (#6)
```
  a2f5b74b
- Implement LAMBADA · 822fcc6f
  Leo Gao authored Jan 28, 2021
  
  822fcc6f
- Revert "Update `doc_to_text` to show candidate fill-ins" · da016758
  Jonathan Tow authored Jan 29, 2021
```
This reverts commit 5d431ba9.
```
  da016758
- Update `doc_to_text` to show candidate fill-ins · 5d431ba9
  Jon Tow authored Jan 28, 2021
  
  5d431ba9
28 Jan, 2021 6 commits
- Add `HellaSwag` evaluation implementation · 65c46d22
  Jon Tow authored Jan 28, 2021
  
  65c46d22
- Merge pull request #98 from zphang/wsc · 7031c324
  Leo Gao authored Jan 28, 2021
```
SGWinogradSchemaChallenge
```
  7031c324
- Add space before target · 21d527d4
  Leo Gao authored Jan 28, 2021
  
  21d527d4
- wsc · 63e32f9c
  Jason Phang authored Jan 28, 2021
  
  63e32f9c
- Merge pull request #97 from EleutherAI/unit-testing · 94d782a0
  Stella Biderman authored Jan 28, 2021
```
Implement unit testing and fix lots of problems with tasks
```
  94d782a0
- Implement unit testing and fix lots of problems with tasks · 60a6fd8c
  Leo Gao authored Jan 27, 2021
  
  60a6fd8c
27 Jan, 2021 2 commits
- Merge pull request #87 from VitamintK/master · 693c19e2
  Leo Gao authored Jan 27, 2021
```
first attempt at adding arithmetic evaluations
```
  693c19e2
- use openai's gpt3 datafiles instead of generating equations · baadb2da
  Kevin Wang authored Jan 27, 2021
  
  baadb2da
24 Jan, 2021 2 commits
- Merge pull request #91 from zphang/superglue · 38892f0a
  Leo Gao authored Jan 24, 2021
```
SuperGLUE (excluding WSC)
```
  38892f0a
- superglue ex wsc · 2e707b87
  Jason Phang authored Jan 23, 2021
  
  2e707b87
23 Jan, 2021 2 commits
- prepending a space, because it significantly increases log-likelihood of the correct answer. · 183f4721
  Kevin Wang authored Jan 14, 2021
  
  183f4721
- first attempt at adding arithmetic evaluations · a58e5e4b
  Kevin Wang authored Jan 14, 2021
  
  a58e5e4b
22 Jan, 2021 5 commits
- Merge pull request #90 from EleutherAI/no_footguns · 8ae88962
  Stella Biderman authored Jan 21, 2021
```
Get rid of some footguns
```
  8ae88962
- Remove rewrite reminder comment from everything except SuperGLUE · f4120e59
  Leo Gao authored Jan 21, 2021
  
  f4120e59
- Clean up code, remove some footguns · e31b4b31
  Leo Gao authored Jan 21, 2021
  
  e31b4b31
- Implement isgreedy · e723d3d5
  Leo Gao authored Jan 21, 2021
  
  e723d3d5
- Merge pull request #88 from jon-tow/glue-evaluation-refactor · 27a859e2
  Leo Gao authored Jan 21, 2021
```
Adopt new framework for `glue`
```
  27a859e2
21 Jan, 2021 1 commit
- Adopt new framework for `glue` · 36467c0e
  Jonathan Tow authored Jan 21, 2021
  
  36467c0e
16 Jan, 2021 3 commits
- Merge pull request #85 from EleutherAI/sglue-eval-fazz · 4c8d22db
  Leo Gao authored Jan 16, 2021
```
SuperGLUE commitmentbank implementation (new framework)
```
  4c8d22db
- Replace tab with spaces, formatting · 173b33a4
  thefazzer authored Jan 16, 2021
  
  173b33a4
- Minor formatting correction · 2c05f0e4
  thefazzer authored Jan 16, 2021
  
  2c05f0e4