Commits · 602d3e204130e06ff3b77c1618d3d461803792c9 · gaoqiong / lm-evaluation-harness

06 Feb, 2021 6 commits
- Merge remote-tracking branch 'origin/master' into fazz/refactor-task-coqa · 602d3e20
  thefazzer authored Feb 06, 2021
  
  602d3e20
- HFTask -> Task, line break correction · 222027b9
  thefazzer authored Feb 06, 2021
  
  222027b9
- Merge pull request #128 from jon-tow/superglue-wsc-fix · 01630657
  Leo Gao authored Feb 05, 2021
```
Fix `doc_to_text` passage bug
```
  01630657
- Merge pull request #129 from jon-tow/write-out-fix · f5f60934
  Leo Gao authored Feb 05, 2021
```
Fix `write_out.py` training set bug
```
  f5f60934
- Fix `write_out.py` training set bug · 990d5e31
  Jonathan Tow authored Feb 05, 2021
  
  990d5e31
- Fix `doc_to_text` passage bug · 715dc759
  Jonathan Tow authored Feb 05, 2021
  
  715dc759
05 Feb, 2021 13 commits
- Add task table · c0fbf9e8
  Leo Gao authored Feb 04, 2021
  
  c0fbf9e8
- Add test to make sure right space conventions are used · 049dfa34
  Leo Gao authored Feb 04, 2021
  
  049dfa34
- Fix SAT · aea63162
  Leo Gao authored Feb 04, 2021
  
  aea63162
- Convert SAT to use MultipleChoiceTask · d7c08d5b
  Leo Gao authored Feb 04, 2021
  
  d7c08d5b
- Add MultipleChoiceTest target · 2f5f42c6
  Leo Gao authored Feb 04, 2021
  
  2f5f42c6
- Add sqlitedict dependency · 7422d593
  Leo Gao authored Feb 04, 2021
  
  7422d593
- Add MultipleChoiceTask · 706cb53a
  Leo Gao authored Feb 04, 2021
  
  706cb53a
- Merge branch 'master' of github.com:EleutherAI/lm_evaluation_harness · 2a1d7d87
  Leo Gao authored Feb 04, 2021
  
  2a1d7d87
- Add retry with backoff for GPT3 · b1f7284e
  Leo Gao authored Feb 04, 2021
  
  b1f7284e
- Update arithmetic.py · a55a5c52
  Leo Gao authored Feb 04, 2021
  
  a55a5c52
- Merge pull request #124 from jon-tow/arc-evaluation · 612c69c3
  Leo Gao authored Feb 04, 2021
```
Implement `ARC` evaluation
```
  612c69c3
- Get rid of annoying logging · c55e8237
  Leo Gao authored Feb 04, 2021
  
  c55e8237
- Implement caching · d5cd9655
  Leo Gao authored Feb 04, 2021
  
  d5cd9655
04 Feb, 2021 10 commits
- Merge remote-tracking branch 'origin/master' into fazz/refactor-task-coqa · a21df355
  thefazzer authored Feb 04, 2021
  
  a21df355
- Implement `ARC` evaluation · d72ad5e2
  Jonathan Tow authored Feb 04, 2021
  
  d72ad5e2
- Add missing import · 1815286c
  Leo Gao authored Feb 03, 2021
  
  1815286c
- Implement gpt3 greedy_until · 5f4c7c50
  Leo Gao authored Feb 03, 2021
  
  5f4c7c50
- Get rid of utils_stream · 38e8858f
  Leo Gao authored Feb 03, 2021
  
  38e8858f
- Deprecate utils_stream and remove unused code · 88ec855d
  Leo Gao authored Feb 03, 2021
  
  88ec855d
- Massive refactor · 778e0f91
  Leo Gao authored Feb 03, 2021
```
- Extract evaluator (still needs work to clean up)
- Add tests for evaluator
- Fix all the things that break on the new tests
- Misc cleanup
```
  778e0f91
- Implement gpt3 logprobs · 52c1c56a
  Leo Gao authored Feb 03, 2021
  
  52c1c56a
- Fix lambada · b57d059a
  Leo Gao authored Feb 03, 2021
  
  b57d059a
- Add tests for GPT2 · 8eada53b
  Leo Gao authored Feb 03, 2021
  
  8eada53b
03 Feb, 2021 11 commits
- Merge pull request #123 from jon-tow/winogrande-evaluation · a75a9481
  Leo Gao authored Feb 03, 2021
```
Implement `Winogrande` evaluation
```
  a75a9481
- Implement `Winogrande` evaluation · 485e45b8
  Jonathan Tow authored Feb 03, 2021
  
  485e45b8
- Remove .download mocking · a737c802
  Leo Gao authored Feb 02, 2021
  
  a737c802
- Merge pull request #111 from jon-tow/wsc273-evaluation · 19b0f529
  Leo Gao authored Feb 02, 2021
```
Implement `WSC273` evaluation and data processing
```
  19b0f529
- Merge branch 'master' into wsc273-evaluation · bc5495d2
  Leo Gao authored Feb 02, 2021
  
  bc5495d2
- Merge pull request #109 from EleutherAI/webqs · e12d0078
  Leo Gao authored Feb 02, 2021
```
add webqs evaluation and test set fall back
```
  e12d0078
- Add todo comment · 8444c486
  Leo Gao authored Feb 02, 2021
  
  8444c486
- Merge branch 'master' into wsc273-evaluation · b5e86d3f
  Jonathan Tow authored Feb 02, 2021
  
  b5e86d3f
- Merge branch 'master' into webqs · 269d3683
  Leo Gao authored Feb 02, 2021
  
  269d3683
- Merge pull request #119 from jon-tow/task-refactor · a1a4a32e
  Leo Gao authored Feb 02, 2021
```
Refactor `Dataset` naming and `HFTask` properties
```
  a1a4a32e
- Fix naming convention to avoid `pytest` name mangling invocation · 5cfb7308
  Jonathan Tow authored Feb 02, 2021
  
  5cfb7308