Commits · 0971cffe83fc91ebb52812561efebab6411e252c · gaoqiong / lm-evaluation-harness

06 Feb, 2021 1 commit
- separated sciq and pubmedqa to different files · 0971cffe
  jeffhsu3 authored Feb 06, 2021
  
  0971cffe
05 Feb, 2021 12 commits
- sciq · 620c0d26
  jeffhsu3 authored Feb 05, 2021
  
  620c0d26
- Added SciQ · c71dcb91
  jeffhsu3 authored Feb 05, 2021
  
  c71dcb91
- Add task table · c0fbf9e8
  Leo Gao authored Feb 04, 2021
  
  c0fbf9e8
- Add test to make sure right space conventions are used · 049dfa34
  Leo Gao authored Feb 04, 2021
  
  049dfa34
- Fix SAT · aea63162
  Leo Gao authored Feb 04, 2021
  
  aea63162
- Convert SAT to use MultipleChoiceTask · d7c08d5b
  Leo Gao authored Feb 04, 2021
  
  d7c08d5b
- Add MultipleChoiceTest target · 2f5f42c6
  Leo Gao authored Feb 04, 2021
  
  2f5f42c6
- Add MultipleChoiceTask · 706cb53a
  Leo Gao authored Feb 04, 2021
  
  706cb53a
- Add retry with backoff for GPT3 · b1f7284e
  Leo Gao authored Feb 04, 2021
  
  b1f7284e
- Update arithmetic.py · a55a5c52
  Leo Gao authored Feb 04, 2021
  
  a55a5c52
- Get rid of annoying logging · c55e8237
  Leo Gao authored Feb 04, 2021
  
  c55e8237
- Implement caching · d5cd9655
  Leo Gao authored Feb 04, 2021
  
  d5cd9655
04 Feb, 2021 8 commits
- Implement `ARC` evaluation · d72ad5e2
  Jonathan Tow authored Feb 04, 2021
  
  d72ad5e2
- Add missing import · 1815286c
  Leo Gao authored Feb 03, 2021
  
  1815286c
- Implement gpt3 greedy_until · 5f4c7c50
  Leo Gao authored Feb 03, 2021
  
  5f4c7c50
- Get rid of utils_stream · 38e8858f
  Leo Gao authored Feb 03, 2021
  
  38e8858f
- Deprecate utils_stream and remove unused code · 88ec855d
  Leo Gao authored Feb 03, 2021
  
  88ec855d
- Massive refactor · 778e0f91
  Leo Gao authored Feb 03, 2021
```
- Extract evaluator (still needs work to clean up)
- Add tests for evaluator
- Fix all the things that break on the new tests
- Misc cleanup
```
  778e0f91
- Implement gpt3 logprobs · 52c1c56a
  Leo Gao authored Feb 03, 2021
  
  52c1c56a
- Fix lambada · b57d059a
  Leo Gao authored Feb 03, 2021
  
  b57d059a
03 Feb, 2021 3 commits
- Implement `Winogrande` evaluation · 485e45b8
  Jonathan Tow authored Feb 03, 2021
  
  485e45b8
- Fix naming convention to avoid `pytest` name mangling invocation · 5cfb7308
  Jonathan Tow authored Feb 02, 2021
  
  5cfb7308
- Refactor `Dataset` naming and `HFTask` properties · a60ef6fa
  Jonathan Tow authored Feb 02, 2021
  
  a60ef6fa
02 Feb, 2021 2 commits
- pubmedqa · 1b35f6b9
  jeffhsu3 authored Feb 02, 2021
  
  1b35f6b9
- Implement `WSC273` evaluation and data processing · c32a13e8
  Jonathan Tow authored Feb 02, 2021
  
  c32a13e8
01 Feb, 2021 3 commits
- Fix coverage again again again again · 2d6102c4
  Leo Gao authored Feb 01, 2021
  
  2d6102c4
- Make linter happy · 31863cde
  Leo Gao authored Feb 01, 2021
  
  31863cde
- Fix linting problems · fe4a1efd
  Leo Gao authored Feb 01, 2021
  
  fe4a1efd
31 Jan, 2021 1 commit
- add webqs evaluation and fallback to test set when validation is unavailable · 34eb121f
  Anthony DiPofi authored Jan 31, 2021
  
  34eb121f
30 Jan, 2021 4 commits
- Handle aliases correctly in TriviaQA · b2b5a122
  Leo Gao authored Jan 30, 2021
  
  b2b5a122
- Make *_docs not abstract · 1bf97c9e
  Leo Gao authored Jan 30, 2021
  
  1bf97c9e
- add evaluation for TriviaQA dataset based on loglikelihood · 0601d0bb
  Anthony DiPofi authored Jan 30, 2021
  
  0601d0bb
- Implement `ANLI` evaluations · 0a53c06e
  Jonathan Tow authored Jan 30, 2021
  
  0a53c06e
29 Jan, 2021 6 commits
- Update dependencies · f37dcab6
  Jon Tow authored Jan 29, 2021
  
  f37dcab6
- Implement `RACE` evaluation · 6e1a76f0
  Jon Tow authored Jan 29, 2021
  
  6e1a76f0
- Implement PiQA · 63854c10
  Leo Gao authored Jan 29, 2021
  
  63854c10
- Remove brackets from documents · 97b88570
  Jonathan Tow authored Jan 29, 2021
  
  97b88570
- Implement LAMBADA · 822fcc6f
  Leo Gao authored Jan 28, 2021
  
  822fcc6f
- Revert "Update `doc_to_text` to show candidate fill-ins" · da016758
  Jonathan Tow authored Jan 29, 2021
```
This reverts commit 5d431ba9.
```
  da016758