Commits · a21df3552e2b1db6fd0e0ea6efd5d8205f89d1df · gaoqiong / lm-evaluation-harness

04 Feb, 2021 7 commits
- Add missing import · 1815286c
  Leo Gao authored Feb 03, 2021
  
  1815286c
- Implement gpt3 greedy_until · 5f4c7c50
  Leo Gao authored Feb 03, 2021
  
  5f4c7c50
- Get rid of utils_stream · 38e8858f
  Leo Gao authored Feb 03, 2021
  
  38e8858f
- Deprecate utils_stream and remove unused code · 88ec855d
  Leo Gao authored Feb 03, 2021
  
  88ec855d
- Massive refactor · 778e0f91
  Leo Gao authored Feb 03, 2021
```
- Extract evaluator (still needs work to clean up)
- Add tests for evaluator
- Fix all the things that break on the new tests
- Misc cleanup
```
  778e0f91
- Implement gpt3 logprobs · 52c1c56a
  Leo Gao authored Feb 03, 2021
  
  52c1c56a
- Fix lambada · b57d059a
  Leo Gao authored Feb 03, 2021
  
  b57d059a
03 Feb, 2021 4 commits
- Implement `Winogrande` evaluation · 485e45b8
  Jonathan Tow authored Feb 03, 2021
  
  485e45b8
- Fix naming convention to avoid `pytest` name mangling invocation · 5cfb7308
  Jonathan Tow authored Feb 02, 2021
  
  5cfb7308
- Refactor `Dataset` naming and `HFTask` properties · a60ef6fa
  Jonathan Tow authored Feb 02, 2021
  
  a60ef6fa
- Score computation, use squad metrics · efa810f0
  thefazzer authored Feb 03, 2021
  
  efa810f0
02 Feb, 2021 1 commit
- Implement `WSC273` evaluation and data processing · c32a13e8
  Jonathan Tow authored Feb 02, 2021
  
  c32a13e8
01 Feb, 2021 3 commits
- Fix coverage again again again again · 2d6102c4
  Leo Gao authored Feb 01, 2021
  
  2d6102c4
- Make linter happy · 31863cde
  Leo Gao authored Feb 01, 2021
  
  31863cde
- Fix linting problems · fe4a1efd
  Leo Gao authored Feb 01, 2021
  
  fe4a1efd
31 Jan, 2021 4 commits
- Text & target impl, support fns, refactoring · c0862026
  thefazzer authored Jan 31, 2021
  
  c0862026
- add webqs evaluation and fallback to test set when validation is unavailable · 34eb121f
  Anthony DiPofi authored Jan 31, 2021
  
  34eb121f
- Add CoQa to task registry · 47384df7
  thefazzer authored Jan 31, 2021
  
  47384df7
- Initial skeleton refactoring · 12880f1c
  thefazzer authored Jan 31, 2021
  
  12880f1c
30 Jan, 2021 4 commits
- Handle aliases correctly in TriviaQA · b2b5a122
  Leo Gao authored Jan 30, 2021
  
  b2b5a122
- Make *_docs not abstract · 1bf97c9e
  Leo Gao authored Jan 30, 2021
  
  1bf97c9e
- add evaluation for TriviaQA dataset based on loglikelihood · 0601d0bb
  Anthony DiPofi authored Jan 30, 2021
  
  0601d0bb
- Implement `ANLI` evaluations · 0a53c06e
  Jonathan Tow authored Jan 30, 2021
  
  0a53c06e
29 Jan, 2021 7 commits
- Update dependencies · f37dcab6
  Jon Tow authored Jan 29, 2021
  
  f37dcab6
- Implement `RACE` evaluation · 6e1a76f0
  Jon Tow authored Jan 29, 2021
  
  6e1a76f0
- Implement PiQA · 63854c10
  Leo Gao authored Jan 29, 2021
  
  63854c10
- Remove brackets from documents · 97b88570
  Jonathan Tow authored Jan 29, 2021
  
  97b88570
- Implement LAMBADA · 822fcc6f
  Leo Gao authored Jan 28, 2021
  
  822fcc6f
- Revert "Update `doc_to_text` to show candidate fill-ins" · da016758
  Jonathan Tow authored Jan 29, 2021
```
This reverts commit 5d431ba9.
```
  da016758
- Update `doc_to_text` to show candidate fill-ins · 5d431ba9
  Jon Tow authored Jan 28, 2021
  
  5d431ba9
28 Jan, 2021 4 commits
- Add `HellaSwag` evaluation implementation · 65c46d22
  Jon Tow authored Jan 28, 2021
  
  65c46d22
- Add space before target · 21d527d4
  Leo Gao authored Jan 28, 2021
  
  21d527d4
- wsc · 63e32f9c
  Jason Phang authored Jan 28, 2021
  
  63e32f9c
- Implement unit testing and fix lots of problems with tasks · 60a6fd8c
  Leo Gao authored Jan 27, 2021
  
  60a6fd8c
27 Jan, 2021 1 commit
- use openai's gpt3 datafiles instead of generating equations · baadb2da
  Kevin Wang authored Jan 27, 2021
  
  baadb2da
24 Jan, 2021 1 commit
- superglue ex wsc · 2e707b87
  Jason Phang authored Jan 23, 2021
  
  2e707b87
23 Jan, 2021 2 commits
- prepending a space, because it significantly increases log-likelihood of the correct answer. · 183f4721
  Kevin Wang authored Jan 14, 2021
  
  183f4721
- first attempt at adding arithmetic evaluations · a58e5e4b
  Kevin Wang authored Jan 14, 2021
  
  a58e5e4b
22 Jan, 2021 2 commits
- Remove rewrite reminder comment from everything except SuperGLUE · f4120e59
  Leo Gao authored Jan 21, 2021
  
  f4120e59
- Clean up code, remove some footguns · e31b4b31
  Leo Gao authored Jan 21, 2021
  
  e31b4b31