Commits · d5cd9655c8ce48a5d7e04f879de76cce59a0888d · gaoqiong / lm-evaluation-harness

05 Feb, 2021 1 commit
- Implement caching · d5cd9655
  Leo Gao authored Feb 04, 2021
  
  d5cd9655
04 Feb, 2021 8 commits
- Add missing import · 1815286c
  Leo Gao authored Feb 03, 2021
  
  1815286c
- Implement gpt3 greedy_until · 5f4c7c50
  Leo Gao authored Feb 03, 2021
  
  5f4c7c50
- Get rid of utils_stream · 38e8858f
  Leo Gao authored Feb 03, 2021
  
  38e8858f
- Deprecate utils_stream and remove unused code · 88ec855d
  Leo Gao authored Feb 03, 2021
  
  88ec855d
- Massive refactor · 778e0f91
  Leo Gao authored Feb 03, 2021
```
- Extract evaluator (still needs work to clean up)
- Add tests for evaluator
- Fix all the things that break on the new tests
- Misc cleanup
```
  778e0f91
- Implement gpt3 logprobs · 52c1c56a
  Leo Gao authored Feb 03, 2021
  
  52c1c56a
- Fix lambada · b57d059a
  Leo Gao authored Feb 03, 2021
  
  b57d059a
- Add tests for GPT2 · 8eada53b
  Leo Gao authored Feb 03, 2021
  
  8eada53b
03 Feb, 2021 12 commits
- Merge pull request #123 from jon-tow/winogrande-evaluation · a75a9481
  Leo Gao authored Feb 03, 2021
```
Implement `Winogrande` evaluation
```
  a75a9481
- Implement `Winogrande` evaluation · 485e45b8
  Jonathan Tow authored Feb 03, 2021
  
  485e45b8
- Remove .download mocking · a737c802
  Leo Gao authored Feb 02, 2021
  
  a737c802
- Merge pull request #111 from jon-tow/wsc273-evaluation · 19b0f529
  Leo Gao authored Feb 02, 2021
```
Implement `WSC273` evaluation and data processing
```
  19b0f529
- Merge branch 'master' into wsc273-evaluation · bc5495d2
  Leo Gao authored Feb 02, 2021
  
  bc5495d2
- Merge pull request #109 from EleutherAI/webqs · e12d0078
  Leo Gao authored Feb 02, 2021
```
add webqs evaluation and test set fall back
```
  e12d0078
- Add todo comment · 8444c486
  Leo Gao authored Feb 02, 2021
  
  8444c486
- Merge branch 'master' into wsc273-evaluation · b5e86d3f
  Jonathan Tow authored Feb 02, 2021
  
  b5e86d3f
- Merge branch 'master' into webqs · 269d3683
  Leo Gao authored Feb 02, 2021
  
  269d3683
- Merge pull request #119 from jon-tow/task-refactor · a1a4a32e
  Leo Gao authored Feb 02, 2021
```
Refactor `Dataset` naming and `HFTask` properties
```
  a1a4a32e
- Fix naming convention to avoid `pytest` name mangling invocation · 5cfb7308
  Jonathan Tow authored Feb 02, 2021
  
  5cfb7308
- Refactor `Dataset` naming and `HFTask` properties · a60ef6fa
  Jonathan Tow authored Feb 02, 2021
  
  a60ef6fa
02 Feb, 2021 1 commit
- Implement `WSC273` evaluation and data processing · c32a13e8
  Jonathan Tow authored Feb 02, 2021
  
  c32a13e8
01 Feb, 2021 13 commits
- Add badges · 826d90e2
  Leo Gao authored Feb 01, 2021
  
  826d90e2
- Fix coverage again again again again again · 5b6182d5
  Leo Gao authored Feb 01, 2021
  
  5b6182d5
- Fix coverage again again again again · 2d6102c4
  Leo Gao authored Feb 01, 2021
  
  2d6102c4
- Fix coverage again again again · 2708ae4c
  Leo Gao authored Feb 01, 2021
  
  2708ae4c
- Add unit tests · a3a2152c
  Leo Gao authored Feb 01, 2021
  
  a3a2152c
- Fix coverage again again · 8bd843c0
  Leo Gao authored Feb 01, 2021
  
  8bd843c0
- Fix coverage again · b7da653c
  Leo Gao authored Feb 01, 2021
  
  b7da653c
- Fix pytest path · 1033bab6
  Leo Gao authored Feb 01, 2021
  
  1033bab6
- Upload to codecov · 12067538
  Leo Gao authored Feb 01, 2021
  
  12067538
- Make linter happy · 31863cde
  Leo Gao authored Feb 01, 2021
  
  31863cde
- Fix linting problems · fe4a1efd
  Leo Gao authored Feb 01, 2021
  
  fe4a1efd
- Merge pull request #110 from EleutherAI/leogao2-patch-2 · 7ab3f8ca
  Stella Biderman authored Feb 01, 2021
```
Add CI pipeline
```
  7ab3f8ca
- Add CI pipeline · 8dd1983b
  Leo Gao authored Feb 01, 2021
  
  8dd1983b
31 Jan, 2021 1 commit
- add webqs evaluation and fallback to test set when validation is unavailable · 34eb121f
  Anthony DiPofi authored Jan 31, 2021
  
  34eb121f
30 Jan, 2021 4 commits
- Merge pull request #106 from jon-tow/anli-evaluation · 6598967b
  Leo Gao authored Jan 30, 2021
```
Implement `ANLI` evaluations
```
  6598967b
- Merge pull request #108 from EleutherAI/leogao2-patch-2 · ec45d0aa
  Stella Biderman authored Jan 30, 2021
```
Make *_docs not abstract
```
  ec45d0aa
- Merge pull request #107 from EleutherAI/triviaqa_evaluation · 66558b35
  Leo Gao authored Jan 30, 2021
```
add evaluation for TriviaQA dataset based on loglikelihood method
```
  66558b35
- Merge branch 'master' into triviaqa_evaluation · c8032a1a
  Leo Gao authored Jan 30, 2021
  
  c8032a1a