Commits · 6b6303d4cb3fae65a16a6ae2ec6d6453c2b1c9f5 · gaoqiong / lm-evaluation-harness

"next_docs/en/_static/image/flowchart_en.png" did not exist on "c479245e7a2e9dbed0654d6c60b49a1fc6464199"

20 Jun, 2021 7 commits
- Stable file order and don't lower · 6b6303d4
  Leo Gao authored Jun 20, 2021
  
  6b6303d4
- Fix `MuTual` loglikelihood request bug · 69c83456
  Jonathan Tow authored Jun 20, 2021
  
  69c83456
- Merge pull request #201 from jon-tow/mc-taco · 6fa1a4fa
  Leo Gao authored Jun 19, 2021
```
Implement `MC-TACO`
```
  6fa1a4fa
- Add MC-TACO stable test data · 9676b93f
  Leo Gao authored Jun 19, 2021
  
  9676b93f
- Add MuTual test data · 3e1bcf71
  Leo Gao authored Jun 19, 2021
  
  3e1bcf71
- Merge pull request #198 from jon-tow/mutual · 8516571b
  Leo Gao authored Jun 19, 2021
```
Implement `MuTual`
```
  8516571b
- Merge branch 'master' of github.com:EleutherAI/lm_evaluation_harness into mutual · cb8a1b23
  Leo Gao authored Jun 19, 2021
  
  cb8a1b23
19 Jun, 2021 1 commit
- Grammar cleanup in the task docstring · 9a58cee6
  Jonathan Tow authored Jun 19, 2021
  
  9a58cee6
18 Jun, 2021 4 commits
- Merge pull request #203 from jon-tow/best-download-update · 213149ad
  Leo Gao authored Jun 18, 2021
```
Update `download_file` calls to reflect api arg name change
```
  213149ad
- Update `download_file` calls to reflect api arg name change · 6b0a8926
  Jonathan Tow authored Jun 18, 2021
  
  6b0a8926
- Replace `wget` downloading with `best_download` · c7fd34c2
  Jonathan Tow authored Jun 18, 2021
  
  c7fd34c2
- Implement `MC-TACO` · e5db2a1c
  Jonathan Tow authored Jun 18, 2021
  
  e5db2a1c
16 Jun, 2021 3 commits
- Add detokenizer · ebcb28f2
  Jonathan Tow authored Jun 16, 2021
  
  ebcb28f2
- Remove unnecessary parentheses · d235355a
  Jonathan Tow authored Jun 15, 2021
  
  d235355a
- Fix data path location · 31150fac
  Jonathan Tow authored Jun 15, 2021
  
  31150fac
15 Jun, 2021 1 commit
- Implement `MuTual` · d7076a63
  Jonathan Tow authored Jun 15, 2021
  
  d7076a63
14 Jun, 2021 2 commits
- Update readme · be893e9d
  Leo Gao authored Jun 13, 2021
  
  be893e9d
- Fix typo · 0f4e3df4
  Leo Gao authored Jun 13, 2021
  
  0f4e3df4
12 Jun, 2021 7 commits
- Test training docs too and don't cache downloads · 893ef63f
  Leo Gao authored Jun 12, 2021
  
  893ef63f
- Look at all docs of task in test · 7cbd0bba
  Leo Gao authored Jun 12, 2021
  
  7cbd0bba
- Merge pull request #195 from EleutherAI/sdtblck-patch-1 · dc648d84
  Leo Gao authored Jun 12, 2021
```
Change generate until sequence in coqa to be \nQ:
```
  dc648d84
- Update test data for coqa v1 · 41b0ae22
  Leo Gao authored Jun 11, 2021
  
  41b0ae22
- Actually do things correctly this time · f14571fb
  Leo Gao authored Jun 11, 2021
  
  f14571fb
- Bump CoQA version · 278863b5
  Leo Gao authored Jun 11, 2021
  
  278863b5
- Make CoQA take the first line of output excluding any leading whitespace · 2078adba
  Leo Gao authored Jun 11, 2021
  
  2078adba
11 Jun, 2021 1 commit
- Set dummy secret key · 7069583b
  Leo Gao authored Jun 10, 2021
  
  7069583b
10 Jun, 2021 13 commits
- Add openai dependency · 21a1178c
  Leo Gao authored Jun 10, 2021
  
  21a1178c
- Add gpt3 tests · c59cd334
  Leo Gao authored Jun 10, 2021
  
  c59cd334
- Change generate until sequence in coqa to be \nQ: · cc474a94
  sdtblck authored Jun 11, 2021
  
  cc474a94
- Exclude abstractmethod and notimplemented stuff from coverage · 9b9d8dd7
  Leo Gao authored Jun 10, 2021
  
  9b9d8dd7
- Remove a bunch of dead code · 8f8a1dff
  Leo Gao authored Jun 10, 2021
  
  8f8a1dff
- Add stable version hash for prost · 6e5b3cde
  Leo Gao authored Jun 10, 2021
  
  6e5b3cde
- Merge pull request #194 from jon-tow/prost-task · 51a567cf
  Leo Gao authored Jun 10, 2021
```
Implement `PROST` task
```
  51a567cf
- Implement `PROST` task · f4e38081
  Jonathan Tow authored Jun 10, 2021
  
  f4e38081
- Use hashed version stability test instead · f9cc0267
  Leo Gao authored Jun 10, 2021
  
  f9cc0267
- Minor WikiText fix · 10d4b64a
  Leo Gao authored Jun 09, 2021
  
  10d4b64a
- Implement WikiText · 5e31e40e
  Leo Gao authored Jun 09, 2021
  
  5e31e40e
- TriviaQA memory fix · efa99cb2
  Leo Gao authored Jun 09, 2021
  
  efa99cb2
- Omit unimplemented tasks from coverage · c4ecbd6d
  Leo Gao authored Jun 09, 2021
  
  c4ecbd6d
09 Jun, 2021 1 commit
- Update README.md · 142ee24f
  Leo Gao authored Jun 08, 2021
  
  142ee24f