Commits · d7076a634ec7cb95c0e98ab794cf1dc8e612d5d2 · gaoqiong / lm-evaluation-harness

15 Jun, 2021 1 commit
- Implement `MuTual` · d7076a63
  Jonathan Tow authored Jun 15, 2021
  
  d7076a63
12 Jun, 2021 3 commits
- Actually do things correctly this time · f14571fb
  Leo Gao authored Jun 11, 2021
  
  f14571fb
- Bump CoQA version · 278863b5
  Leo Gao authored Jun 11, 2021
  
  278863b5
- Make CoQA take the first line of output excluding any leading whitespace · 2078adba
  Leo Gao authored Jun 11, 2021
  
  2078adba
10 Jun, 2021 7 commits
- Change generate until sequence in coqa to be \nQ: · cc474a94
  sdtblck authored Jun 11, 2021
  
  cc474a94
- Remove a bunch of dead code · 8f8a1dff
  Leo Gao authored Jun 10, 2021
  
  8f8a1dff
- Implement `PROST` task · f4e38081
  Jonathan Tow authored Jun 10, 2021
  
  f4e38081
- Minor WikiText fix · 10d4b64a
  Leo Gao authored Jun 09, 2021
  
  10d4b64a
- Implement WikiText · 5e31e40e
  Leo Gao authored Jun 09, 2021
  
  5e31e40e
- TriviaQA memory fix · efa99cb2
  Leo Gao authored Jun 09, 2021
  
  efa99cb2
- Omit unimplemented tasks from coverage · c4ecbd6d
  Leo Gao authored Jun 09, 2021
  
  c4ecbd6d
08 Jun, 2021 2 commits
- make hendrycks math order deterministic · 3b990af0
  Leo Gao authored Jun 07, 2021
  
  3b990af0
- Actually fix hendrycks math · 90429563
  Leo Gao authored Jun 07, 2021
  
  90429563
07 Jun, 2021 1 commit
- Fix hendrycks MATH to use new download link · 004b63e3
  Leo Gao authored Jun 07, 2021
  
  004b63e3
05 Jun, 2021 3 commits
- convert all enabled tasks to use best_download for better download robustness · 20229183
  Leo Gao authored Jun 04, 2021
  
  20229183
- fix bug where interrupted pile download doesnt correctly resume · 85277915
  Leo Gao authored Jun 04, 2021
  
  85277915
- Add task versioning · 105fa974
  Leo Gao authored Jun 04, 2021
  
  105fa974
16 May, 2021 3 commits
- naturalqs: fix spacing · 9c76176b
  Leo Gao authored May 15, 2021
  
  9c76176b
- fix hendrycksMath for \fixed without curly braces · aecf13dd
  Leo Gao authored May 15, 2021
  
  aecf13dd
- remove erroneous entry for Pile-CC · 9bbf2406
  Leo Gao authored May 15, 2021
  
  9bbf2406
13 May, 2021 1 commit
- Remove PileCommonCrawl (empty task) · 43f4d418
  Leo Gao authored May 12, 2021
  
  43f4d418
11 May, 2021 1 commit
- switch lambada over to wget while researcher2 fixes best-download · ab8da1f0
  Leo Gao authored May 11, 2021
  
  ab8da1f0
06 May, 2021 2 commits
- Fix Typo · dd573a95
  Leo Gao authored May 06, 2021
  
  dd573a95
- Rename a ton of stuff · 45127aa7
  Leo Gao authored May 05, 2021
  
  45127aa7
05 May, 2021 1 commit
- Fix some things for tests · 80a739eb
  Leo Gao authored May 05, 2021
  
  80a739eb
03 May, 2021 2 commits
- add pile tasks · 82880154
  Jason Phang authored May 02, 2021
  
  82880154
- gpt2 perplexity · 9454c839
  Jason Phang authored May 02, 2021
  
  9454c839
22 Apr, 2021 2 commits
- Make hendrycks math tar extraction quieter · 853afa36
  Leo Gao authored Apr 22, 2021
  
  853afa36
- Fix first-run error · dd8d3529
  Leo Gao authored Apr 22, 2021
  
  dd8d3529
21 Apr, 2021 1 commit
- Disable triviaqa and cbt tasks · ab0e4d26
  Leo Gao authored Apr 20, 2021
  
  ab0e4d26
11 Apr, 2021 6 commits
- Fix triviaqa · 5dc4692a
  Leo Gao authored Apr 10, 2021
  
  5dc4692a
- Fix typo · 29c1f228
  Leo Gao authored Apr 10, 2021
  
  29c1f228
- Factor _convert_standard into MultipleChoiceTask · ec16d81d
  Leo Gao authored Apr 10, 2021
  
  ec16d81d
- Fix import · 1ebf41d3
  Leo Gao authored Apr 10, 2021
  
  1ebf41d3
- Make piqa inherit MultipleChoiceTask · 4462e415
  Leo Gao authored Apr 10, 2021
  
  4462e415
- also apply normalization to piqa · 2522c27f
  Ben Wang authored Apr 11, 2021
  
  2522c27f
10 Apr, 2021 3 commits
- Use more inheritance · 7ba8c183
  Leo Gao authored Apr 10, 2021
  
  7ba8c183
- add cloze variant of lambada task for allegeldy improved few shot results · 47382717
  Ben Wang authored Apr 10, 2021
  
  47382717
- Fix triviaqa memory consumption problem · e971baac
  Leo Gao authored Apr 09, 2021
```
(or rather, move it to the data download phase)
```
  e971baac
31 Mar, 2021 1 commit
- Add note on accuracy reporting · d78917a7
  Jon Tow authored Mar 31, 2021
  
  d78917a7