Commits · 6caa0afd96a7a7efb2ec4c1f24ad1756e48f3aa7 · gaoqiong / lm-evaluation-harness

18 Mar, 2022 1 commit
- Refactor `Task` download · 7c9da714
  Jonathan Tow authored Mar 11, 2022
  
  7c9da714
08 Feb, 2022 1 commit
- Add initial draft of QASPER; register with package; yet to complete process_results · 79492627
  Stephen Hogg authored Feb 08, 2022
  
  79492627
23 Jan, 2022 1 commit
- Add grade school math `GSM8K` · f04a91c1
  Jonathan Tow authored Jan 23, 2022
  
  f04a91c1
26 Dec, 2021 1 commit
- add asdiv task · b0a12310
  rokosbasilisk authored Dec 26, 2021
  
  b0a12310
24 Dec, 2021 4 commits
- Update new `task` arg and task dict getter · 0e232f7a
  Jonathan Tow authored Dec 24, 2021
  
  0e232f7a
- more changes · d86aabc4
  Leo Gao authored Dec 23, 2021
  
  d86aabc4
- Make simple_evaluate take LM and Task objects directly too · 7b2b2a23
  Leo Gao authored Dec 23, 2021
  
  7b2b2a23
- headqa: maintain backwards compatibility · cdab2c03
  Leo Gao authored Dec 23, 2021
  
  cdab2c03
23 Dec, 2021 1 commit
- Allow users to specify en headqa or es · 5855f482
  thomasw21 authored Dec 23, 2021
  
  5855f482
22 Dec, 2021 1 commit
- Add BLiMP · 5f48dfc2
  Igor Ostrovsky authored Dec 21, 2021
  
  5f48dfc2
19 Nov, 2021 1 commit
- Add `truthfulqa` back to the task registry · 33ccde86
  Jonathan Tow authored Nov 19, 2021
  
  33ccde86
16 Nov, 2021 1 commit
- Comment out truthfulqa before tests are fixed · 134150a2
  Leo Gao authored Nov 15, 2021
  
  134150a2
25 Sep, 2021 1 commit
- Implement `TruthfulQA` · 17c47812
  Jonathan Tow authored Sep 24, 2021
  
  17c47812
19 Jun, 2021 2 commits
- Update __init__.py · 40bdb0c4
  sdtblck authored Jun 19, 2021
  
  40bdb0c4
- Update __init__.py · a92d47eb
  sdtblck authored Jun 19, 2021
  
  a92d47eb
18 Jun, 2021 1 commit
- Implement `MC-TACO` · e5db2a1c
  Jonathan Tow authored Jun 18, 2021
  
  e5db2a1c
15 Jun, 2021 1 commit
- Implement `MuTual` · d7076a63
  Jonathan Tow authored Jun 15, 2021
  
  d7076a63
10 Jun, 2021 4 commits
- Implement `PROST` task · f4e38081
  Jonathan Tow authored Jun 10, 2021
  
  f4e38081
- Implement WikiText · 5e31e40e
  Leo Gao authored Jun 09, 2021
  
  5e31e40e
- TriviaQA memory fix · efa99cb2
  Leo Gao authored Jun 09, 2021
  
  efa99cb2
- Omit unimplemented tasks from coverage · c4ecbd6d
  Leo Gao authored Jun 09, 2021
  
  c4ecbd6d
16 May, 2021 1 commit
- remove erroneous entry for Pile-CC · 9bbf2406
  Leo Gao authored May 15, 2021
  
  9bbf2406
06 May, 2021 1 commit
- Rename a ton of stuff · 45127aa7
  Leo Gao authored May 05, 2021
  
  45127aa7
03 May, 2021 2 commits
- add pile tasks · 82880154
  Jason Phang authored May 02, 2021
  
  82880154
- gpt2 perplexity · 9454c839
  Jason Phang authored May 02, 2021
  
  9454c839
21 Apr, 2021 1 commit
- Disable triviaqa and cbt tasks · ab0e4d26
  Leo Gao authored Apr 20, 2021
  
  ab0e4d26
10 Apr, 2021 2 commits
- add cloze variant of lambada task for allegeldy improved few shot results · 47382717
  Ben Wang authored Apr 10, 2021
  
  47382717
- Fix triviaqa memory consumption problem · e971baac
  Leo Gao authored Apr 09, 2021
```
(or rather, move it to the data download phase)
```
  e971baac
29 Mar, 2021 1 commit
- Implement `Childrens Book Test` dataset download and evaluation · a10856dc
  Jon Tow authored Mar 29, 2021
  
  a10856dc
28 Mar, 2021 2 commits
- Rename hendrycks ethics and math · 42659c34
  Leo Gao authored Mar 28, 2021
  
  42659c34
- Rename task · 8de85534
  Leo Gao authored Mar 28, 2021
  
  8de85534
21 Mar, 2021 2 commits
- adjusted naming · 8888a6be
  Andy Zou authored Mar 20, 2021
  
  8888a6be
- add hendrycksTest · a6b01474
  Andy Zou authored Mar 20, 2021
  
  a6b01474
19 Mar, 2021 1 commit
- Added math tasks · 72f91040
  Eric Tang authored Mar 19, 2021
  
  72f91040
08 Mar, 2021 1 commit
- Implement `LogiQA` data download and evaluation · a187cd44
  Jonathan Tow authored Mar 08, 2021
  
  a187cd44
06 Mar, 2021 1 commit
- Flag down a bunch of todos · 491283c5
  Leo Gao authored Mar 05, 2021
  
  491283c5
01 Mar, 2021 1 commit
- Mimic original Ethics Util procedure · 001c0a05
  Muennighoff authored Mar 01, 2021
  
  001c0a05
24 Feb, 2021 2 commits
- Implement all word scrambling and manipulation tasks · 41a94e13
  Jon Tow authored Feb 24, 2021
  
  41a94e13
- Implement `Anagrams` evaluations · a5e135b6
  Jon Tow authored Feb 23, 2021
  
  a5e135b6
21 Feb, 2021 1 commit
- Implement `DROP` evaluation · fc7cd630
  Jon Tow authored Feb 20, 2021
  
  fc7cd630