Commits · d34ae3cfc9fba7f17da48c8a33ea87f2a9d27995 · gaoqiong / lm-evaluation-harness

16 Dec, 2021 1 commit
- Add `provide_description` arg for backward compat · ee53be21
  Jonathan Tow authored Dec 15, 2021
  
  ee53be21
29 Nov, 2021 2 commits
- Pin datasets to 1.15.1 · 784a5c54
  Leo Gao authored Nov 28, 2021
  
  784a5c54
- Pin datasets to 1.16.0 · 13916e32
  Leo Gao authored Nov 28, 2021
  
  13916e32
19 Nov, 2021 1 commit
- Add `truthfulqa` back to the task registry · 33ccde86
  Jonathan Tow authored Nov 19, 2021
  
  33ccde86
16 Nov, 2021 1 commit
- Comment out truthfulqa before tests are fixed · 134150a2
  Leo Gao authored Nov 15, 2021
  
  134150a2
06 Nov, 2021 1 commit
- Add newline separator for correct `rougeLsum` · 83252575
  Jonathan Tow authored Nov 06, 2021
  
  83252575
04 Nov, 2021 3 commits
- Remove `t5` imports` · b0746b35
  Jonathan Tow authored Nov 04, 2021
  
  b0746b35
- Add newline to EOF · e3c74cc0
  Jonathan Tow authored Nov 04, 2021
  
  e3c74cc0
- Remove `t5` dependency · 0f27316c
  Jonathan Tow authored Nov 04, 2021
  
  0f27316c
03 Nov, 2021 1 commit
- Bump the `truthfulqa` version to 1 · 05ed92a4
  Jonathan Tow authored Nov 03, 2021
  
  05ed92a4
31 Oct, 2021 2 commits
- Pin the datasets to the original source · 72257924
  Jonathan Tow authored Oct 30, 2021
  
  72257924
- Update `truthfulqa` dataset checksums · 2cfbfd72
  Jonathan Tow authored Oct 30, 2021
  
  2cfbfd72
30 Oct, 2021 2 commits
- Replace the `fewshot_description` API with a `description_dict` based interface · 8ac99269
  Jonathan Tow authored Oct 30, 2021
  
  8ac99269
- Always mkdir with parents · e7dd78ef
  Jun Shern Chan authored Oct 30, 2021
  
  e7dd78ef
07 Oct, 2021 1 commit
- Add underscores to metrics · 85ffb7fc
  Leo Gao authored Oct 06, 2021
  
  85ffb7fc
06 Oct, 2021 1 commit
- Adhere to spacing convention · dd59b0ef
  Jonathan Tow authored Oct 06, 2021
  
  dd59b0ef
25 Sep, 2021 1 commit
- Implement `TruthfulQA` · 17c47812
  Jonathan Tow authored Sep 24, 2021
  
  17c47812
03 Sep, 2021 1 commit
- fix file download for existing files · 031677b6
  Samuel Weinbach authored Sep 03, 2021
  
  031677b6
01 Sep, 2021 1 commit
- Fix lambada multilingual tasks · 754f1f24
  sdtblck authored Sep 01, 2021
```
the old task constructor was making everything spanish
```
  754f1f24
31 Aug, 2021 1 commit
- Move jieba and nagisa imports into their respective functions · 9d46382b
  Leo Gao authored Aug 31, 2021
  
  9d46382b
28 Aug, 2021 1 commit
- triviaqa OOM fix · 52f270dc
  Leo Gao authored Aug 27, 2021
  
  52f270dc
26 Aug, 2021 1 commit
- Update translation.py · c0f0f7e1
  Leo Gao authored Aug 26, 2021
  
  c0f0f7e1
15 Aug, 2021 1 commit
- Add space tokenization for JA/ZH · f046d331
  Muennighoff authored Aug 15, 2021
  
  f046d331
13 Aug, 2021 1 commit
- Update version and testdata for DROP · 81122e3d
  Leo Gao authored Aug 13, 2021
  
  81122e3d
11 Aug, 2021 2 commits

Bug fix · 5dbeba0d
silentv0x authored Aug 11, 2021

5dbeba0d

silentv0x authored Aug 11, 2021

Following the official implementation, the following changes were made
 - Validated answers are considered gold answers
 - EM/F1 are the max over all gold answers

35d809c3

09 Aug, 2021 1 commit
- fix squad breakage due to HF change · 198ca732
  Leo Gao authored Aug 09, 2021
  
  198ca732
03 Jul, 2021 1 commit
- Fix version issue · e35386d9
  Leo Gao authored Jul 02, 2021
  
  e35386d9
24 Jun, 2021 1 commit
- Don't download lambada if it already exists · 75519897
  sdtblck authored Jun 24, 2021
  
  75519897
23 Jun, 2021 1 commit
- Add missing functools import · db85d5ae
  Leo Gao authored Jun 22, 2021
  
  db85d5ae
22 Jun, 2021 1 commit
- Add missing json import · a10b05c5
  Leo Gao authored Jun 22, 2021
  
  a10b05c5
20 Jun, 2021 3 commits
- Indicate that it's machine translated in task name and comments · 11f0e6d8
  Leo Gao authored Jun 20, 2021
  
  11f0e6d8
- Stable file order and don't lower · 6b6303d4
  Leo Gao authored Jun 20, 2021
  
  6b6303d4
- Fix `MuTual` loglikelihood request bug · 69c83456
  Jonathan Tow authored Jun 20, 2021
  
  69c83456
19 Jun, 2021 5 commits
- Grammar cleanup in the task docstring · 9a58cee6
  Jonathan Tow authored Jun 19, 2021
  
  9a58cee6
- Update __init__.py · 40bdb0c4
  sdtblck authored Jun 19, 2021
  
  40bdb0c4
- Update lambada_multilingual.py · 5b008dc0
  sdtblck authored Jun 19, 2021
  
  5b008dc0
- Update __init__.py · a92d47eb
  sdtblck authored Jun 19, 2021
  
  a92d47eb
- Create lambada_multilingual.py · 4cd4b05c
  sdtblck authored Jun 19, 2021
  
  4cd4b05c
18 Jun, 2021 1 commit
- Update `download_file` calls to reflect api arg name change · 6b0a8926
  Jonathan Tow authored Jun 18, 2021
  
  6b0a8926