Commits · d67c77beab707a682adbdb7e76a7b3f414badf1e · gaoqiong / lm-evaluation-harness

26 Aug, 2021 4 commits
- Merge pull request #214 from Muennighoff/tokenization · d67c77be
  Leo Gao authored Aug 26, 2021
```
Add space tokenization for JA/ZH
```
  d67c77be
- Merge pull request #213 from silentv0x/master · 4d2c1522
  Leo Gao authored Aug 26, 2021
```
Fixes DROP implementation
```
  4d2c1522
- evaluator: fix bleu/chrf/ter taking forever · 82ec4f53
  Leo Gao authored Aug 26, 2021
  
  82ec4f53
- Update translation.py · c0f0f7e1
  Leo Gao authored Aug 26, 2021
  
  c0f0f7e1
15 Aug, 2021 1 commit
- Add space tokenization for JA/ZH · f046d331
  Muennighoff authored Aug 15, 2021
  
  f046d331
13 Aug, 2021 1 commit
- Update version and testdata for DROP · 81122e3d
  Leo Gao authored Aug 13, 2021
  
  81122e3d
11 Aug, 2021 2 commits

Bug fix · 5dbeba0d
silentv0x authored Aug 11, 2021

5dbeba0d

silentv0x authored Aug 11, 2021

Following the official implementation, the following changes were made
 - Validated answers are considered gold answers
 - EM/F1 are the max over all gold answers

35d809c3

09 Aug, 2021 1 commit
- fix squad breakage due to HF change · 198ca732
  Leo Gao authored Aug 09, 2021
  
  198ca732
08 Aug, 2021 1 commit
- Update README.md · fc0d2377
  Leo Gao authored Aug 07, 2021
  
  fc0d2377
02 Jul, 2021 3 commits
- Merge pull request #210 from EleutherAI/StellaAthena-patch-1 · 592b2a23
  Leo Gao authored Jul 01, 2021
```
Updated readme for clarity
```
  592b2a23
- Update README.md · adec7faa
  Leo Gao authored Jul 01, 2021
  
  adec7faa
- Update README.md · ba3fa9c8
  Leo Gao authored Jul 01, 2021
  
  ba3fa9c8
25 Jun, 2021 2 commits
- Update main.py · f6e7ae25
  Leo Gao authored Jun 25, 2021
  
  f6e7ae25
- Updated readme for clarity · 93ea091e
  Stella Biderman authored Jun 25, 2021
  
  93ea091e
24 Jun, 2021 2 commits
- Merge pull request #209 from EleutherAI/sdtblck-patch-1 · af0b1e26
  Leo Gao authored Jun 24, 2021
```
Don't download lambada if it already exists
```
  af0b1e26
- Don't download lambada if it already exists · 75519897
  sdtblck authored Jun 24, 2021
  
  75519897
22 Jun, 2021 1 commit
- Merge pull request #206 from jon-tow/mutual-fix · 3473ea80
  Leo Gao authored Jun 22, 2021
```
Fix `MuTual` loglikelihood request bug
```
  3473ea80
20 Jun, 2021 8 commits
- Add mutual v1 test data · 0fec4555
  Leo Gao authored Jun 20, 2021
  
  0fec4555
- Stable file order and don't lower · 6b6303d4
  Leo Gao authored Jun 20, 2021
  
  6b6303d4
- Fix `MuTual` loglikelihood request bug · 69c83456
  Jonathan Tow authored Jun 20, 2021
  
  69c83456
- Merge pull request #201 from jon-tow/mc-taco · 6fa1a4fa
  Leo Gao authored Jun 19, 2021
```
Implement `MC-TACO`
```
  6fa1a4fa
- Add MC-TACO stable test data · 9676b93f
  Leo Gao authored Jun 19, 2021
  
  9676b93f
- Add MuTual test data · 3e1bcf71
  Leo Gao authored Jun 19, 2021
  
  3e1bcf71
- Merge pull request #198 from jon-tow/mutual · 8516571b
  Leo Gao authored Jun 19, 2021
```
Implement `MuTual`
```
  8516571b
- Merge branch 'master' of github.com:EleutherAI/lm_evaluation_harness into mutual · cb8a1b23
  Leo Gao authored Jun 19, 2021
  
  cb8a1b23
19 Jun, 2021 1 commit
- Grammar cleanup in the task docstring · 9a58cee6
  Jonathan Tow authored Jun 19, 2021
  
  9a58cee6
18 Jun, 2021 4 commits
- Merge pull request #203 from jon-tow/best-download-update · 213149ad
  Leo Gao authored Jun 18, 2021
```
Update `download_file` calls to reflect api arg name change
```
  213149ad
- Update `download_file` calls to reflect api arg name change · 6b0a8926
  Jonathan Tow authored Jun 18, 2021
  
  6b0a8926
- Replace `wget` downloading with `best_download` · c7fd34c2
  Jonathan Tow authored Jun 18, 2021
  
  c7fd34c2
- Implement `MC-TACO` · e5db2a1c
  Jonathan Tow authored Jun 18, 2021
  
  e5db2a1c
16 Jun, 2021 3 commits
- Add detokenizer · ebcb28f2
  Jonathan Tow authored Jun 16, 2021
  
  ebcb28f2
- Remove unnecessary parentheses · d235355a
  Jonathan Tow authored Jun 15, 2021
  
  d235355a
- Fix data path location · 31150fac
  Jonathan Tow authored Jun 15, 2021
  
  31150fac
15 Jun, 2021 1 commit
- Implement `MuTual` · d7076a63
  Jonathan Tow authored Jun 15, 2021
  
  d7076a63
14 Jun, 2021 2 commits
- Update readme · be893e9d
  Leo Gao authored Jun 13, 2021
  
  be893e9d
- Fix typo · 0f4e3df4
  Leo Gao authored Jun 13, 2021
  
  0f4e3df4
12 Jun, 2021 3 commits
- Test training docs too and don't cache downloads · 893ef63f
  Leo Gao authored Jun 12, 2021
  
  893ef63f
- Look at all docs of task in test · 7cbd0bba
  Leo Gao authored Jun 12, 2021
  
  7cbd0bba
- Merge pull request #195 from EleutherAI/sdtblck-patch-1 · dc648d84
  Leo Gao authored Jun 12, 2021
```
Change generate until sequence in coqa to be \nQ:
```
  dc648d84