Commits · dd3b4f7a05da5a3bec925a478556f71c20fd74e8 · gaoqiong / lm-evaluation-harness

27 Aug, 2021 1 commit
- evaloator: take min bootstrap steps for blue/chrf/ter · dd3b4f7a
  Leo Gao authored Aug 26, 2021
  
  dd3b4f7a
26 Aug, 2021 2 commits
- evaluator: fix bleu/chrf/ter taking forever · 82ec4f53
  Leo Gao authored Aug 26, 2021
  
  82ec4f53
- Update translation.py · c0f0f7e1
  Leo Gao authored Aug 26, 2021
  
  c0f0f7e1
15 Aug, 2021 1 commit
- Add space tokenization for JA/ZH · f046d331
  Muennighoff authored Aug 15, 2021
  
  f046d331
13 Aug, 2021 1 commit
- Update version and testdata for DROP · 81122e3d
  Leo Gao authored Aug 13, 2021
  
  81122e3d
11 Aug, 2021 2 commits

Bug fix · 5dbeba0d
silentv0x authored Aug 11, 2021

5dbeba0d

silentv0x authored Aug 11, 2021

Following the official implementation, the following changes were made
 - Validated answers are considered gold answers
 - EM/F1 are the max over all gold answers

35d809c3

09 Aug, 2021 1 commit
- fix squad breakage due to HF change · 198ca732
  Leo Gao authored Aug 09, 2021
  
  198ca732
24 Jun, 2021 1 commit
- Don't download lambada if it already exists · 75519897
  sdtblck authored Jun 24, 2021
  
  75519897
20 Jun, 2021 2 commits
- Stable file order and don't lower · 6b6303d4
  Leo Gao authored Jun 20, 2021
  
  6b6303d4
- Fix `MuTual` loglikelihood request bug · 69c83456
  Jonathan Tow authored Jun 20, 2021
  
  69c83456
19 Jun, 2021 1 commit
- Grammar cleanup in the task docstring · 9a58cee6
  Jonathan Tow authored Jun 19, 2021
  
  9a58cee6
18 Jun, 2021 3 commits
- Update `download_file` calls to reflect api arg name change · 6b0a8926
  Jonathan Tow authored Jun 18, 2021
  
  6b0a8926
- Replace `wget` downloading with `best_download` · c7fd34c2
  Jonathan Tow authored Jun 18, 2021
  
  c7fd34c2
- Implement `MC-TACO` · e5db2a1c
  Jonathan Tow authored Jun 18, 2021
  
  e5db2a1c
16 Jun, 2021 3 commits
- Add detokenizer · ebcb28f2
  Jonathan Tow authored Jun 16, 2021
  
  ebcb28f2
- Remove unnecessary parentheses · d235355a
  Jonathan Tow authored Jun 15, 2021
  
  d235355a
- Fix data path location · 31150fac
  Jonathan Tow authored Jun 15, 2021
  
  31150fac
15 Jun, 2021 1 commit
- Implement `MuTual` · d7076a63
  Jonathan Tow authored Jun 15, 2021
  
  d7076a63
12 Jun, 2021 3 commits
- Actually do things correctly this time · f14571fb
  Leo Gao authored Jun 11, 2021
  
  f14571fb
- Bump CoQA version · 278863b5
  Leo Gao authored Jun 11, 2021
  
  278863b5
- Make CoQA take the first line of output excluding any leading whitespace · 2078adba
  Leo Gao authored Jun 11, 2021
  
  2078adba
10 Jun, 2021 9 commits
- Add gpt3 tests · c59cd334
  Leo Gao authored Jun 10, 2021
  
  c59cd334
- Change generate until sequence in coqa to be \nQ: · cc474a94
  sdtblck authored Jun 11, 2021
  
  cc474a94
- Remove a bunch of dead code · 8f8a1dff
  Leo Gao authored Jun 10, 2021
  
  8f8a1dff
- Implement `PROST` task · f4e38081
  Jonathan Tow authored Jun 10, 2021
  
  f4e38081
- Use hashed version stability test instead · f9cc0267
  Leo Gao authored Jun 10, 2021
  
  f9cc0267
- Minor WikiText fix · 10d4b64a
  Leo Gao authored Jun 09, 2021
  
  10d4b64a
- Implement WikiText · 5e31e40e
  Leo Gao authored Jun 09, 2021
  
  5e31e40e
- TriviaQA memory fix · efa99cb2
  Leo Gao authored Jun 09, 2021
  
  efa99cb2
- Omit unimplemented tasks from coverage · c4ecbd6d
  Leo Gao authored Jun 09, 2021
  
  c4ecbd6d
08 Jun, 2021 2 commits
- make hendrycks math order deterministic · 3b990af0
  Leo Gao authored Jun 07, 2021
  
  3b990af0
- Actually fix hendrycks math · 90429563
  Leo Gao authored Jun 07, 2021
  
  90429563
07 Jun, 2021 1 commit
- Fix hendrycks MATH to use new download link · 004b63e3
  Leo Gao authored Jun 07, 2021
  
  004b63e3
05 Jun, 2021 3 commits
- convert all enabled tasks to use best_download for better download robustness · 20229183
  Leo Gao authored Jun 04, 2021
  
  20229183
- fix bug where interrupted pile download doesnt correctly resume · 85277915
  Leo Gao authored Jun 04, 2021
  
  85277915
- Add task versioning · 105fa974
  Leo Gao authored Jun 04, 2021
  
  105fa974
22 May, 2021 1 commit

make create_from_arg_string fully general · 9d6d291f

Leo Gao authored May 21, 2021

for some reason putting it in LM and having it be inherited breaks
everything. should try to figure this out at some point.

9d6d291f

21 May, 2021 1 commit
- Add missing () in fewshot-no-training-path · 031e6aca
  Leo Gao authored May 21, 2021
  
  031e6aca
16 May, 2021 1 commit
- naturalqs: fix spacing · 9c76176b
  Leo Gao authored May 15, 2021
  
  9c76176b