Commits · 341663a92f0c35171eb9cd464acf6ee3bdebfc55 · gaoqiong / lm-evaluation-harness

30 Jan, 2022 1 commit
- Added more eval tasks and fixed directory error tasks · bb0eafbb
  Quentin Anthony authored Jan 30, 2022
  
  bb0eafbb
23 Jan, 2022 1 commit
- Update `headqa` deprecation warning to display on init only · 03c15e06
  Jonathan Tow authored Jan 23, 2022
  
  03c15e06
13 Jan, 2022 1 commit
- Conform `asidv` to the new description api · caff4f10
  Jonathan Tow authored Jan 13, 2022
  
  caff4f10
11 Jan, 2022 2 commits
- Bump version · b421f052
  thomasw21 authored Jan 11, 2022
  
  b421f052
- Fix CB · ea3fd79b
  thomasw21 authored Jan 11, 2022
  
  ea3fd79b
08 Jan, 2022 3 commits
- Missed asdiv · 2d9fc256
  thomasw21 authored Jan 09, 2022
  
  2d9fc256
- Update blimp fewshot_context · 02a4def2
  Leo Gao authored Jan 08, 2022
  
  02a4def2
- Actually it shouldn't be hard to fix it to be compatible with future version · c65412e5
  thomasw21 authored Jan 08, 2022
  
  c65412e5
03 Jan, 2022 1 commit
- Improve pile test efficiency · cb3babd3
  Leo Gao authored Jan 03, 2022
  
  cb3babd3
02 Jan, 2022 1 commit
- asdiv: space convention · 4b3dee67
  Leo Gao authored Jan 01, 2022
  
  4b3dee67
31 Dec, 2021 2 commits
- removed strip_bracket function · 83e1a116
  rokosbasilisk authored Dec 31, 2021
  
  83e1a116
- remove _strip_bracket function · 72d7cc0c
  rokosbasilisk authored Dec 31, 2021
  
  72d7cc0c
30 Dec, 2021 1 commit
- Bump the version number for all tasks based on PerplexityTask · 6653cc50
  Igor Ostrovsky authored Dec 30, 2021
```
This is due to the change in the bits_per_byte calculation.
```
  6653cc50
29 Dec, 2021 4 commits
- remove unrequired files&add pin commit hash · 04635731
  rokosbasilisk authored Dec 29, 2021
  
  04635731
- Pretty sure questions need to be paragraph dependent also · d7e32489
  thomasw21 authored Dec 29, 2021
  
  d7e32489
- Revert "Add capital letters" · 0bde7589
  thomasw21 authored Dec 29, 2021
```
This reverts commit 23a42067.
```
  0bde7589
- pile: Switch download over to backup host temporarily · 5a53c368
  Leo Gao authored Dec 28, 2021
```
We can have fallback instead after @researcher2 implements it in best_download
```
  5a53c368
27 Dec, 2021 1 commit
- Questions in BoolQ don't have interrogation punctuation at the end · 440216db
  thomasw21 authored Dec 27, 2021
  
  440216db
26 Dec, 2021 5 commits
- remove apps · bce9f289
  rokosbasilisk authored Dec 26, 2021
  
  bce9f289
- add asdiv task · b0a12310
  rokosbasilisk authored Dec 26, 2021
  
  b0a12310
- Implement decontamination of evals against training set 13 grams. · 6597c347
  researcher2 authored Dec 25, 2021
  
  6597c347
- Add capital letters · 23a42067
  thomasw21 authored Dec 26, 2021
  
  23a42067
- Fix multirc · df1fc6c3
  thomasw21 authored Dec 26, 2021
  
  df1fc6c3
24 Dec, 2021 5 commits
- Update new `task` arg and task dict getter · 0e232f7a
  Jonathan Tow authored Dec 24, 2021
  
  0e232f7a
- Remove all `provide_description` argument uses · 57d0718a
  Jonathan Tow authored Dec 24, 2021
  
  57d0718a
- more changes · d86aabc4
  Leo Gao authored Dec 23, 2021
  
  d86aabc4
- Make simple_evaluate take LM and Task objects directly too · 7b2b2a23
  Leo Gao authored Dec 23, 2021
  
  7b2b2a23
- headqa: maintain backwards compatibility · cdab2c03
  Leo Gao authored Dec 23, 2021
  
  cdab2c03
23 Dec, 2021 3 commits
- Bump version · a48e3cf2
  thomasw21 authored Dec 23, 2021
  
  a48e3cf2
- Allow users to specify en headqa or es · 5855f482
  thomasw21 authored Dec 23, 2021
  
  5855f482
- FIX wnli · 682e5c44
  thomasw21 authored Dec 23, 2021
  
  682e5c44
22 Dec, 2021 1 commit
- Add BLiMP · 5f48dfc2
  Igor Ostrovsky authored Dec 21, 2021
  
  5f48dfc2
21 Dec, 2021 2 commits
- Format for consistency · aea963a1
  Jonathan Tow authored Dec 21, 2021
  
  aea963a1
- Add positional arg deprecation decorator · 8ebe36b2
  Jonathan Tow authored Dec 21, 2021
  
  8ebe36b2
16 Dec, 2021 1 commit
- Add `provide_description` arg for backward compat · ee53be21
  Jonathan Tow authored Dec 15, 2021
  
  ee53be21
29 Nov, 2021 2 commits
- Pin datasets to 1.15.1 · 784a5c54
  Leo Gao authored Nov 28, 2021
  
  784a5c54
- Pin datasets to 1.16.0 · 13916e32
  Leo Gao authored Nov 28, 2021
  
  13916e32
19 Nov, 2021 1 commit
- Add `truthfulqa` back to the task registry · 33ccde86
  Jonathan Tow authored Nov 19, 2021
  
  33ccde86
16 Nov, 2021 1 commit
- Comment out truthfulqa before tests are fixed · 134150a2
  Leo Gao authored Nov 15, 2021
  
  134150a2
06 Nov, 2021 1 commit
- Add newline separator for correct `rougeLsum` · 83252575
  Jonathan Tow authored Nov 06, 2021
  
  83252575