Commits · 8dbd24f6603fa219991f4621545f0ca0edc30709 · gaoqiong / lm-evaluation-harness

03 Jan, 2022 4 commits
- Merge branch 'master' of github.com:EleutherAI/lm-evaluation-harness into asdiv · 8dbd24f6
  Leo Gao authored Jan 03, 2022
  
  8dbd24f6
- Merge branch 'master' of github.com:EleutherAI/lm-evaluation-harness · a67c17e0
  Leo Gao authored Jan 03, 2022
```
# Conflicts:
#	lm_eval/tasks/pile.py
```
  a67c17e0
- Delete test_cache.db · 9d87d476
  Leo Gao authored Jan 03, 2022
  
  9d87d476
- Improve pile test efficiency · cb3babd3
  Leo Gao authored Jan 03, 2022
  
  cb3babd3
02 Jan, 2022 1 commit
- asdiv: space convention · 4b3dee67
  Leo Gao authored Jan 01, 2022
  
  4b3dee67
31 Dec, 2021 3 commits
- removed strip_bracket function · 83e1a116
  rokosbasilisk authored Dec 31, 2021
  
  83e1a116
- remove _strip_bracket function · 72d7cc0c
  rokosbasilisk authored Dec 31, 2021
  
  72d7cc0c
- Merge pull request #245 from bigscience-workshop/thomas/improve_boolq · 0fe2e80a
  Leo Gao authored Dec 30, 2021
```
Nit on BoolQ
```
  0fe2e80a
29 Dec, 2021 6 commits
- remove unrequired files&add pin commit hash · 04635731
  rokosbasilisk authored Dec 29, 2021
  
  04635731
- Add testdata for boolq v1 · f42a8d66
  thomasw21 authored Dec 29, 2021
  
  f42a8d66
- pile: Switch download over to backup host temporarily · 5a53c368
  Leo Gao authored Dec 28, 2021
```
We can have fallback instead after @researcher2 implements it in best_download
```
  5a53c368
- Update README.md · 0d9d47da
  Leo Gao authored Dec 28, 2021
  
  0d9d47da
- Update README.md · 97ca18e0
  Leo Gao authored Dec 28, 2021
  
  97ca18e0
- Merge pull request #240 from bigscience-workshop/thomas/fix_head_qa · 50ac7dff
  Leo Gao authored Dec 28, 2021
```
Allow users to specify headqa subset
```
  50ac7dff
27 Dec, 2021 1 commit
- Questions in BoolQ don't have interrogation punctuation at the end · 440216db
  thomasw21 authored Dec 27, 2021
  
  440216db
26 Dec, 2021 2 commits
- remove apps · bce9f289
  rokosbasilisk authored Dec 26, 2021
  
  bce9f289
- add asdiv task · b0a12310
  rokosbasilisk authored Dec 26, 2021
  
  b0a12310
24 Dec, 2021 8 commits
- Fix README · e34c6bd6
  thomasw21 authored Dec 24, 2021
  
  e34c6bd6
- Fix README · 666b615f
  thomasw21 authored Dec 24, 2021
  
  666b615f
- Merge remote-tracking branch 'origin/master' into thomas/fix_head_qa · 377a1f45
  thomasw21 authored Dec 24, 2021
  
  377a1f45
- Merge pull request #237 from igor0/blimp · f16e8b5c
  Leo Gao authored Dec 23, 2021
```
Add BLiMP
```
  f16e8b5c
- wnli: add testdata for v1 · 66fffcfc
  Leo Gao authored Dec 23, 2021
  
  66fffcfc
- Add new testdata · 22c4124f
  Leo Gao authored Dec 23, 2021
  
  22c4124f
- headqa: maintain backwards compatibility · cdab2c03
  Leo Gao authored Dec 23, 2021
  
  cdab2c03
- Merge pull request #239 from bigscience-workshop/thomas/fix_wnli · e8e4873a
  Stella Biderman authored Dec 23, 2021
```
FIX wnli
```
  e8e4873a
23 Dec, 2021 3 commits
- Bump version · a48e3cf2
  thomasw21 authored Dec 23, 2021
  
  a48e3cf2
- Allow users to specify en headqa or es · 5855f482
  thomasw21 authored Dec 23, 2021
  
  5855f482
- FIX wnli · 682e5c44
  thomasw21 authored Dec 23, 2021
  
  682e5c44
22 Dec, 2021 1 commit
- Add BLiMP · 5f48dfc2
  Igor Ostrovsky authored Dec 21, 2021
  
  5f48dfc2
09 Dec, 2021 3 commits
- Merge pull request #229 from EleutherAI/lm_refactor · df5d7cf0
  Leo Gao authored Dec 08, 2021
```
Refactor LM organization for more reuse
```
  df5d7cf0
- Bump version to 0.1.0 · 9590f366
  Leo Gao authored Dec 08, 2021
  
  9590f366
- Update evaluator.py · c7ae6df1
  Leo Gao authored Dec 08, 2021
  
  c7ae6df1
05 Dec, 2021 3 commits
- Update test_gpt3.py · a6bb79b4
  Leo Gao authored Dec 05, 2021
  
  a6bb79b4
- Update test_gpt3.py · 01b98d2d
  Leo Gao authored Dec 05, 2021
  
  01b98d2d
- Get rid of get_token_logprobs · 7c011370
  Leo Gao authored Dec 05, 2021
  
  7c011370
01 Dec, 2021 1 commit
- Merge pull request #232 from zphang/lm_refactor_update1 · ae63eeb7
  Leo Gao authored Nov 30, 2021
```
Additional clean-up of LM Refactor
```
  ae63eeb7
29 Nov, 2021 4 commits
- Merge pull request #234 from EleutherAI/race_fix · 67e2bf8b
  Leo Gao authored Nov 29, 2021
```
Pin datasets to 1.15.1 to fix RACE
```
  67e2bf8b
- Pin datasets to 1.15.1 · 784a5c54
  Leo Gao authored Nov 28, 2021
  
  784a5c54
- Update req_type to request_type · 6b6e6d77
  Leo Gao authored Nov 28, 2021
  
  6b6e6d77
- Make everything request_type · 3b386db4
  Leo Gao authored Nov 28, 2021
  
  3b386db4