Commits · baf54a03ce7a11a6796dcbdfa022343c4d04e509 · gaoqiong / lm-evaluation-harness

10 May, 2021 1 commit
- Make bootstrap test work · baf54a03
  Leo Gao authored May 10, 2021
  
  baf54a03
06 May, 2021 5 commits
- Fix Typo · dd573a95
  Leo Gao authored May 06, 2021
  
  dd573a95
- Merge branch 'master' of github.com:EleutherAI/lm_evaluation_harness into perplexity · 1731613e
  Leo Gao authored May 05, 2021
```
# Conflicts:
#	requirements.txt
```
  1731613e
- Merge pull request #180 from EleutherAI/packaging · f9d87ad3
  Leo Gao authored May 05, 2021
```
Shuffle deps
```
  f9d87ad3
- move deps to setup.py so they get installed automatically · 965038da
  Ben Wang authored May 06, 2021
  
  965038da
- Rename a ton of stuff · 45127aa7
  Leo Gao authored May 05, 2021
  
  45127aa7
05 May, 2021 4 commits
- Fix some things for tests · 80a739eb
  Leo Gao authored May 05, 2021
  
  80a739eb
- Minor changes · f16c301e
  Leo Gao authored May 05, 2021
  
  f16c301e
- Refactor PerplexityTask · 1e7f884d
  Leo Gao authored May 05, 2021
  
  1e7f884d
- Begin refactoring perplexity code · b0cf0163
  Leo Gao authored May 04, 2021
  
  b0cf0163
04 May, 2021 1 commit
- adding tests sortof · ee5467ff
  Jason Phang authored May 03, 2021
  
  ee5467ff
03 May, 2021 7 commits
- Test bootstrapping · 218d105d
  Leo Gao authored May 03, 2021
  
  218d105d
- Update metrics.py · c1822c02
  Leo Gao authored May 03, 2021
  
  c1822c02
- Update metrics.py · 0d9cc5a8
  Leo Gao authored May 03, 2021
  
  0d9cc5a8
- add lm_dataformat to requirements · 3a490624
  Jason Phang authored May 02, 2021
  
  3a490624
- add pile tasks · 82880154
  Jason Phang authored May 02, 2021
  
  82880154
- gpt3 compatibility · fa80f7bd
  Jason Phang authored May 02, 2021
  
  fa80f7bd
- gpt2 perplexity · 9454c839
  Jason Phang authored May 02, 2021
  
  9454c839
02 May, 2021 1 commit
- implement stderr calculation · 45fefe9f
  Leo Gao authored May 02, 2021
  
  45fefe9f
14 Apr, 2021 1 commit
- Update README.md · 8846bec0
  Leo Gao authored Apr 13, 2021
  
  8846bec0
11 Apr, 2021 16 commits
- Merge branch 'master' of github.com:EleutherAI/lm_evaluation_harness · c5c09c26
  Leo Gao authored Apr 11, 2021
  
  c5c09c26
- More refactoring of model code · a586a5c4
  Leo Gao authored Apr 11, 2021
  
  a586a5c4
- Refactor gpt3 loglikelihood · ab1fdc54
  Leo Gao authored Apr 11, 2021
  
  ab1fdc54
- Refactor gpt2 loglikelihood · 8352e671
  Leo Gao authored Apr 11, 2021
  
  8352e671
- Merge pull request #169 from jon-tow/cbt-evaluation · edc66ecc
  Leo Gao authored Apr 11, 2021
```
Implement `Childrens Book Test` dataset download and evaluation
```
  edc66ecc
- Merge branch 'master' into cbt-evaluation · cd1b32f9
  Leo Gao authored Apr 11, 2021
  
  cd1b32f9
- Add more gpt2 tests · eec18018
  Leo Gao authored Apr 11, 2021
  
  eec18018
- Fix triviaqa · 5dc4692a
  Leo Gao authored Apr 10, 2021
  
  5dc4692a
- Fix typo · 29c1f228
  Leo Gao authored Apr 10, 2021
  
  29c1f228
- Factor _convert_standard into MultipleChoiceTask · ec16d81d
  Leo Gao authored Apr 10, 2021
  
  ec16d81d
- Merge pull request #176 from EleutherAI/per_char_agg · 44f03593
  Leo Gao authored Apr 10, 2021
```
Do per character loss aggregation for multiple choice tasks
```
  44f03593
- Fix import · 1ebf41d3
  Leo Gao authored Apr 10, 2021
  
  1ebf41d3
- Make piqa inherit MultipleChoiceTask · 4462e415
  Leo Gao authored Apr 10, 2021
  
  4462e415
- also apply normalization to piqa · 2522c27f
  Ben Wang authored Apr 11, 2021
  
  2522c27f
- Make acc_norm a separate metric · 3eaa493e
  Leo Gao authored Apr 10, 2021
  
  3eaa493e
- do per character loss aggregation for multiple choice tasks (similar to OAI's... · fae5fe66
  Ben Wang authored Apr 11, 2021
```
do per character loss aggregation for multiple choice tasks (similar to OAI's per token aggregation)
```
  fae5fe66
10 Apr, 2021 4 commits
- Merge pull request #172 from EleutherAI/lambada_cloze · fd26ef16
  Leo Gao authored Apr 10, 2021
```
add cloze variant of lambada task
```
  fd26ef16
- Use more inheritance · 7ba8c183
  Leo Gao authored Apr 10, 2021
  
  7ba8c183
- add cloze variant of lambada task for allegeldy improved few shot results · 47382717
  Ben Wang authored Apr 10, 2021
  
  47382717
- Fix triviaqa memory consumption problem · e971baac
  Leo Gao authored Apr 09, 2021
```
(or rather, move it to the data download phase)
```
  e971baac