Commits · 7f7673ec7ad7cb281b4af3ee7afb2bd3fa8174d6 · gaoqiong / lm-evaluation-harness

"vscode:/vscode.git/clone" did not exist on "7917774a360f518af6c0a34180725ada64d707f3"

10 May, 2021 3 commits
- Add comment · 7f7673ec
  Leo Gao authored May 10, 2021
  
  7f7673ec
- Fix tests · 59aff21d
  Leo Gao authored May 10, 2021
  
  59aff21d
- Use sample stddev · 02249855
  Leo Gao authored May 10, 2021
  
  02249855
03 May, 2021 2 commits
- Update metrics.py · c1822c02
  Leo Gao authored May 03, 2021
  
  c1822c02
- Update metrics.py · 0d9cc5a8
  Leo Gao authored May 03, 2021
  
  0d9cc5a8
02 May, 2021 1 commit
- implement stderr calculation · 45fefe9f
  Leo Gao authored May 02, 2021
  
  45fefe9f
11 Apr, 2021 11 commits
- More refactoring of model code · a586a5c4
  Leo Gao authored Apr 11, 2021
  
  a586a5c4
- Refactor gpt3 loglikelihood · ab1fdc54
  Leo Gao authored Apr 11, 2021
  
  ab1fdc54
- Refactor gpt2 loglikelihood · 8352e671
  Leo Gao authored Apr 11, 2021
  
  8352e671
- Fix triviaqa · 5dc4692a
  Leo Gao authored Apr 10, 2021
  
  5dc4692a
- Fix typo · 29c1f228
  Leo Gao authored Apr 10, 2021
  
  29c1f228
- Factor _convert_standard into MultipleChoiceTask · ec16d81d
  Leo Gao authored Apr 10, 2021
  
  ec16d81d
- Fix import · 1ebf41d3
  Leo Gao authored Apr 10, 2021
  
  1ebf41d3
- Make piqa inherit MultipleChoiceTask · 4462e415
  Leo Gao authored Apr 10, 2021
  
  4462e415
- also apply normalization to piqa · 2522c27f
  Ben Wang authored Apr 11, 2021
  
  2522c27f
- Make acc_norm a separate metric · 3eaa493e
  Leo Gao authored Apr 10, 2021
  
  3eaa493e
- do per character loss aggregation for multiple choice tasks (similar to OAI's... · fae5fe66
  Ben Wang authored Apr 11, 2021
```
do per character loss aggregation for multiple choice tasks (similar to OAI's per token aggregation)
```
  fae5fe66
10 Apr, 2021 3 commits
- Use more inheritance · 7ba8c183
  Leo Gao authored Apr 10, 2021
  
  7ba8c183
- add cloze variant of lambada task for allegeldy improved few shot results · 47382717
  Ben Wang authored Apr 10, 2021
  
  47382717
- Fix triviaqa memory consumption problem · e971baac
  Leo Gao authored Apr 09, 2021
```
(or rather, move it to the data download phase)
```
  e971baac
08 Apr, 2021 1 commit
- gpt2: Mask out all tokens above 50256 · 2b8956b8
  Leo Gao authored Apr 07, 2021
  
  2b8956b8
05 Apr, 2021 1 commit
- Implement partial caching · efbe6e7f
  Leo Gao authored Apr 04, 2021
```
Now, if a run gets interrupted halfway, you can easily resume
```
  efbe6e7f
02 Apr, 2021 1 commit
- Roll back last token optimization · f3fee648
  Leo Gao authored Apr 02, 2021
  
  f3fee648
31 Mar, 2021 5 commits
- Add note on accuracy reporting · d78917a7
  Jon Tow authored Mar 31, 2021
  
  d78917a7
- Clean up `Ethics` few-shot prompts · 48ae5457
  Jon Tow authored Mar 31, 2021
  
  48ae5457
- Remove unused simple_accuracy_metric · 59a0104d
  Leo Gao authored Mar 30, 2021
  
  59a0104d
- Add "choices" to logiqa, hendrycksTest · 2ced79f5
  Leo Gao authored Mar 30, 2021
  
  2ced79f5
- Handle GPTNeoConfig not having n_ctx · 49ee4db2
  Leo Gao authored Mar 30, 2021
  
  49ee4db2
29 Mar, 2021 3 commits
- Add example header · 310ca7a8
  Leo Gao authored Mar 29, 2021
  
  310ca7a8
- Implement `Childrens Book Test` dataset download and evaluation · a10856dc
  Jon Tow authored Mar 29, 2021
  
  a10856dc
- fix translation scoring format · 3a2b7df4
  & authored Mar 29, 2021
  
  3a2b7df4
28 Mar, 2021 4 commits
- Rename hendrycks ethics and math · 42659c34
  Leo Gao authored Mar 28, 2021
  
  42659c34
- Rename task · 8de85534
  Leo Gao authored Mar 28, 2021
  
  8de85534
- squad: fix aggregation · cbc5c9c8
  Leo Gao authored Mar 28, 2021
  
  cbc5c9c8
- Update superglue winograd has_test_docs · 5643f20f
  Leo Gao authored Mar 28, 2021
  
  5643f20f
27 Mar, 2021 5 commits
- Fix translation prompting · a69ba385
  Leo Gao authored Mar 26, 2021
  
  a69ba385
- Mark GLUE and SuperGLUE tasks without labelled test docs as not having test docs at all · fa7b9643
  Leo Gao authored Mar 26, 2021
  
  fa7b9643
- Remove import of the non-existent `NLP_TASK` module · ddc044eb
  Jonathan Tow authored Mar 27, 2021
  
  ddc044eb
- Remove unused imports and format imports · d5d19219
  Jonathan Tow authored Mar 26, 2021
  
  d5d19219
- Patch gpt3lm · ca57061c
  Leo Gao authored Mar 26, 2021
  
  ca57061c