Commits · 44d7b2fa0e13d46b6897d4b38f6fb2ca17d1daae · gaoqiong / lm-evaluation-harness

29 Mar, 2021 2 commits
- Add example header · 310ca7a8
  Leo Gao authored Mar 29, 2021
  
  310ca7a8
- fix translation scoring format · 3a2b7df4
  & authored Mar 29, 2021
  
  3a2b7df4
28 Mar, 2021 4 commits
- Rename hendrycks ethics and math · 42659c34
  Leo Gao authored Mar 28, 2021
  
  42659c34
- Rename task · 8de85534
  Leo Gao authored Mar 28, 2021
  
  8de85534
- squad: fix aggregation · cbc5c9c8
  Leo Gao authored Mar 28, 2021
  
  cbc5c9c8
- Update superglue winograd has_test_docs · 5643f20f
  Leo Gao authored Mar 28, 2021
  
  5643f20f
27 Mar, 2021 11 commits
- Fix translation prompting · a69ba385
  Leo Gao authored Mar 26, 2021
  
  a69ba385
- Mark GLUE and SuperGLUE tasks without labelled test docs as not having test docs at all · fa7b9643
  Leo Gao authored Mar 26, 2021
  
  fa7b9643
- Remove import of the non-existent `NLP_TASK` module · ddc044eb
  Jonathan Tow authored Mar 27, 2021
  
  ddc044eb
- Remove unused imports and format imports · d5d19219
  Jonathan Tow authored Mar 26, 2021
  
  d5d19219
- Patch gpt3lm · ca57061c
  Leo Gao authored Mar 26, 2021
  
  ca57061c
- Minor updates · bc5478a1
  Leo Gao authored Mar 26, 2021
  
  bc5478a1
- Fix stuff and make tests pass · c971fa82
  Leo Gao authored Mar 26, 2021
  
  c971fa82
- Move fewshot stuff back into the right block · 0966e7b6
  Leo Gao authored Mar 26, 2021
  
  0966e7b6
- Pass random state around to fewshot_examples and fewshot_context · 3b2d5f6c
  Leo Gao authored Mar 26, 2021
  
  3b2d5f6c
- Change to test before val · 35f58b95
  Leo Gao authored Mar 26, 2021
  
  35f58b95
- Fix fewshot for tasks without train docs · 0c402690
  Leo Gao authored Mar 26, 2021
  
  0c402690
26 Mar, 2021 7 commits
- Add Reorderer and implement in gpt2 and gpt3 · 8e8d7c6d
  Leo Gao authored Mar 26, 2021
  
  8e8d7c6d
- More changes to make neo work · 1b4242c1
  Leo Gao authored Mar 26, 2021
  
  1b4242c1
- Update GPT2LM to handle neo based models as well · 747b851d
  Leo Gao authored Mar 26, 2021
  
  747b851d
- qa4mre: remove fewshot_examples that uses test docs · c84a4af4
  Leo Gao authored Mar 25, 2021
  
  c84a4af4
- naturalqs: update random · d7b20fe2
  Leo Gao authored Mar 25, 2021
  
  d7b20fe2
- wsc273: change random · d42af30d
  Leo Gao authored Mar 25, 2021
  
  d42af30d
- PubmedQA: remove redundant fewshot_experiments · dc1a17b1
  Leo Gao authored Mar 25, 2021
  
  dc1a17b1
25 Mar, 2021 3 commits
- Update fewshot_examples · e2112196
  Leo Gao authored Mar 25, 2021
  
  e2112196
- Deterministic fewshot_experiments · 6a78fdaf
  Leo Gao authored Mar 25, 2021
  
  6a78fdaf
- Remove fewshot_examples · 58589f39
  Leo Gao authored Mar 25, 2021
```
Drawing fewshot from train_docs has two main advantages: a) you can do way more than 8 shots and b) your fewshot examples are from the same type of problem
```
  58589f39
24 Mar, 2021 4 commits
- Add `answer` key to training docs for uniformity · d22ea1f3
  Jonathan Tow authored Mar 24, 2021
  
  d22ea1f3
- Split `test` and `train` doc loading and address minor fix-ups · 410ca65f
  Jonathan Tow authored Mar 24, 2021
  
  410ca65f
- Fixed calling of loglikelihood within SQuAD task · 14dd29c4
  Charles Foster authored Mar 23, 2021
  
  14dd29c4
- SQuAD fixed to use loglikelihood API to calculate the probability of an unanswerable question. · 5be42b4d
  Charles Foster authored Mar 23, 2021
  
  5be42b4d
22 Mar, 2021 5 commits
- Include `auxiliary_train` and all `dev` sets in `training_docs` · e3769082
  Jonathan Tow authored Mar 22, 2021
  
  e3769082
- Split `dev` and `val` sets and cleanup prompt spacing · 364e54d4
  Jonathan Tow authored Mar 22, 2021
  
  364e54d4
- Fix `LogiQA` prompts · aadf2f41
  Jonathan Tow authored Mar 22, 2021
  
  aadf2f41
- Restore proper prompt formatting · b2bab740
  Jonathan Tow authored Mar 22, 2021
  
  b2bab740
- Refactor and simple formatting · 806b022b
  Jonathan Tow authored Mar 21, 2021
  
  806b022b
21 Mar, 2021 4 commits
- minor · 13b97626
  Andy Zou authored Mar 21, 2021
  
  13b97626
- added training · 7989168d
  Andy Zou authored Mar 21, 2021
  
  7989168d
- adjusted naming · 8888a6be
  Andy Zou authored Mar 20, 2021
  
  8888a6be
- add hendrycksTest · a6b01474
  Andy Zou authored Mar 20, 2021
  
  a6b01474