Commits · 8e8d7c6de3b5d334389305f417e551807eb8d74e · gaoqiong / lm-evaluation-harness

26 Mar, 2021 7 commits
- Add Reorderer and implement in gpt2 and gpt3 · 8e8d7c6d
  Leo Gao authored Mar 26, 2021
  
  8e8d7c6d
- More changes to make neo work · 1b4242c1
  Leo Gao authored Mar 26, 2021
  
  1b4242c1
- Update GPT2LM to handle neo based models as well · 747b851d
  Leo Gao authored Mar 26, 2021
  
  747b851d
- qa4mre: remove fewshot_examples that uses test docs · c84a4af4
  Leo Gao authored Mar 25, 2021
  
  c84a4af4
- naturalqs: update random · d7b20fe2
  Leo Gao authored Mar 25, 2021
  
  d7b20fe2
- wsc273: change random · d42af30d
  Leo Gao authored Mar 25, 2021
  
  d42af30d
- PubmedQA: remove redundant fewshot_experiments · dc1a17b1
  Leo Gao authored Mar 25, 2021
  
  dc1a17b1
25 Mar, 2021 3 commits
- Update fewshot_examples · e2112196
  Leo Gao authored Mar 25, 2021
  
  e2112196
- Deterministic fewshot_experiments · 6a78fdaf
  Leo Gao authored Mar 25, 2021
  
  6a78fdaf
- Remove fewshot_examples · 58589f39
  Leo Gao authored Mar 25, 2021
```
Drawing fewshot from train_docs has two main advantages: a) you can do way more than 8 shots and b) your fewshot examples are from the same type of problem
```
  58589f39
24 Mar, 2021 2 commits
- Add `answer` key to training docs for uniformity · d22ea1f3
  Jonathan Tow authored Mar 24, 2021
  
  d22ea1f3
- Split `test` and `train` doc loading and address minor fix-ups · 410ca65f
  Jonathan Tow authored Mar 24, 2021
  
  410ca65f
22 Mar, 2021 5 commits
- Include `auxiliary_train` and all `dev` sets in `training_docs` · e3769082
  Jonathan Tow authored Mar 22, 2021
  
  e3769082
- Split `dev` and `val` sets and cleanup prompt spacing · 364e54d4
  Jonathan Tow authored Mar 22, 2021
  
  364e54d4
- Fix `LogiQA` prompts · aadf2f41
  Jonathan Tow authored Mar 22, 2021
  
  aadf2f41
- Restore proper prompt formatting · b2bab740
  Jonathan Tow authored Mar 22, 2021
  
  b2bab740
- Refactor and simple formatting · 806b022b
  Jonathan Tow authored Mar 21, 2021
  
  806b022b
21 Mar, 2021 4 commits
- minor · 13b97626
  Andy Zou authored Mar 21, 2021
  
  13b97626
- added training · 7989168d
  Andy Zou authored Mar 21, 2021
  
  7989168d
- adjusted naming · 8888a6be
  Andy Zou authored Mar 20, 2021
  
  8888a6be
- add hendrycksTest · a6b01474
  Andy Zou authored Mar 20, 2021
  
  a6b01474
19 Mar, 2021 6 commits
- remove testing code again · 6d997710
  Eric Tang authored Mar 19, 2021
  
  6d997710
- fixed linter issue · 1e044b92
  Eric Tang authored Mar 19, 2021
  
  1e044b92
- removed testing comments · 52c7ec8d
  Eric Tang authored Mar 19, 2021
  
  52c7ec8d
- Fixes to process_results string parsing · dea15510
  Eric Tang authored Mar 19, 2021
  
  dea15510
- Update minor fixes in math.py · bbac75cf
  Eric Tang authored Mar 19, 2021
  
  bbac75cf
- Added math tasks · 72f91040
  Eric Tang authored Mar 19, 2021
  
  72f91040
08 Mar, 2021 1 commit
- Implement `LogiQA` data download and evaluation · a187cd44
  Jonathan Tow authored Mar 08, 2021
  
  a187cd44
07 Mar, 2021 2 commits
- Minor tweaks · f1ac6435
  Leo Gao authored Mar 07, 2021
  
  f1ac6435
- Address the `EthicsUtilitarianism` `TODO`s · ab4939d3
  Jonathan Tow authored Mar 07, 2021
  
  ab4939d3
06 Mar, 2021 1 commit
- Flag down a bunch of todos · 491283c5
  Leo Gao authored Mar 05, 2021
  
  491283c5
01 Mar, 2021 2 commits
- Cleanups · 90b56db5
  Muennighoff authored Mar 01, 2021
  
  90b56db5
- Mimic original Ethics Util procedure · 001c0a05
  Muennighoff authored Mar 01, 2021
  
  001c0a05
27 Feb, 2021 3 commits
- Fix splitting · d6f6ff1d
  Muennighoff authored Feb 27, 2021
  
  d6f6ff1d
- Reasonable & Unreasonable loglikelihood · 90f1709b
  Muennighoff authored Feb 27, 2021
  
  90f1709b
- Swap .index for .split · 5b06573e
  Muennighoff authored Feb 27, 2021
  
  5b06573e
26 Feb, 2021 2 commits
- Make some very minor changes to the prompt · 35323047
  Leo Gao authored Feb 26, 2021
  
  35323047
- Remove custom fewshot examples · 37d4f617
  Leo Gao authored Feb 26, 2021
  
  37d4f617
24 Feb, 2021 2 commits
- Implement all word scrambling and manipulation tasks · 41a94e13
  Jon Tow authored Feb 24, 2021
  
  41a94e13
- Implement `Anagrams` evaluations · a5e135b6
  Jon Tow authored Feb 23, 2021
  
  a5e135b6