Commits · 8809c5f15bc53f2543006ea9811ac576f645e30b · gaoqiong / lm-evaluation-harness

10 Mar, 2021 1 commit
- Merge pull request #159 from jon-tow/logiqa-evaluation · 8809c5f1
  Leo Gao authored Mar 09, 2021
```
Implement `LogiQA` data download and evaluation
```
  8809c5f1
08 Mar, 2021 1 commit
- Implement `LogiQA` data download and evaluation · a187cd44
  Jonathan Tow authored Mar 08, 2021
  
  a187cd44
07 Mar, 2021 4 commits
- Update README.md · 36485d7a
  Leo Gao authored Mar 07, 2021
  
  36485d7a
- Merge pull request #154 from Muennighoff/master · 1de3b743
  Leo Gao authored Mar 07, 2021
```
Add Ethics Dataset
```
  1de3b743
- Minor tweaks · f1ac6435
  Leo Gao authored Mar 07, 2021
  
  f1ac6435
- Address the `EthicsUtilitarianism` `TODO`s · ab4939d3
  Jonathan Tow authored Mar 07, 2021
  
  ab4939d3
06 Mar, 2021 5 commits
- Merge branch 'master' into master · bf27b2c1
  Leo Gao authored Mar 06, 2021
  
  bf27b2c1
- Merge pull request #1 from EleutherAI/mhf · 94977a62
  Muennighoff authored Mar 06, 2021
```
try to unbreak things
```
  94977a62
- Merge pull request #155 from jon-tow/drop-evaluation · b720a9cb
  Leo Gao authored Mar 05, 2021
```
Implement `DROP` evaluation
```
  b720a9cb
- Merge branch 'master' of github.com:EleutherAI/lm_evaluation_harness into mhf · e48b7884
  Leo Gao authored Mar 05, 2021
```
# Conflicts:
#	lm_eval/tasks/__init__.py
```
  e48b7884
- Flag down a bunch of todos · 491283c5
  Leo Gao authored Mar 05, 2021
  
  491283c5
01 Mar, 2021 2 commits
- Cleanups · 90b56db5
  Muennighoff authored Mar 01, 2021
  
  90b56db5
- Mimic original Ethics Util procedure · 001c0a05
  Muennighoff authored Mar 01, 2021
  
  001c0a05
28 Feb, 2021 1 commit
- Add proper normalization, checksum, and formatting · d986fd3c
  Jon Tow authored Feb 28, 2021
  
  d986fd3c
27 Feb, 2021 4 commits
- Implement fewshot description experiment · 879aabd6
  Leo Gao authored Feb 27, 2021
  
  879aabd6
- Fix splitting · d6f6ff1d
  Muennighoff authored Feb 27, 2021
  
  d6f6ff1d
- Reasonable & Unreasonable loglikelihood · 90f1709b
  Muennighoff authored Feb 27, 2021
  
  90f1709b
- Swap .index for .split · 5b06573e
  Muennighoff authored Feb 27, 2021
  
  5b06573e
26 Feb, 2021 3 commits
- Make some very minor changes to the prompt · 35323047
  Leo Gao authored Feb 26, 2021
  
  35323047
- Merge pull request #156 from jon-tow/anagrams-evaluation · 2c41ecf6
  Leo Gao authored Feb 26, 2021
```
Implement symbol manipulation based evaluations
```
  2c41ecf6
- Remove custom fewshot examples · 37d4f617
  Leo Gao authored Feb 26, 2021
  
  37d4f617
24 Feb, 2021 2 commits
- Implement all word scrambling and manipulation tasks · 41a94e13
  Jon Tow authored Feb 24, 2021
  
  41a94e13
- Implement `Anagrams` evaluations · a5e135b6
  Jon Tow authored Feb 23, 2021
  
  a5e135b6
23 Feb, 2021 1 commit
- Fix translation context · ba9c13b2
  Leo Gao authored Feb 22, 2021
  
  ba9c13b2
21 Feb, 2021 9 commits
- Add acknowledgement to official implementation · 805b541c
  Jon Tow authored Feb 21, 2021
  
  805b541c
- Remove print statements · 5894761a
  Jon Tow authored Feb 21, 2021
  
  5894761a
- Undo formatting · 41a8b582
  Jon Tow authored Feb 21, 2021
  
  41a8b582
- Clean up · e4d852a0
  Jon Tow authored Feb 21, 2021
  
  e4d852a0
- Fixes · f4f7618a
  Jon Tow authored Feb 21, 2021
  
  f4f7618a
- Merge branch 'master' of https://github.com/EleutherAI/lm-evaluation-harness into drop-evaluation · 93ebce43
  Jon Tow authored Feb 21, 2021
  
  93ebce43
- Fix gpt3 batching bug · 03f34463
  Leo Gao authored Feb 20, 2021
  
  03f34463
- Add gpt3 chunking · 1ff4e07f
  Leo Gao authored Feb 20, 2021
  
  1ff4e07f
- Implement `DROP` evaluation · fc7cd630
  Jon Tow authored Feb 20, 2021
  
  fc7cd630
19 Feb, 2021 2 commits
- Merge branch 'master' of github.com:EleutherAI/lm_evaluation_harness · f3bf1c07
  Leo Gao authored Feb 18, 2021
  
  f3bf1c07
- Add gpt2/3 tokenizer sanity check · 77b44470
  Leo Gao authored Feb 18, 2021
  
  77b44470
18 Feb, 2021 3 commits
- Adapt prompts to Ethics paper · 86c4115d
  Muennighoff authored Feb 18, 2021
  
  86c4115d
- Add Exact Match Metric · a7564ff3
  Muennighoff authored Feb 18, 2021
  
  a7564ff3
- Update main.py · a64df04b
  Leo Gao authored Feb 17, 2021
  
  a64df04b
16 Feb, 2021 2 commits
- Add Ethics · fef52be4
  Muennighoff authored Feb 16, 2021
  
  fef52be4
- Correct Scoring · e3bdca80
  Muennighoff authored Feb 16, 2021
  
  e3bdca80