Commits · b720a9cb31bfb1c02884b982a547befbc3752e21 · gaoqiong / lm-evaluation-harness

26 Feb, 2021 1 commit
- Remove custom fewshot examples · 37d4f617
  Leo Gao authored Feb 26, 2021
  
  37d4f617
24 Feb, 2021 2 commits
- Implement all word scrambling and manipulation tasks · 41a94e13
  Jon Tow authored Feb 24, 2021
  
  41a94e13
- Implement `Anagrams` evaluations · a5e135b6
  Jon Tow authored Feb 23, 2021
  
  a5e135b6
23 Feb, 2021 1 commit
- Fix translation context · ba9c13b2
  Leo Gao authored Feb 22, 2021
  
  ba9c13b2
21 Feb, 2021 8 commits
- Add acknowledgement to official implementation · 805b541c
  Jon Tow authored Feb 21, 2021
  
  805b541c
- Remove print statements · 5894761a
  Jon Tow authored Feb 21, 2021
  
  5894761a
- Undo formatting · 41a8b582
  Jon Tow authored Feb 21, 2021
  
  41a8b582
- Clean up · e4d852a0
  Jon Tow authored Feb 21, 2021
  
  e4d852a0
- Fixes · f4f7618a
  Jon Tow authored Feb 21, 2021
  
  f4f7618a
- Fix gpt3 batching bug · 03f34463
  Leo Gao authored Feb 20, 2021
  
  03f34463
- Add gpt3 chunking · 1ff4e07f
  Leo Gao authored Feb 20, 2021
  
  1ff4e07f
- Implement `DROP` evaluation · fc7cd630
  Jon Tow authored Feb 20, 2021
  
  fc7cd630
19 Feb, 2021 1 commit
- Add gpt2/3 tokenizer sanity check · 77b44470
  Leo Gao authored Feb 18, 2021
  
  77b44470
14 Feb, 2021 9 commits
- Typo · 7d5aa3f7
  Leo Gao authored Feb 14, 2021
  
  7d5aa3f7
- Shuffle order of docs in evaluator · 624c038e
  Leo Gao authored Feb 14, 2021
  
  624c038e
- Fix CoQA for tests · 431e53de
  Leo Gao authored Feb 14, 2021
  
  431e53de
- Add GPT2 greedy_until truncation · 8966289a
  Leo Gao authored Feb 14, 2021
  
  8966289a
- fix metric iterable checks · d0a301cc
  & authored Feb 14, 2021
  
  d0a301cc
- move task list to init file · b1619834
  & authored Feb 14, 2021
  
  b1619834
- look at available pairs languages · b5bb7f6c
  & authored Feb 14, 2021
  
  b5bb7f6c
- ReCoRD fixup · 2cfdd80a
  Jason Phang authored Feb 13, 2021
  
  2cfdd80a
- tweak doc format · 2719a522
  Jason Phang authored Feb 13, 2021
  
  2719a522
13 Feb, 2021 6 commits
- Call greedy_until continuation, download fix · 68122ef4
  thefazzer authored Feb 13, 2021
  
  68122ef4
- ReCoRD fix · e4e9228e
  Jason Phang authored Feb 13, 2021
  
  e4e9228e
- Refactor `HeadQA` as a `MultipleChoiceTask` · 97174266
  Jonathan Tow authored Feb 13, 2021
  
  97174266
- Update lambada data location · e5a9de80
  Leo Gao authored Feb 12, 2021
  
  e5a9de80
- small change to regex · b07ecab4
  Anthony DiPofi authored Feb 12, 2021
  
  b07ecab4
- change mathqa to use numeric answer strings instead of a,b,c,d,e as choices · 81e42932
  Anthony DiPofi authored Feb 12, 2021
  
  81e42932
12 Feb, 2021 10 commits
- ... · e56381f4
  & authored Feb 12, 2021
  
  e56381f4
- adds missing task error · 1eb91d36
  & authored Feb 12, 2021
  
  1eb91d36
- Fixed metrics. Translation works · 61aa411b
  & authored Feb 12, 2021
  
  61aa411b
- add tasks to registry · 1297e342
  & authored Feb 12, 2021
  
  1297e342
- metrics · 12ba8426
  & authored Feb 12, 2021
  
  12ba8426
- refactor MathQA as MultipleChoiceTask, uses full words for Q. and A. · c6c67272
  Anthony DiPofi authored Feb 12, 2021
  
  c6c67272
- translation task draft 1. Not hooked up, no metrics · bd3fcce0
  & authored Feb 12, 2021
  
  bd3fcce0
- metrics file · 1fb90b91
  & authored Feb 12, 2021
  
  1fb90b91
- Refactor `ARC` as a `MultipleChoiceTask` · 24ac76df
  Jon Tow authored Feb 11, 2021
  
  24ac76df
- add headqa and mathqa datasets · 742b5df2
  Anthony DiPofi authored Feb 11, 2021
  
  742b5df2
11 Feb, 2021 2 commits
- Fixes to make greedy_until work · 432bd44c
  Leo Gao authored Feb 10, 2021
```
# Conflicts:
#	lm_eval/models/gpt2.py
#	lm_eval/tasks/squad.py
```
  432bd44c
- Implement GPT2 greedy_until · e8f9dc71
  Leo Gao authored Feb 10, 2021
  
  e8f9dc71