Commits · 3c0fa5a602ce60c17b7790fbc48539a1cdfcd7b7 · gaoqiong / lm-evaluation-harness

03 Mar, 2022 1 commit
- Refactor for PR · f495bfb4
  researcher2 authored Mar 03, 2022
  
  f495bfb4
25 Feb, 2022 4 commits
- Change function signature so package root is always found from utils.py · 606b98f7
  Stephen Hogg authored Feb 25, 2022
  
  606b98f7
- Add integrity check · 4d8dde31
  Stephen Hogg authored Feb 25, 2022
  
  4d8dde31
- further interface changes · 0f283a9c
  Leo Gao authored Feb 24, 2022
  
  0f283a9c
- pull stuff into lm_eval.decontamination · b2460099
  Leo Gao authored Feb 24, 2022
  
  b2460099
20 Feb, 2022 1 commit

researcher2 authored Feb 21, 2022

Remove arguments from main.py.

Add "decontamination" prefix to ngrams arguments.

98d75af0

26 Dec, 2021 1 commit
- Implement decontamination of evals against training set 13 grams. · 6597c347
  researcher2 authored Dec 25, 2021
  
  6597c347
24 Dec, 2021 3 commits
- Remove more `provide_description` uses · a34bbe69
  Jonathan Tow authored Dec 24, 2021
  
  a34bbe69
- more changes · d86aabc4
  Leo Gao authored Dec 23, 2021
  
  d86aabc4
- Make simple_evaluate take LM and Task objects directly too · 7b2b2a23
  Leo Gao authored Dec 23, 2021
  
  7b2b2a23
21 Dec, 2021 2 commits
- Add positional arg deprecation decorator · 8ebe36b2
  Jonathan Tow authored Dec 21, 2021
  
  8ebe36b2
- Add `description_dict` to results config · d34ae3cf
  Jonathan Tow authored Dec 21, 2021
  
  d34ae3cf
16 Dec, 2021 4 commits
- Add type info to doc-string · 1bc6cdb1
  Jonathan Tow authored Dec 16, 2021
  
  1bc6cdb1
- Remove unused import · 3f06b603
  Jonathan Tow authored Dec 16, 2021
  
  3f06b603
- Make `evaluate` and `simple_evaluate` description args consistent · 10dd7d38
  Jonathan Tow authored Dec 15, 2021
  
  10dd7d38
- Add `provide_description` arg for backward compat · ee53be21
  Jonathan Tow authored Dec 15, 2021
  
  ee53be21
15 Dec, 2021 1 commit
- Merge · 1d04c42d
  Jonathan Tow authored Dec 15, 2021
  
  1d04c42d
09 Dec, 2021 1 commit
- Update evaluator.py · c7ae6df1
  Leo Gao authored Dec 08, 2021
  
  c7ae6df1
29 Nov, 2021 1 commit
- Update req_type to request_type · 6b6e6d77
  Leo Gao authored Nov 28, 2021
  
  6b6e6d77
24 Nov, 2021 2 commits
- update GPT3 test data and more docs · 53f6bc34
  Jason Phang authored Nov 23, 2021
  
  53f6bc34
- Some refactor/clean-up, and GPT-3 update · 76ebb792
  Jason Phang authored Nov 23, 2021
  
  76ebb792
30 Oct, 2021 1 commit
- Replace the `fewshot_description` API with a `description_dict` based interface · 8ac99269
  Jonathan Tow authored Oct 30, 2021
  
  8ac99269
30 Sep, 2021 1 commit
- Improve interface and upgrade results dict format · 7a058d69
  Leo Gao authored Sep 30, 2021
  
  7a058d69
27 Aug, 2021 1 commit
- evaloator: take min bootstrap steps for blue/chrf/ter · dd3b4f7a
  Leo Gao authored Aug 26, 2021
  
  dd3b4f7a
26 Aug, 2021 1 commit
- evaluator: fix bleu/chrf/ter taking forever · 82ec4f53
  Leo Gao authored Aug 26, 2021
  
  82ec4f53
05 Jun, 2021 1 commit
- Add task versioning · 105fa974
  Leo Gao authored Jun 04, 2021
  
  105fa974
11 May, 2021 1 commit
- Make bootstrap_iters configurable and fix some tests · 7a39d68b
  Leo Gao authored May 11, 2021
  
  7a39d68b
05 May, 2021 1 commit
- Minor changes · f16c301e
  Leo Gao authored May 05, 2021
  
  f16c301e
03 May, 2021 2 commits
- gpt3 compatibility · fa80f7bd
  Jason Phang authored May 02, 2021
  
  fa80f7bd
- gpt2 perplexity · 9454c839
  Jason Phang authored May 02, 2021
  
  9454c839
02 May, 2021 2 commits
- implement stderr calculation · 45fefe9f
  Leo Gao authored May 02, 2021
  
  45fefe9f
- implement stderr calculation · 726ea95e
  Leo Gao authored May 02, 2021
  
  726ea95e
27 Mar, 2021 2 commits
- Pass random state around to fewshot_examples and fewshot_context · 3b2d5f6c
  Leo Gao authored Mar 26, 2021
  
  3b2d5f6c
- Change to test before val · 35f58b95
  Leo Gao authored Mar 26, 2021
  
  35f58b95
14 Feb, 2021 2 commits
- Typo · 7d5aa3f7
  Leo Gao authored Feb 14, 2021
  
  7d5aa3f7
- Shuffle order of docs in evaluator · 624c038e
  Leo Gao authored Feb 14, 2021
  
  624c038e
11 Feb, 2021 2 commits
- Fixes to make greedy_until work · 432bd44c
  Leo Gao authored Feb 10, 2021
```
# Conflicts:
#	lm_eval/models/gpt2.py
#	lm_eval/tasks/squad.py
```
  432bd44c
- Fixes to make greedy_until work · 7b649ded
  Leo Gao authored Feb 10, 2021
  
  7b649ded
04 Feb, 2021 1 commit

Massive refactor · 778e0f91

Leo Gao authored Feb 03, 2021

- Extract evaluator (still needs work to clean up)
- Add tests for evaluator
- Fix all the things that break on the new tests
- Misc cleanup

778e0f91

03 Feb, 2021 1 commit
- Add todo comment · 8444c486
  Leo Gao authored Feb 02, 2021
  
  8444c486