Commits · 0cdcc9891f5fbd182d2f30ce2f0c17bb0afc7330 · gaoqiong / lm-evaluation-harness

26 Apr, 2022 1 commit
- Turned off generation tasks for now. Changed process to look at the metrics.... · 0cdcc989
  jon-tow authored Apr 25, 2022
```
Turned off generation tasks for now. Changed process to look at the metrics. Only accuracy implemented.
```
  0cdcc989
25 Apr, 2022 3 commits
- Fix prompt source rank choice accuracy · 2bfa4518
  jon-tow authored Apr 25, 2022
  
  2bfa4518
- Fix coqa · 9484eecc
  jon-tow authored Apr 25, 2022
  
  9484eecc
- Initial integration · 88745155
  cjlovering authored Apr 25, 2022
  
  88745155
25 Feb, 2022 2 commits
- Change function signature so package root is always found from utils.py · 606b98f7
  Stephen Hogg authored Feb 25, 2022
  
  606b98f7
- Add integrity check · 4d8dde31
  Stephen Hogg authored Feb 25, 2022
  
  4d8dde31
24 Dec, 2021 3 commits
- Remove more `provide_description` uses · a34bbe69
  Jonathan Tow authored Dec 24, 2021
  
  a34bbe69
- more changes · d86aabc4
  Leo Gao authored Dec 23, 2021
  
  d86aabc4
- Make simple_evaluate take LM and Task objects directly too · 7b2b2a23
  Leo Gao authored Dec 23, 2021
  
  7b2b2a23
21 Dec, 2021 2 commits
- Add positional arg deprecation decorator · 8ebe36b2
  Jonathan Tow authored Dec 21, 2021
  
  8ebe36b2
- Add `description_dict` to results config · d34ae3cf
  Jonathan Tow authored Dec 21, 2021
  
  d34ae3cf
16 Dec, 2021 4 commits
- Add type info to doc-string · 1bc6cdb1
  Jonathan Tow authored Dec 16, 2021
  
  1bc6cdb1
- Remove unused import · 3f06b603
  Jonathan Tow authored Dec 16, 2021
  
  3f06b603
- Make `evaluate` and `simple_evaluate` description args consistent · 10dd7d38
  Jonathan Tow authored Dec 15, 2021
  
  10dd7d38
- Add `provide_description` arg for backward compat · ee53be21
  Jonathan Tow authored Dec 15, 2021
  
  ee53be21
15 Dec, 2021 1 commit
- Merge · 1d04c42d
  Jonathan Tow authored Dec 15, 2021
  
  1d04c42d
09 Dec, 2021 1 commit
- Update evaluator.py · c7ae6df1
  Leo Gao authored Dec 08, 2021
  
  c7ae6df1
29 Nov, 2021 1 commit
- Update req_type to request_type · 6b6e6d77
  Leo Gao authored Nov 28, 2021
  
  6b6e6d77
24 Nov, 2021 2 commits
- update GPT3 test data and more docs · 53f6bc34
  Jason Phang authored Nov 23, 2021
  
  53f6bc34
- Some refactor/clean-up, and GPT-3 update · 76ebb792
  Jason Phang authored Nov 23, 2021
  
  76ebb792
30 Oct, 2021 1 commit
- Replace the `fewshot_description` API with a `description_dict` based interface · 8ac99269
  Jonathan Tow authored Oct 30, 2021
  
  8ac99269
30 Sep, 2021 1 commit
- Improve interface and upgrade results dict format · 7a058d69
  Leo Gao authored Sep 30, 2021
  
  7a058d69
27 Aug, 2021 1 commit
- evaloator: take min bootstrap steps for blue/chrf/ter · dd3b4f7a
  Leo Gao authored Aug 26, 2021
  
  dd3b4f7a
26 Aug, 2021 1 commit
- evaluator: fix bleu/chrf/ter taking forever · 82ec4f53
  Leo Gao authored Aug 26, 2021
  
  82ec4f53
05 Jun, 2021 1 commit
- Add task versioning · 105fa974
  Leo Gao authored Jun 04, 2021
  
  105fa974
11 May, 2021 1 commit
- Make bootstrap_iters configurable and fix some tests · 7a39d68b
  Leo Gao authored May 11, 2021
  
  7a39d68b
05 May, 2021 1 commit
- Minor changes · f16c301e
  Leo Gao authored May 05, 2021
  
  f16c301e
03 May, 2021 2 commits
- gpt3 compatibility · fa80f7bd
  Jason Phang authored May 02, 2021
  
  fa80f7bd
- gpt2 perplexity · 9454c839
  Jason Phang authored May 02, 2021
  
  9454c839
02 May, 2021 2 commits
- implement stderr calculation · 45fefe9f
  Leo Gao authored May 02, 2021
  
  45fefe9f
- implement stderr calculation · 726ea95e
  Leo Gao authored May 02, 2021
  
  726ea95e
27 Mar, 2021 2 commits
- Pass random state around to fewshot_examples and fewshot_context · 3b2d5f6c
  Leo Gao authored Mar 26, 2021
  
  3b2d5f6c
- Change to test before val · 35f58b95
  Leo Gao authored Mar 26, 2021
  
  35f58b95
14 Feb, 2021 2 commits
- Typo · 7d5aa3f7
  Leo Gao authored Feb 14, 2021
  
  7d5aa3f7
- Shuffle order of docs in evaluator · 624c038e
  Leo Gao authored Feb 14, 2021
  
  624c038e
11 Feb, 2021 2 commits
- Fixes to make greedy_until work · 432bd44c
  Leo Gao authored Feb 10, 2021
```
# Conflicts:
#	lm_eval/models/gpt2.py
#	lm_eval/tasks/squad.py
```
  432bd44c
- Fixes to make greedy_until work · 7b649ded
  Leo Gao authored Feb 10, 2021
  
  7b649ded
04 Feb, 2021 1 commit

Leo Gao authored Feb 03, 2021

- Extract evaluator (still needs work to clean up)
- Add tests for evaluator
- Fix all the things that break on the new tests
- Misc cleanup

778e0f91

03 Feb, 2021 1 commit
- Add todo comment · 8444c486
  Leo Gao authored Feb 02, 2021
  
  8444c486
02 Feb, 2021 1 commit
- pubmedqa · 1b35f6b9
  jeffhsu3 authored Feb 02, 2021
  
  1b35f6b9