Commits · ff314d62f38cd1699267df948b3e1f480412f92b · gaoqiong / lm-evaluation-harness

09 Dec, 2021 1 commit
- Update evaluator.py · c7ae6df1
  Leo Gao authored Dec 08, 2021
  
  c7ae6df1
29 Nov, 2021 1 commit
- Update req_type to request_type · 6b6e6d77
  Leo Gao authored Nov 28, 2021
  
  6b6e6d77
24 Nov, 2021 2 commits
- update GPT3 test data and more docs · 53f6bc34
  Jason Phang authored Nov 23, 2021
  
  53f6bc34
- Some refactor/clean-up, and GPT-3 update · 76ebb792
  Jason Phang authored Nov 23, 2021
  
  76ebb792
30 Oct, 2021 1 commit
- Replace the `fewshot_description` API with a `description_dict` based interface · 8ac99269
  Jonathan Tow authored Oct 30, 2021
  
  8ac99269
30 Sep, 2021 1 commit
- Improve interface and upgrade results dict format · 7a058d69
  Leo Gao authored Sep 30, 2021
  
  7a058d69
27 Aug, 2021 1 commit
- evaloator: take min bootstrap steps for blue/chrf/ter · dd3b4f7a
  Leo Gao authored Aug 26, 2021
  
  dd3b4f7a
26 Aug, 2021 1 commit
- evaluator: fix bleu/chrf/ter taking forever · 82ec4f53
  Leo Gao authored Aug 26, 2021
  
  82ec4f53
05 Jun, 2021 1 commit
- Add task versioning · 105fa974
  Leo Gao authored Jun 04, 2021
  
  105fa974
11 May, 2021 1 commit
- Make bootstrap_iters configurable and fix some tests · 7a39d68b
  Leo Gao authored May 11, 2021
  
  7a39d68b
05 May, 2021 1 commit
- Minor changes · f16c301e
  Leo Gao authored May 05, 2021
  
  f16c301e
03 May, 2021 2 commits
- gpt3 compatibility · fa80f7bd
  Jason Phang authored May 02, 2021
  
  fa80f7bd
- gpt2 perplexity · 9454c839
  Jason Phang authored May 02, 2021
  
  9454c839
02 May, 2021 2 commits
- implement stderr calculation · 45fefe9f
  Leo Gao authored May 02, 2021
  
  45fefe9f
- implement stderr calculation · 726ea95e
  Leo Gao authored May 02, 2021
  
  726ea95e
27 Mar, 2021 2 commits
- Pass random state around to fewshot_examples and fewshot_context · 3b2d5f6c
  Leo Gao authored Mar 26, 2021
  
  3b2d5f6c
- Change to test before val · 35f58b95
  Leo Gao authored Mar 26, 2021
  
  35f58b95
14 Feb, 2021 2 commits
- Typo · 7d5aa3f7
  Leo Gao authored Feb 14, 2021
  
  7d5aa3f7
- Shuffle order of docs in evaluator · 624c038e
  Leo Gao authored Feb 14, 2021
  
  624c038e
11 Feb, 2021 2 commits
- Fixes to make greedy_until work · 432bd44c
  Leo Gao authored Feb 10, 2021
```
# Conflicts:
#	lm_eval/models/gpt2.py
#	lm_eval/tasks/squad.py
```
  432bd44c
- Fixes to make greedy_until work · 7b649ded
  Leo Gao authored Feb 10, 2021
  
  7b649ded
04 Feb, 2021 1 commit

Leo Gao authored Feb 03, 2021

- Extract evaluator (still needs work to clean up)
- Add tests for evaluator
- Fix all the things that break on the new tests
- Misc cleanup

778e0f91

03 Feb, 2021 1 commit
- Add todo comment · 8444c486
  Leo Gao authored Feb 02, 2021
  
  8444c486
02 Feb, 2021 1 commit
- pubmedqa · 1b35f6b9
  jeffhsu3 authored Feb 02, 2021
  
  1b35f6b9
31 Jan, 2021 1 commit
- add webqs evaluation and fallback to test set when validation is unavailable · 34eb121f
  Anthony DiPofi authored Jan 31, 2021
  
  34eb121f
09 Jan, 2021 2 commits
- Move higher_is_better and aggregation into their own functions · a18104a4
  Leo Gao authored Jan 08, 2021
  
  a18104a4
- Refactor and implement SAT evaluation · 0f9c1624
  Leo Gao authored Jan 08, 2021
  
  0f9c1624
02 Jan, 2021 3 commits
- Remove print statement · 984f617b
  Leo Gao authored Jan 02, 2021
  
  984f617b
- Fix up bugs · 04e72904
  Leo Gao authored Jan 02, 2021
  
  04e72904
- Finish plumbing · 011a0b0c
  Leo Gao authored Jan 02, 2021
  
  011a0b0c
28 Dec, 2020 1 commit
- Update · e41a082c
  Leo Gao authored Dec 27, 2020
  
  e41a082c
23 Nov, 2020 4 commits
- Changed arg name to match master · 37fab200
  Stella Biderman authored Nov 23, 2020
  
  37fab200
- added import statement · 760788d5
  Stella Biderman authored Nov 23, 2020
  
  760788d5
- Added truncation flag · 3f5d4beb
  Stella Biderman authored Nov 23, 2020
```
This should allow the user to only do the first few eval tasks.
```
  3f5d4beb
- Add --limit flag · fb5aaf51
  Leo Gao authored Nov 23, 2020
  
  fb5aaf51
14 Sep, 2020 3 commits
- Update docs · f1ec7b06
  Jason Phang authored Sep 14, 2020
  
  f1ec7b06
- cleanup · a85ad214
  Jason Phang authored Sep 14, 2020
  
  a85ad214
- SuperGLUE, and truncation · 8161c22e
  Jason Phang authored Sep 14, 2020
  
  8161c22e
07 Sep, 2020 2 commits
- seed · 9987203f
  Jason Phang authored Sep 07, 2020
  
  9987203f
- lib · f88bb827
  Jason Phang authored Sep 07, 2020
  
  f88bb827