Commits · b2f6bce86c62a40498d863d103bd3c25c6d7f672 · gaoqiong / lm-evaluation-harness

08 Jan, 2022 1 commit
- Fix fewshot_context method handling · b2f6bce8
  Leo Gao authored Jan 08, 2022
  
  b2f6bce8
04 Jan, 2022 1 commit
- Enforce `rnd` args with assertions · d2636b4e
  Jonathan Tow authored Jan 04, 2022
  
  d2636b4e
24 Dec, 2021 5 commits
- Update new `task` arg and task dict getter · 0e232f7a
  Jonathan Tow authored Dec 24, 2021
  
  0e232f7a
- Remove all `provide_description` argument uses · 57d0718a
  Jonathan Tow authored Dec 24, 2021
  
  57d0718a
- Remove more `provide_description` uses · a34bbe69
  Jonathan Tow authored Dec 24, 2021
  
  a34bbe69
- more changes · d86aabc4
  Leo Gao authored Dec 23, 2021
  
  d86aabc4
- Make simple_evaluate take LM and Task objects directly too · 7b2b2a23
  Leo Gao authored Dec 23, 2021
  
  7b2b2a23
21 Dec, 2021 3 commits
- Format for consistency · aea963a1
  Jonathan Tow authored Dec 21, 2021
  
  aea963a1
- Add positional arg deprecation decorator · 8ebe36b2
  Jonathan Tow authored Dec 21, 2021
  
  8ebe36b2
- Add `description_dict` to results config · d34ae3cf
  Jonathan Tow authored Dec 21, 2021
  
  d34ae3cf
16 Dec, 2021 6 commits
- Add type info to doc-string · 1bc6cdb1
  Jonathan Tow authored Dec 16, 2021
  
  1bc6cdb1
- Remove unused import · 3f06b603
  Jonathan Tow authored Dec 16, 2021
  
  3f06b603
- Fix assertion error message · 09e1de93
  Jonathan Tow authored Dec 16, 2021
  
  09e1de93
- Make `evaluate` and `simple_evaluate` description args consistent · 10dd7d38
  Jonathan Tow authored Dec 15, 2021
  
  10dd7d38
- Fix assertion error string · 09cd76c7
  Jonathan Tow authored Dec 15, 2021
  
  09cd76c7
- Add `provide_description` arg for backward compat · ee53be21
  Jonathan Tow authored Dec 15, 2021
  
  ee53be21
15 Dec, 2021 1 commit
- Merge · 1d04c42d
  Jonathan Tow authored Dec 15, 2021
  
  1d04c42d
09 Dec, 2021 1 commit
- Update evaluator.py · c7ae6df1
  Leo Gao authored Dec 08, 2021
  
  c7ae6df1
05 Dec, 2021 1 commit
- Get rid of get_token_logprobs · 7c011370
  Leo Gao authored Dec 05, 2021
  
  7c011370
29 Nov, 2021 5 commits
- Pin datasets to 1.15.1 · 784a5c54
  Leo Gao authored Nov 28, 2021
  
  784a5c54
- Update req_type to request_type · 6b6e6d77
  Leo Gao authored Nov 28, 2021
  
  6b6e6d77
- Make everything request_type · 3b386db4
  Leo Gao authored Nov 28, 2021
  
  3b386db4
- Update base.py · 24fde4a6
  Leo Gao authored Nov 28, 2021
  
  24fde4a6
- Pin datasets to 1.16.0 · 13916e32
  Leo Gao authored Nov 28, 2021
  
  13916e32
24 Nov, 2021 2 commits
- update GPT3 test data and more docs · 53f6bc34
  Jason Phang authored Nov 23, 2021
  
  53f6bc34
- Some refactor/clean-up, and GPT-3 update · 76ebb792
  Jason Phang authored Nov 23, 2021
  
  76ebb792
19 Nov, 2021 1 commit
- Add `truthfulqa` back to the task registry · 33ccde86
  Jonathan Tow authored Nov 19, 2021
  
  33ccde86
16 Nov, 2021 1 commit
- Comment out truthfulqa before tests are fixed · 134150a2
  Leo Gao authored Nov 15, 2021
  
  134150a2
06 Nov, 2021 1 commit
- Add newline separator for correct `rougeLsum` · 83252575
  Jonathan Tow authored Nov 06, 2021
  
  83252575
05 Nov, 2021 3 commits
- Add imports and fix comment · 2ff1f5a2
  Leo Gao authored Nov 05, 2021
  
  2ff1f5a2
- Switch gpt3 to BaseLM · 8ef7a515
  Leo Gao authored Nov 05, 2021
  
  8ef7a515
- Merge TokenizedLM and TorchLM into BaseLM · fea936e6
  Leo Gao authored Nov 05, 2021
  
  fea936e6
04 Nov, 2021 3 commits
- Remove `t5` imports` · b0746b35
  Jonathan Tow authored Nov 04, 2021
  
  b0746b35
- Add newline to EOF · e3c74cc0
  Jonathan Tow authored Nov 04, 2021
  
  e3c74cc0
- Remove `t5` dependency · 0f27316c
  Jonathan Tow authored Nov 04, 2021
  
  0f27316c
03 Nov, 2021 1 commit
- Bump the `truthfulqa` version to 1 · 05ed92a4
  Jonathan Tow authored Nov 03, 2021
  
  05ed92a4
31 Oct, 2021 2 commits
- Pin the datasets to the original source · 72257924
  Jonathan Tow authored Oct 30, 2021
  
  72257924
- Update `truthfulqa` dataset checksums · 2cfbfd72
  Jonathan Tow authored Oct 30, 2021
  
  2cfbfd72
30 Oct, 2021 2 commits
- Replace the `fewshot_description` API with a `description_dict` based interface · 8ac99269
  Jonathan Tow authored Oct 30, 2021
  
  8ac99269
- Always mkdir with parents · e7dd78ef
  Jun Shern Chan authored Oct 30, 2021
  
  e7dd78ef