Commits · 96ea7ddce14daaabe3da8ec3a9ce520e72c9c535 · gaoqiong / lm-evaluation-harness

"vscode:/vscode.git/clone" did not exist on "73ea484af19b1b411f1ce0cbdd4cf3a256c7c7f2"

26 Apr, 2022 19 commits
- Added stoppping criteria for generation · 96ea7ddc
  Tian Yun authored Apr 26, 2022
  
  96ea7ddc
- Force no caching while testing · c27e29e1
  jon-tow authored Apr 26, 2022
  
  c27e29e1
- Merge branch 'master' of github.com:cjlovering/ps-eh · f33717c2
  cjlovering authored Apr 25, 2022
  
  f33717c2
- Update the token encode · c26f1d4c
  cjlovering authored Apr 25, 2022
  
  c26f1d4c
- Fix `wsc` subset name · b988137d
  jon-tow authored Apr 25, 2022
  
  b988137d
- Print table across task and prompt · 422380cc
  cjlovering authored Apr 25, 2022
  
  422380cc
- Merge branch 'master' of github.com:cjlovering/ps-eh · 053afe05
  cjlovering authored Apr 25, 2022
  
  053afe05
- Turned off generation tasks for now. Changed process to look at the metrics.... · 0cdcc989
  jon-tow authored Apr 25, 2022
```
Turned off generation tasks for now. Changed process to look at the metrics. Only accuracy implemented.
```
  0cdcc989
- Add `HANS` · 0e578306
  jon-tow authored Apr 25, 2022
  
  0e578306
- QQP with PS integration. · b1a3c6e3
  cjlovering authored Apr 25, 2022
  
  b1a3c6e3
- MRPC with PS integration. · 26e94211
  cjlovering authored Apr 25, 2022
  
  26e94211
- QNLI with PS integration. · b44aa554
  cjlovering authored Apr 25, 2022
  
  b44aa554
- SST with PS integration. (It was already done.) · 4c201b97
  cjlovering authored Apr 25, 2022
  
  4c201b97
- SST with PS integration. (It was already done.) · e49cf8da
  cjlovering authored Apr 25, 2022
  
  e49cf8da
- Temp. commenting our e2e · 31a019c2
  cjlovering authored Apr 25, 2022
  
  31a019c2
- Add promptsource to setup.py · ec2eb117
  cjlovering authored Apr 25, 2022
  
  ec2eb117
- Removed the default option for an acc task · c93093b6
  cjlovering authored Apr 25, 2022
  
  c93093b6
- Add `higher_is_better` & `aggregation` defaults to `PromptSourceTask` · 4ae2ab37
  jon-tow authored Apr 25, 2022
  
  4ae2ab37
- Add `eos_token` property · 6ec93da2
  jon-tow authored Apr 25, 2022
  
  6ec93da2
25 Apr, 2022 6 commits
- Add multiple tasks · 34f591af
  jon-tow authored Apr 25, 2022
  
  34f591af
- Fix prompt source rank choice accuracy · 2bfa4518
  jon-tow authored Apr 25, 2022
  
  2bfa4518
- Fix task name to template creation · 9f388461
  jon-tow authored Apr 25, 2022
  
  9f388461
- Fix coqa · 9484eecc
  jon-tow authored Apr 25, 2022
  
  9484eecc
- Add PromptSourceTask to the updated tasks · 7d282b5f
  cjlovering authored Apr 25, 2022
  
  7d282b5f
- Initial integration · 88745155
  cjlovering authored Apr 25, 2022
  
  88745155
01 Apr, 2022 1 commit
- Merge pull request #300 from jon-tow/hf-dataset-refactor · 6caa0afd
  Leo Gao authored Apr 01, 2022
```
Refactor `Task` downloading to use `HuggingFace.datasets`
```
  6caa0afd
31 Mar, 2022 3 commits
- Fix docstring · 9434722c
  Jon Tow authored Mar 31, 2022
  
  9434722c
- Adopt `_process_doc` method · 39c8c87f
  Jon Tow authored Mar 31, 2022
  
  39c8c87f
- Add specific `load_dataset` args · d79a4389
  Jon Tow authored Mar 31, 2022
  
  d79a4389
19 Mar, 2022 2 commits
- Whitespace formatting · 4f8b99f6
  Jon Tow authored Mar 19, 2022
  
  4f8b99f6
- Revert `pile` coverage omit · ad164f36
  Jon Tow authored Mar 19, 2022
  
  ad164f36
18 Mar, 2022 5 commits
- Omit task coverage · daa1f436
  Jon Tow authored Mar 18, 2022
  
  daa1f436
- Fix dataset value feature · c2e32987
  Jon Tow authored Mar 18, 2022
  
  c2e32987
- Add custom `image`less `HeadQA` dataset · 897ae76d
  Jon Tow authored Mar 18, 2022
  
  897ae76d
- Fix training docs caching · 13317f8c
  Jon Tow authored Mar 18, 2022
  
  13317f8c
- Refactor `Task` download · 7c9da714
  Jonathan Tow authored Mar 11, 2022
  
  7c9da714
07 Mar, 2022 1 commit
- Bump version to 0.2.0 · 7064d6b9
  Leo Gao authored Mar 06, 2022
  
  7064d6b9
06 Mar, 2022 3 commits
- Merge pull request #296 from jon-tow/gsm8k-fix · fcaea0e3
  Leo Gao authored Mar 06, 2022
```
Add positional args to `gsm8k` `download_file`
```
  fcaea0e3
- Add positional args to `gsm8k` `download_file` · f91a70d6
  Jonathan Tow authored Mar 06, 2022
  
  f91a70d6
- Merge pull request #294 from jon-tow/asdiv-remove-fewshot-description · cec2fe23
  Leo Gao authored Mar 06, 2022
```
Remove `Asdiv` `fewshot_description` method
```
  cec2fe23