Commits · 8c83a8219d5faf7cbac5ff34a8ecbbc2eace21eb · gaoqiong / lm-evaluation-harness

25 Apr, 2022 2 commits
- Add SPLIT · 8c83a821
  ken authored Apr 25, 2022
  
  8c83a821
- Add GEM/xsum · 6a2b94b2
  ken authored Apr 25, 2022
  
  6a2b94b2
27 Apr, 2022 9 commits
- Merge pull request #2 from cjlovering/master · 54999199
  Stella Biderman authored Apr 27, 2022
```
Pulling eval harness updates
```
  54999199
- Merge pull request #5 from Shashi456/update-promptsource-version · 18af502b
  Charles Lovering authored Apr 27, 2022
```
Use eval-hackathon branch for installing prompt-source.
```
  18af502b
- Use eval-hackathon branch for installing prompt-source · 60a07f10
  Pawan Sasanka Ammanamanchi authored Apr 27, 2022
  
  60a07f10
- Merge pull request #3 from cjlovering/add-rouge · 448bcbfb
  Charles Lovering authored Apr 27, 2022
```
Add `ROUGE` metric to `PromptSourceTask`
```
  448bcbfb
- Merge branch 'master' of https://github.com/cjlovering/ps-eh · d40a7ce0
  jon-tow authored Apr 26, 2022
  
  d40a7ce0
- Add doc getter methods to template · 62a706fc
  jon-tow authored Apr 26, 2022
  
  62a706fc
- Add ROUGE metric to `PromptSourceTask` · ff89667f
  jon-tow authored Apr 26, 2022
  
  ff89667f
- Merge pull request #2 from cjlovering/cjlovering/request_args · 5b0d95a0
  Jonathan Tow authored Apr 26, 2022
```
Updated the requests so that its easier to understand.
```
  5b0d95a0
- Added GPT-J · a3a9a7c2
  Tian Yun authored Apr 26, 2022
  
  a3a9a7c2
26 Apr, 2022 29 commits
- Updated the requests so that its easier to understand. · 21d897db
  cjlovering authored Apr 26, 2022
  
  21d897db
- Updated doc: If the answer choices is empty, then it is generation; else... · 4f85bcf9
  cjlovering authored Apr 26, 2022
```
Updated doc: If the answer choices is empty, then it is generation; else ranked choice. This will be the canonical approach when using PS.
```
  4f85bcf9
- Merge branch 'master' of https://github.com/cjlovering/ps-eh · af3cccc8
  Tian Yun authored Apr 26, 2022
  
  af3cccc8
- Added t0, t5, mt5 · 716c87d6
  Tian Yun authored Apr 26, 2022
  
  716c87d6
- Add multi-reference ROUGE support · 94218002
  jon-tow authored Apr 26, 2022
  
  94218002
- Merge pull request #1 from cjlovering/cjlovering/gen_max_len · 4941a8bb
  Jonathan Tow authored Apr 26, 2022
```
Add optional max length to generation.
```
  4941a8bb
- A dependency required this but it was not installed by default · 9384ec91
  cjlovering authored Apr 26, 2022
  
  9384ec91
- Merge branch 'master' of github.com:cjlovering/ps-eh · 02ec7889
  cjlovering authored Apr 26, 2022
  
  02ec7889
- A dependency required this but it was not installed by default · b8c203cd
  cjlovering authored Apr 26, 2022
  
  b8c203cd
- Merge branch 'master' of https://github.com/cjlovering/ps-eh · ca414b0e
  jon-tow authored Apr 26, 2022
  
  ca414b0e
- Add `PromptSourceTask` template · 941fe268
  jon-tow authored Apr 26, 2022
  
  941fe268
- Merge branch 'master' of github.com:cjlovering/ps-eh · 842139ae
  cjlovering authored Apr 26, 2022
  
  842139ae
- Changing stopping criteria for coqa · 23a6ee7f
  cjlovering authored Apr 26, 2022
  
  23a6ee7f
- Modified stopping criteria for gpt2 · 61b57292
  Tian Yun authored Apr 26, 2022
  
  61b57292
- Merge with master branch · 0b8cb8b9
  Tian Yun authored Apr 26, 2022
  
  0b8cb8b9
- Added stoppping criteria for generation · 96ea7ddc
  Tian Yun authored Apr 26, 2022
  
  96ea7ddc
- Rename task specific to · b2838b8d
  cjlovering authored Apr 26, 2022
  
  b2838b8d
- Minor updates to documentation. · 1dcca55c
  cjlovering authored Apr 26, 2022
  
  1dcca55c
- Added default behavior for bleu to the promtsourcetask class · d4c00093
  cjlovering authored Apr 26, 2022
  
  d4c00093
- Rename task specific to · f39c27c2
  cjlovering authored Apr 26, 2022
  
  f39c27c2
- Force no caching while testing · c27e29e1
  jon-tow authored Apr 26, 2022
  
  c27e29e1
- Merge branch 'master' of github.com:cjlovering/ps-eh · f33717c2
  cjlovering authored Apr 25, 2022
  
  f33717c2
- Update the token encode · c26f1d4c
  cjlovering authored Apr 25, 2022
  
  c26f1d4c
- Fix `wsc` subset name · b988137d
  jon-tow authored Apr 25, 2022
  
  b988137d
- Print table across task and prompt · 422380cc
  cjlovering authored Apr 25, 2022
  
  422380cc
- Merge branch 'master' of github.com:cjlovering/ps-eh · 053afe05
  cjlovering authored Apr 25, 2022
  
  053afe05
- Turned off generation tasks for now. Changed process to look at the metrics.... · 0cdcc989
  jon-tow authored Apr 25, 2022
```
Turned off generation tasks for now. Changed process to look at the metrics. Only accuracy implemented.
```
  0cdcc989
- Add `HANS` · 0e578306
  jon-tow authored Apr 25, 2022
  
  0e578306
- QQP with PS integration. · b1a3c6e3
  cjlovering authored Apr 25, 2022
  
  b1a3c6e3