Commits · 10492f4b97975c70a3c6e89f04f87aebf5fa9121 · gaoqiong / lm-evaluation-harness

"docs/en_US/CommunitySharings/NasComparision.md" did not exist on "3d43937b1f0756b3762e6a488faa3e8b9aaf324f"

28 Apr, 2022 10 commits
- Add `axg` and `axb` to `super_glue` · 10492f4b
  jon-tow authored Apr 28, 2022
  
  10492f4b
- Merge pull request #15 from bigscience-workshop/update-mlsum-task-name · 9cd70235
  Jonathan Tow authored Apr 28, 2022
```
Fix `mlsum` task names after split update
```
  9cd70235
- Fix `mlsum` task names after split update · 22ea032f
  jon-tow authored Apr 28, 2022
  
  22ea032f
- Merge pull request #14 from bigscience-workshop/fix-gpt-j · efd319b3
  Jonathan Tow authored Apr 28, 2022
```
Add `eot_token` property to gpt-j
```
  efd319b3
- Add `eot_token` property to gpt-j · 32d0658c
  jon-tow authored Apr 28, 2022
  
  32d0658c
- Merge pull request #5 from Shashi456/mlsum · f65e196e
  Jonathan Tow authored Apr 28, 2022
```
Add MLsum Tasks
```
  f65e196e
- Prettify task __init__ · 6d5031ac
  Shashi456 authored Apr 28, 2022
  
  6d5031ac
- Merge branch 'master' of https://github.com/Shashi456/lm-evaluation-harness into mlsum · 666bab5f
  Shashi456 authored Apr 28, 2022
  
  666bab5f
- Update gem_mlsum · 9abb1814
  Shashi456 authored Apr 28, 2022
  
  9abb1814
- Merge pull request #7 from bigscience-workshop/kkawamu1/gem_xsum · 6e56cd0d
  Charles Lovering authored Apr 27, 2022
```
Add GEM/xsum
```
  6e56cd0d
27 Apr, 2022 30 commits
- Merge branch 'master' into mlsum · 7e25c240
  Stella Biderman authored Apr 27, 2022
  
  7e25c240
- Merge pull request #12 from tttyuntian/master · fce17ee1
  Jonathan Tow authored Apr 27, 2022
```
Added null prompt support for T5 & Added BLIMP task template
```
  fce17ee1
- Added blimp task template · 3ee4da8e
  Tian Yun authored Apr 27, 2022
  
  3ee4da8e
- Adding null prompt support in T5 · 7da753f2
  Tian Yun authored Apr 27, 2022
  
  7da753f2
- Merge pull request #11 from bigscience-workshop/update-context-separator · 20820c3c
  Jonathan Tow authored Apr 27, 2022
```
Add `"\n###\n"` example separator
```
  20820c3c
- Add `"\n###\n"` example separator · 597bf747
  jon-tow authored Apr 27, 2022
  
  597bf747
- Merge pull request #10 from tttyuntian/master · 567e24c9
  Charles Lovering authored Apr 27, 2022
```
Fixing stopping criteria & Modifying T0, T5 to BaseLM
```
  567e24c9
- Modified T0, T5 to BaseLM · 3c390461
  Tian Yun authored Apr 27, 2022
  
  3c390461
- Fixing stopping criteria · ecde9d2e
  Tian Yun authored Apr 27, 2022
  
  ecde9d2e
- Merge pull request #9 from jordiclive/webnlg · e51880d8
  Charles Lovering authored Apr 27, 2022
```
webnlg
```
  e51880d8
- webnlg · afcea267
  jordiclive authored Apr 27, 2022
  
  afcea267
- Merge pull request #8 from dirkgr/PytestUpdate · 1cd4ec01
  Stella Biderman authored Apr 27, 2022
```
Pytest update
```
  1cd4ec01
- Merge branch 'master' into PytestUpdate · 38a240ce
  Stella Biderman authored Apr 27, 2022
  
  38a240ce
- Delete main.yml · 33f5572a
  Stella Biderman authored Apr 27, 2022
  
  33f5572a
- Turning on data flags · e9e5305d
  Shashi456 authored Apr 27, 2022
  
  e9e5305d
- Create main.yml · fc781c2f
  Stella Biderman authored Apr 27, 2022
  
  fc781c2f
- Merge pull request #6 from cjlovering/master · 025547c9
  Stella Biderman authored Apr 27, 2022
```
Update with new PR
```
  025547c9
- Merge pull request #4 from cjlovering/cjlovering/output_examples · e5bc4354
  Jonathan Tow authored Apr 27, 2022
```
Enable saving all the examples.
```
  e5bc4354
- Save logging information even if the saving example is false · 408e7802
  cjlovering authored Apr 27, 2022
  
  408e7802
- Add MLsum Tasks · f42f168c
  Shashi456 authored Apr 27, 2022
  
  f42f168c
- Added field for comment; serves as an empty placeholder · 960f3780
  cjlovering authored Apr 27, 2022
  
  960f3780
- Add field to track if it was the original task · 60dc4ad2
  cjlovering authored Apr 27, 2022
  
  60dc4ad2
- Merge pull request #2 from cjlovering/master · 54999199
  Stella Biderman authored Apr 27, 2022
```
Pulling eval harness updates
```
  54999199
- Merge pull request #5 from Shashi456/update-promptsource-version · 18af502b
  Charles Lovering authored Apr 27, 2022
```
Use eval-hackathon branch for installing prompt-source.
```
  18af502b
- Use eval-hackathon branch for installing prompt-source · 60a07f10
  Pawan Sasanka Ammanamanchi authored Apr 27, 2022
  
  60a07f10
- Enable saving all the examples. · 13d53048
  cjlovering authored Apr 27, 2022
```
- saves the doc idx by order in dataset
- saves the full ctx. we can turn this off it is too much memory
- saves the idx of the fewshot examples
- saves the source (test/val) of the fewshot examples
- TODO: We now need to return the pred/target from process_results
- TODO: Make it so override-ed process-resutls don't have to re-impl
this.
```
  13d53048
- Merge pull request #3 from cjlovering/add-rouge · 448bcbfb
  Charles Lovering authored Apr 27, 2022
```
Add `ROUGE` metric to `PromptSourceTask`
```
  448bcbfb
- Merge branch 'master' of https://github.com/cjlovering/ps-eh · d40a7ce0
  jon-tow authored Apr 26, 2022
  
  d40a7ce0
- Add doc getter methods to template · 62a706fc
  jon-tow authored Apr 26, 2022
  
  62a706fc
- Add ROUGE metric to `PromptSourceTask` · ff89667f
  jon-tow authored Apr 26, 2022
  
  ff89667f