Commits · 2e0b659a9ead8a862cbf321d8294d40e4e62153c · gaoqiong / lm-evaluation-harness

29 Apr, 2022 3 commits
- Fix max generation length · 9af73872
  Tian Yun authored Apr 29, 2022
  
  9af73872
- integrate HuffPost to EvalHarness · 2c165797
  KhalidAlt authored Apr 29, 2022
  
  2c165797
- Fix max generation limit · eda365f6
  jon-tow authored Apr 28, 2022
  
  eda365f6
28 Apr, 2022 20 commits
- Added CrowsPairs for English and French · 2d861a29
  Oskar authored Apr 28, 2022
  
  2d861a29
- Updated T0 and GPT-J · 29bff88d
  Tian Yun authored Apr 28, 2022
  
  29bff88d
- Update stopping criteria for few-shot · b62d1bec
  Tian Yun authored Apr 28, 2022
  
  b62d1bec
- Modified stopping criteria for T5 and GPT-2 · 5e59320b
  Tian Yun authored Apr 28, 2022
  
  5e59320b
- Fixed generation trunction for GPT-2 and T5 · 6c86eb47
  Tian Yun authored Apr 28, 2022
  
  6c86eb47
- Add `axg` and `axb` to `super_glue` · 10492f4b
  jon-tow authored Apr 28, 2022
  
  10492f4b
- e2e_nlg_cleaned: Disable stopping criteria · 4be48911
  Zdenek Kasner authored Apr 28, 2022
  
  4be48911
- Fixed few-shot context generation or datasets without a training split · 49f0699c
  tomlimi authored Apr 28, 2022
  
  49f0699c
- e2e_nlg_cleaned: Update stopping criteria to "\n\n", exclude an accuracy-based prompt · 6588bc74
  Zdenek Kasner authored Apr 28, 2022
  
  6588bc74
- add docstring · ec80b255
  jordiclive authored Apr 28, 2022
  
  ec80b255
- GEM Simplification · 8cd91624
  jordiclive authored Apr 28, 2022
  
  8cd91624
- adding challenge sets · d42d5423
  jordiclive authored Apr 28, 2022
  
  d42d5423
- Remove custom max_length method · 9aba053a
  Zdenek Kasner authored Apr 28, 2022
  
  9aba053a
- Revert changes in base.py · 5a62765b
  Zdenek Kasner authored Apr 28, 2022
  
  5a62765b
- WinoBias support added · 7010e452
  tomlimi authored Apr 28, 2022
  
  7010e452
- Fix `mlsum` task names after split update · 22ea032f
  jon-tow authored Apr 28, 2022
  
  22ea032f
- Add `eot_token` property to gpt-j · 32d0658c
  jon-tow authored Apr 28, 2022
  
  32d0658c
- Fix functions · a80d78db
  Shashi456 authored Apr 28, 2022
  
  a80d78db
- Prettify task __init__ · 6d5031ac
  Shashi456 authored Apr 28, 2022
  
  6d5031ac
- Update gem_mlsum · 9abb1814
  Shashi456 authored Apr 28, 2022
  
  9abb1814
27 Apr, 2022 17 commits
- Added blimp task template · 3ee4da8e
  Tian Yun authored Apr 27, 2022
  
  3ee4da8e
- Adding null prompt support in T5 · 7da753f2
  Tian Yun authored Apr 27, 2022
  
  7da753f2
- Add `"\n###\n"` example separator · 597bf747
  jon-tow authored Apr 27, 2022
  
  597bf747
- Modified T0, T5 to BaseLM · 3c390461
  Tian Yun authored Apr 27, 2022
  
  3c390461
- Fixing stopping criteria · ecde9d2e
  Tian Yun authored Apr 27, 2022
  
  ecde9d2e
- webnlg · afcea267
  jordiclive authored Apr 27, 2022
  
  afcea267
- Turning on data flags · e9e5305d
  Shashi456 authored Apr 27, 2022
  
  e9e5305d
- Save logging information even if the saving example is false · 408e7802
  cjlovering authored Apr 27, 2022
  
  408e7802
- Add MLsum Tasks · f42f168c
  Shashi456 authored Apr 27, 2022
  
  f42f168c
- Added field for comment; serves as an empty placeholder · 960f3780
  cjlovering authored Apr 27, 2022
  
  960f3780
- Add field to track if it was the original task · 60dc4ad2
  cjlovering authored Apr 27, 2022
  
  60dc4ad2
- e2e_nlg_cleaned: Remove custom aggregation() and higher_is_better() methods · 56b2aa94
  Zdenek Kasner authored Apr 27, 2022
  
  56b2aa94
- Align BLEU computation with ROUGE · 0033f1b6
  Zdenek Kasner authored Apr 27, 2022
  
  0033f1b6
- e2e_nlg_cleaned: Add custom code for filtering out invalid examples · 795b7912
  Zdenek Kasner authored Apr 27, 2022
  
  795b7912
- e2e_nlg_cleaned: Exclude QA prompts for now · 3eb3d8b7
  Zdenek Kasner authored Apr 27, 2022
  
  3eb3d8b7
- Add a main part of e2e_nlg_cleaned · f777d840
  Zdenek Kasner authored Apr 27, 2022
  
  f777d840
- Enable saving all the examples. · 13d53048
  cjlovering authored Apr 27, 2022
```
- saves the doc idx by order in dataset
- saves the full ctx. we can turn this off it is too much memory
- saves the idx of the fewshot examples
- saves the source (test/val) of the fewshot examples
- TODO: We now need to return the pred/target from process_results
- TODO: Make it so override-ed process-resutls don't have to re-impl
this.
```
  13d53048