GitLab

* add bypass metric

* fixed `bypass` metric.

* add task attributes if predict_only

* add `predict_only` checks

* add docs

* added `overide_metric`, `override_config` to `Task`

* nits

* nit

* changed --predict_only to generations; nits

* nits

* nits

* change gen_kwargs warning

* add note about `--predict_only` in README.md

* added `predict_only`

* move table to bottom

* nit

* change null aggregation to bypass (conflict)

* bugfix; default `temp=0.0`

* typo

* rewor documentation for explaining local dataset

* fix typo

* Update new_task_guide.md

* benchmark yamls allow minor edits of already registered tasks

* add documentation

* removed print

* change version field formatting in metadata

* mention versioning in new task guide

* add instructions for changelog

* run linters

Co-authored-by: Paul O'Leary McCann <polm@dampfkraft.com>

* doc_to_decontamination_query can use function

* add option for doc_to_decontamination_query to follow doc_to_text

* added documentation for doc_to_decontamination_query

* adjust description

* format

* Additional process for doc_to_choice

* doc_to_choice can also parse a string