GitLab

Co-authored-by: Daniel Furman <dryanfurman@gmail.com>

Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>

* add WIP device_map overrides

* update handling outside of accelerate launcher

* change .to(device) log to debug level

* run linter

* benchmark yamls allow minor edits of already registered tasks

* add documentation

* removed print

* Add: kobest config file

* Add: kobest utils

* Add: README

* Update utils.py

* fix incorrect lookback protections

* bump generate_until task versions

* multimedqa

* Update medqa.yaml

* move to benchmarks folder

* add README.md

---------
Co-authored-by: Lintang Sutawika <lintang@sutawika.com>

* Refine scoring logic for multiple_target "exact_match" metric

* skip old tests from master

* skip old tests from master

* delete tests from master