GitLab

Set PAD token to EOS token

This makes comparing the results of different models easier because tasks are ordered in the same way.

Fix bugs introduced in #394 #406 and max length bug

Fix automatic batching for greedy_until
Fix stop sequences bug for greedy_until
Fix max length bug when model max length is not specified in tokenizer

single GPU automatic batching logic

Updated handling for device in lm_eval/models/gpt2.py

* Support bigbench-hard json tasks using multiple_choice_grade

* Add support for greedy decoding in bigbench tasks

* move bigbench_resources to datasets

* rectify changes to rf.greedy_until w upstream

* make path to resource import reflect new location

---------
Co-authored-by: haileyschoelkopf <hailey.schoelkopf@yale.edu>

Document task name wildcard support in README

Hotfix: patch issues with the `huggingface.py` model classes

* Add 8 bit support

* added trust remote code

* Update main.py

* Update huggingface.py

* Update huggingface.py

* Fix bugs

* Fix PR #422 issues

* fix(style): remove empty newline

* fix(stlye,args): run `pre-commit` and re-order args for backward-compat

* fix(style): pre-commit the `README.md`

fix: add seq2seq peft

Update setup.py dependencies

feat: evaluation using peft models with CLM