GitLab

fixed the reference to the inner model

in case `isinstance(model, lm_eval.base.LM)` line 80 gives error, because `model` is not `str`. A simple inline if statement fixes this.

* add anthropic support

* move requirement

* Fix tokenization issue in BaseLM.loglikelihood

* Add a regression script

* Use entire non-continuation length as context

---------
Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>

* allow for hf-causal to take dtype arg

* document this change

* add xcopa dataset

* add xstory_cloze dataset and run pre-commit

* fix xcopa validation and test sets

* add xwinograd dataset

* add pawsx task

* add xnli task

* update task table with recently added tasks

* remove unused metrics from paws-x

* add mgsm task and fix gsm8k

* fix gsm8k until

* update task table

When `limit` is <1, limit represents the percentage of the total number of examples.
If it is >=1,  then it means the number of examples per task (only use this for testing).