GitLab

Signed-off-by: Wang, Yi <yi.a.wang@intel.com>

Add support for loading GPTQ models via AutoGPTQ

* allow for hf-causal to take dtype arg

* document this change

Add perplexity task on arbitrary JSON data

Add option to dump prompts and completions to a JSON file

Evaluation Against Portion of Benchmark Data

Add results of various models in json and md format

Create output path directory if necessary

* add xcopa dataset

* add xstory_cloze dataset and run pre-commit

* fix xcopa validation and test sets

* add xwinograd dataset

* add pawsx task

* add xnli task

* update task table with recently added tasks

* remove unused metrics from paws-x

* add mgsm task and fix gsm8k

* fix gsm8k until

* update task table

fix adaptive batch crash when there are no new requests

Set PAD token to EOS token

When `limit` is <1, limit represents the percentage of the total number of examples.
If it is >=1,  then it means the number of examples per task (only use this for testing).

This makes comparing the results of different models easier because tasks are ordered in the same way.