GitLab

xnli changes for open models

manual xnli, bypass multiple choice logits for openai

add manual xnli prompt, add multichoice for openai models, and adapt multichoice metric for openai model

adding trasnslate direct

Afri mgsm

Afri mgsm modefied

format task name, clean ups

fix exact match bug and restructure mmlu folder

Afrixnli translate test

add squad to afrimgsm

* Add tinyBenchmarks

* Add acknowledgements

* Add ordering of outputs for data-parallel

* Run pre-commit

* Add few_shot specifications

* Add tinyBenchmarks post-processing

* add conditional import ; fix task names

---------
Co-authored-by: haileyschoelkopf <hailey@eleuther.ai>

add swahili to afrimmlu

Afrimmlu