GitLab

* llama-style MMLU CoT

* Refactor MMLU CoT template YAML to simplify 'until' structure

* Add GSM8K task configuration for LLaMA3 with few-shot examples

* Fix missing newline at end of MMLU CoT YAML file

* Add ARC-Challenge task configuration and processing utility

* Add additional MMLU and ARC-Challenge task variants to README

* Update README with notes on arc_challenge_llama dataset preprocessing

* Multilingual MMLU

* Refactor process_docs function calls for clarity and consistency

* Update generation_kwargs in default template to include additional end tokens

* Update filter_list in MMLU Pro configuration to use strict_match

* Update _default_template_yaml

* Update continuation template YAML for MMLU task with new generation and filtering options

* Refactor filter_list structure in continuation template YAML for improved readability

* Add 'take_first' function to filter_list in continuation template YAML

* Update filter_list in continuation template YAML to use 'strict_match' and modify filtering functions

* Add 'do_sample' option to generation_kwargs in MMLU template YAML

* switch arg

* add assistant prefix

* add arc_challenge from llama

* nit

* nit

* nit

* add assistant prefix

* add mmlu_llama

* nit

* nit

* Revert "nit"

This reverts commit 6a97f8356237305e375212b966b30e8de59dd4bc.

* fix regex bug

* add assistant_prefix to vllm

* add `Question:`

* add mmlu_pro

* add fewshot assistant_prefix

* use `assistant_prefill`

* typehints

* nits

* nits

* add to docs

* add readme