Unverified Commit 5a15058e authored by Anna Fontana's avatar Anna Fontana Committed by GitHub
Browse files

Update README.md (#3070)

Wrong task name: mmlu_generation doesn't non exist -> mmlu_generative is the correct one
parent 9fbe48c2
......@@ -36,11 +36,11 @@ Note: The `Flan` variants are derived from [here](https://github.com/jasonwei20/
* `mmlu`: `Original multiple-choice MMLU benchmark`
* `mmlu_continuation`: `MMLU but with continuation prompts`
* `mmlu_generation`: `MMLU generation`
* `mmlu_generative`: `MMLU generation`
MMLU is the original benchmark as implemented by Hendrycks et al. with the choices in context and the answer letters (e.g `A`, `B`, `C`, `D`) in the continuation.
`mmlu_continuation` is a cloze-style variant without the choices in context and the full answer choice in the continuation.
`mmlu_generation` is a generation variant, similar to the original but the LLM is asked to generate the correct answer letter.
`mmlu_generative` is a generation variant, similar to the original but the LLM is asked to generate the correct answer letter.
#### Subgroups
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment