Unverified Commit f49b0377 authored by Naiara Perez's avatar Naiara Perez Committed by GitHub
Browse files

add Basque translation of PIQA (piqa_eu) to BasqueBench (#2531)

parent 1170ef9e
......@@ -8,6 +8,7 @@ The new evaluation datasets included in BasqueBench are:
| Task | Category | Homepage |
|:-------------:|:-----:|:-----:|
| MGSM_eu | Math | https://huggingface.co/datasets/HiTZ/MGSM-eu |
| PIQA_eu | Question Answering | https://huggingface.co/datasets/HiTZ/PIQA-eu |
| WNLI_eu | Natural Language Inference | https://huggingface.co/datasets/HiTZ/wnli-eu |
| XCOPA_eu | Commonsense Reasoning | https://huggingface.co/datasets/HiTZ/XCOPA-eu |
......@@ -63,6 +64,7 @@ The following tasks evaluate tasks on BasqueBench dataset using various scoring
- `flores_pt-eu`
- `mgsm_direct_eu`
- `mgsm_native_cot_eu`
- `piqa_eu`
- `qnlieu`
- `wnli_eu`
- `xcopa_eu`
......
......@@ -14,5 +14,6 @@ task:
- xcopa_eu
- mgsm_direct_eu
- mgsm_native_cot_eu
- piqa_eu
metadata:
version: 1.0
task: piqa_eu
dataset_path: HiTZ/PIQA-eu
dataset_name: null
output_type: multiple_choice
training_split: null
validation_split: validation
test_split: null
doc_to_text: "Galdera: {{goal}}\nErantzuna:"
doc_to_target: label
doc_to_choice: "{{[sol1, sol2]}}"
should_decontaminate: true
doc_to_decontamination_query: goal
metric_list:
- metric: acc
aggregation: mean
higher_is_better: true
- metric: acc_norm
aggregation: mean
higher_is_better: true
metadata:
version: 1.0
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment