Unverified Commit f49b0377 authored by Naiara Perez's avatar Naiara Perez Committed by GitHub
Browse files

add Basque translation of PIQA (piqa_eu) to BasqueBench (#2531)

parent 1170ef9e
...@@ -8,6 +8,7 @@ The new evaluation datasets included in BasqueBench are: ...@@ -8,6 +8,7 @@ The new evaluation datasets included in BasqueBench are:
| Task | Category | Homepage | | Task | Category | Homepage |
|:-------------:|:-----:|:-----:| |:-------------:|:-----:|:-----:|
| MGSM_eu | Math | https://huggingface.co/datasets/HiTZ/MGSM-eu | | MGSM_eu | Math | https://huggingface.co/datasets/HiTZ/MGSM-eu |
| PIQA_eu | Question Answering | https://huggingface.co/datasets/HiTZ/PIQA-eu |
| WNLI_eu | Natural Language Inference | https://huggingface.co/datasets/HiTZ/wnli-eu | | WNLI_eu | Natural Language Inference | https://huggingface.co/datasets/HiTZ/wnli-eu |
| XCOPA_eu | Commonsense Reasoning | https://huggingface.co/datasets/HiTZ/XCOPA-eu | | XCOPA_eu | Commonsense Reasoning | https://huggingface.co/datasets/HiTZ/XCOPA-eu |
...@@ -63,6 +64,7 @@ The following tasks evaluate tasks on BasqueBench dataset using various scoring ...@@ -63,6 +64,7 @@ The following tasks evaluate tasks on BasqueBench dataset using various scoring
- `flores_pt-eu` - `flores_pt-eu`
- `mgsm_direct_eu` - `mgsm_direct_eu`
- `mgsm_native_cot_eu` - `mgsm_native_cot_eu`
- `piqa_eu`
- `qnlieu` - `qnlieu`
- `wnli_eu` - `wnli_eu`
- `xcopa_eu` - `xcopa_eu`
......
...@@ -14,5 +14,6 @@ task: ...@@ -14,5 +14,6 @@ task:
- xcopa_eu - xcopa_eu
- mgsm_direct_eu - mgsm_direct_eu
- mgsm_native_cot_eu - mgsm_native_cot_eu
- piqa_eu
metadata: metadata:
version: 1.0 version: 1.0
task: piqa_eu
dataset_path: HiTZ/PIQA-eu
dataset_name: null
output_type: multiple_choice
training_split: null
validation_split: validation
test_split: null
doc_to_text: "Galdera: {{goal}}\nErantzuna:"
doc_to_target: label
doc_to_choice: "{{[sol1, sol2]}}"
should_decontaminate: true
doc_to_decontamination_query: goal
metric_list:
- metric: acc
aggregation: mean
higher_is_better: true
- metric: acc_norm
aggregation: mean
higher_is_better: true
metadata:
version: 1.0
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment