Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
metabench_secondary_permute.yaml 354 Bytes
Newer Older
Kozzy Voudouris's avatar
Add metabench task to LM Evaluation Harness (#2357)  
Kozzy Voudouris committed
1
2
3
4
5
6
7
8
9
10
11
12
13 
group: metabench_secondary_permute
task:
  - metabench_arc_secondary_permute
  - metabench_hellaswag_secondary_permute
  - metabench_mmlu_secondary_permute
  - metabench_truthfulqa_secondary_permute
  - metabench_winogrande_secondary_permute
aggregate_metric_list:
  - metric: acc
    aggregation: mean
    weight_by_size: false
metadata:
  version: 0.0