Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
metabench_permute.yaml 294 Bytes
Newer Older
Kozzy Voudouris's avatar
Add metabench task to LM Evaluation Harness (#2357)  
Kozzy Voudouris committed
1
2
3
4
5
6
7
8
9
10
11
12
13 
group: metabench_permute
task:
  - metabench_arc_permute
  - metabench_hellaswag_permute
  - metabench_mmlu_permute
  - metabench_truthfulqa_permute
  - metabench_winogrande_permute
aggregate_metric_list:
  - metric: acc
    aggregation: mean
    weight_by_size: false
metadata:
  version: 0.0