Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
_mmlu.yaml 225 Bytes
Newer Older
lintangsutawika's avatar
modfied to add subcategory  
lintangsutawika committed
1
2
3
4
5
6 
group: mmlu
task:
  - mmlu_stem
  - mmlu_other
  - mmlu_social_sciences
  - mmlu_humanities
Lintang Sutawika's avatar
Group agg rework (#1741)  
Lintang Sutawika committed
7
8
9 
aggregate_metric_list:
  - metric: acc
    weight_by_size: True
Baber's avatar
changed mmlu to cloze and added arc_challenge_mmlu  
Baber committed
10
11 
  - metric: acc_norm
    weight_by_size: True
Lintang Sutawika's avatar
Group agg rework (#1741)  
Lintang Sutawika committed
12 
metadata:
Jungwhan Kim's avatar
keep new line for task description (#2116)  
Jungwhan Kim committed
13 
  version: 2