Skip to content

GitLab

"lm_eval/tasks/mrl/gen/afb_arab.yaml" did not exist on "fa9330f91563f257d5e98747f7fb8b832e1221be"
Switch branch/tag
Find file Normal viewHistoryPermalink
humaneval_infilling.yaml 297 Bytes
Newer Older
its-alpesh's avatar
Add humaneval_infilling task (#3299)  
its-alpesh committed
1
2
3
4
5
6
7
8
9
10
11
12 
group: humaneval_infilling
task:
  - humaneval_multi_line_infilling
  - humaneval_single_line_infilling
  - humaneval_random_span_infilling
  - humaneval_random_span_infilling_light
aggregate_metric_list:
  - metric: pass@1
    aggregation: mean
    weight_by_size: false
metadata:
  version: 1.0