Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
openllm.yaml 386 Bytes
Newer Older
Baber Abbasi's avatar
Openllm benchmark (#1526)  
Baber Abbasi committed
1
2
3
4 
group: openllm
group_alias: Open LLM Leaderboard
task:
  - task: arc_challenge
Avelina9X's avatar
Update Legacy OpenLLM leaderboard to use "train" split for ARC fewshot (#2802)  
Avelina9X committed
5 
    fewshot_split: train
Baber Abbasi's avatar
Openllm benchmark (#1526)  
Baber Abbasi committed
6
7
8
9
10
11
12
13
14
15
16
17
18 
    num_fewshot: 25
  - task: hellaswag
    fewshot_split: train
    num_fewshot: 10
  - task: truthfulqa
    num_fewshot: 0
  - task: mmlu
    num_fewshot: 5
  - task: winogrande
    fewshot_split: train
    num_fewshot: 5
  - task: gsm8k
    num_fewshot: 5
Avelina9X's avatar
Update Legacy OpenLLM leaderboard to use "train" split for ARC fewshot (#2802)  
Avelina9X committed
19
20 
metadata:
  version: 2