Skip to content

GitLab

Commit 439c7c6c authored by lintangsutawika's avatar lintangsutawika
Browse files

arc yaml set as 1 group

parent a5f5d92b
Hide whitespace changes
Inline Side-by-side
Showing with 5 additions and 4 deletions
lm_eval/tasks/arc/arc_challenge.yaml
View file @ 439c7c6c
group:
- arc_yaml
task: arc_challenge_yaml task: arc_challenge_yaml
dataset_path: ai2_arc dataset_path: ai2_arc
dataset_name: ARC-Challenge dataset_name: ARC-Challenge
......
lm_eval/tasks/arc/arc_easy.yaml
View file @ 439c7c6c
group:
- arc_yaml
task: arc_easy_yaml task: arc_easy_yaml
dataset_path: ai2_arc dataset_path: ai2_arc
dataset_name: ARC-Easy dataset_name: ARC-Easy
...@@ -17,7 +19,4 @@ metric_list: ...@@ -17,7 +19,4 @@ metric_list:
higher_is_better: true higher_is_better: true
- metric: acc_mutual_info - metric: acc_mutual_info
aggregation: mean aggregation: mean
higher_is_better: true higher_is_better: true
# - metric: exact_match \ No newline at end of file
# aggregation: mean
# higher_is_better: true
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment