Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
e2529a6f
Commit
e2529a6f
authored
Aug 14, 2023
by
lintangsutawika
Browse files
edit arc
parent
0456d543
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
7 additions
and
10 deletions
+7
-10
lm_eval/tasks/arc/README.md
lm_eval/tasks/arc/README.md
+7
-5
lm_eval/tasks/arc/arc_challenge.yaml
lm_eval/tasks/arc/arc_challenge.yaml
+0
-4
lm_eval/tasks/arc/arc_easy.yaml
lm_eval/tasks/arc/arc_easy.yaml
+0
-1
No files found.
lm_eval/tasks/arc/README.md
View file @
e2529a6f
...
...
@@ -29,15 +29,17 @@ Homepage: https://allenai.org/data/arc
}
```
### Subtasks
### Groups and Tasks
#### Groups
*
`ai2_arc`
: Evaluates
`arc_easy`
and
`arc_challenge`
#### Tasks
*
`arc_easy`
*
`arc_challange`
### Groups
*
`ai2_arc`
*
`multiple_choice`
### Checklist
For adding novel benchmarks/datasets to the library:
...
...
lm_eval/tasks/arc/arc_challenge.yaml
View file @
e2529a6f
include
:
arc_easy.yaml
group
:
-
ai2_arc
-
multiple_choice
task
:
arc_challenge
dataset_path
:
ai2_arc
dataset_name
:
ARC-Challenge
lm_eval/tasks/arc/arc_easy.yaml
View file @
e2529a6f
group
:
-
ai2_arc
-
multiple_choice
task
:
arc_easy
dataset_path
:
ai2_arc
dataset_name
:
ARC-Easy
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment