Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
1d3a1601
Commit
1d3a1601
authored
Jul 04, 2023
by
lintangsutawika
Browse files
added benchmark configs
parent
c70383d5
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
38 additions
and
16 deletions
+38
-16
lm_eval/tasks/benchmarks/pythia.yaml
lm_eval/tasks/benchmarks/pythia.yaml
+13
-0
lm_eval/tasks/benchmarks/t0_eval.yaml
lm_eval/tasks/benchmarks/t0_eval.yaml
+25
-0
lm_eval/tasks/benchmarks/t0_eval.yml
lm_eval/tasks/benchmarks/t0_eval.yml
+0
-16
No files found.
lm_eval/tasks/benchmarks/pythia.yaml
0 → 100644
View file @
1d3a1601
group
:
pythia
task
:
-
lambada_openai
-
wikitext
-
piqa
-
sciq
-
wsc
-
winogrande
-
arc_challenge
-
arc_easy
-
logiqa
-
blimp_*
-
hendrycksTest*
lm_eval/tasks/benchmarks/t0_eval.yaml
0 → 100644
View file @
1d3a1601
group
:
-
t0_eval
task
:
-
dataset_path
:
super_glue
# Coreference Resolution
dataset_name
:
wsc.fixed
use_prompt
:
promptsource
-
dataset_path
:
winogrande
# Coreference Resolution
dataset_name
:
winogrande_xl
use_prompt
:
promptsource
-
dataset_path
:
super_glue
# Natural Language Inference
dataset_name
:
cb
use_prompt
:
promptsource
-
dataset_path
:
super_glue
# Natural Language Inference
dataset_name
:
rte
use_prompt
:
promptsource
-
dataset_path
:
anli
# Natural Language Inference
use_prompt
:
promptsource
-
dataset_path
:
super_glue
# Sentence Completion
dataset_name
:
copa
use_prompt
:
promptsource
-
dataset_path
:
hellaswag
# Natural Language Inference
use_prompt
:
promptsource
-
dataset_path
:
super_glue
# Word Sense Disambiguation
dataset_name
:
wic
use_prompt
:
promptsource
lm_eval/tasks/benchmarks/t0_eval.yml
deleted
100644 → 0
View file @
c70383d5
-
dataset_path
:
"
super_glue"
# Coreference Resolution
dataset_name
:
"
wsc.fixed"
-
dataset_path
:
"
winogrande"
# Coreference Resolution
dataset_name
:
"
winogrande_xl"
-
dataset_path
:
"
super_glue"
# Natural Language Inference
dataset_name
:
"
cb"
-
dataset_path
:
"
super_glue"
# Natural Language Inference
dataset_name
:
"
rte"
-
dataset_path
:
"
anli"
# Natural Language Inference
dataset_name
:
null
-
dataset_path
:
"
super_glue"
# Sentence Completion
dataset_name
:
"
copa"
-
dataset_path
:
"
hellaswag"
# Natural Language Inference
dataset_name
:
null
-
dataset_path
:
"
super_glue"
# Word Sense Disambiguation
dataset_name
:
"
wic"
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment