Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
b0040ba0
Unverified
Commit
b0040ba0
authored
Aug 21, 2025
by
James A. Michaelov
Committed by
GitHub
Aug 21, 2025
Browse files
Add BLiMP-NL (#3221)
* add blimp_nl * add template yaml file
parent
1bd96448
Changes
88
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
60 additions
and
0 deletions
+60
-0
lm_eval/tasks/blimp_nl/finite_argument_clause__perception_of.yaml
...tasks/blimp_nl/finite_argument_clause__perception_of.yaml
+3
-0
lm_eval/tasks/blimp_nl/finite_argument_clause__position.yaml
lm_eval/tasks/blimp_nl/finite_argument_clause__position.yaml
+3
-0
lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_1.yaml
...al/tasks/blimp_nl/finite_argument_clause__sluicing_1.yaml
+3
-0
lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_2.yaml
...al/tasks/blimp_nl/finite_argument_clause__sluicing_2.yaml
+3
-0
lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_cluster.yaml
...mp_nl/infinitival_argument_clause__bare_verb_cluster.yaml
+3
-0
lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_1.yaml
...imp_nl/infinitival_argument_clause__bare_verb_type_1.yaml
+3
-0
lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_2.yaml
...imp_nl/infinitival_argument_clause__bare_verb_type_2.yaml
+3
-0
lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_3.yaml
...imp_nl/infinitival_argument_clause__bare_verb_type_3.yaml
+3
-0
lm_eval/tasks/blimp_nl/infinitival_argument_clause__om_te.yaml
...al/tasks/blimp_nl/infinitival_argument_clause__om_te.yaml
+3
-0
lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_1.yaml
...l/infinitival_argument_clause__te_om_te_difference_1.yaml
+3
-0
lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_2.yaml
...l/infinitival_argument_clause__te_om_te_difference_2.yaml
+3
-0
lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_transparant_split.yaml
...nl/infinitival_argument_clause__te_transparant_split.yaml
+3
-0
lm_eval/tasks/blimp_nl/infinitival_argument_clause__verb_type.yaml
...asks/blimp_nl/infinitival_argument_clause__verb_type.yaml
+3
-0
lm_eval/tasks/blimp_nl/nominalization__type_inf_1.yaml
lm_eval/tasks/blimp_nl/nominalization__type_inf_1.yaml
+3
-0
lm_eval/tasks/blimp_nl/nominalization__type_inf_2.yaml
lm_eval/tasks/blimp_nl/nominalization__type_inf_2.yaml
+3
-0
lm_eval/tasks/blimp_nl/parasitic_gaps__scrambling.yaml
lm_eval/tasks/blimp_nl/parasitic_gaps__scrambling.yaml
+3
-0
lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_1.yaml
lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_1.yaml
+3
-0
lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_2.yaml
lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_2.yaml
+3
-0
lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_3.yaml
lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_3.yaml
+3
-0
lm_eval/tasks/blimp_nl/passive__aci.yaml
lm_eval/tasks/blimp_nl/passive__aci.yaml
+3
-0
No files found.
lm_eval/tasks/blimp_nl/finite_argument_clause__perception_of.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
finite_argument_clause__perception_of
include
:
_template_yaml
task
:
blimp_nl__finite_argument_clause__perception_of
lm_eval/tasks/blimp_nl/finite_argument_clause__position.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
finite_argument_clause__position
include
:
_template_yaml
task
:
blimp_nl__finite_argument_clause__position
lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_1.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
finite_argument_clause__sluicing_1
include
:
_template_yaml
task
:
blimp_nl__finite_argument_clause__sluicing_1
lm_eval/tasks/blimp_nl/finite_argument_clause__sluicing_2.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
finite_argument_clause__sluicing_2
include
:
_template_yaml
task
:
blimp_nl__finite_argument_clause__sluicing_2
lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_cluster.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
infinitival_argument_clause__bare_verb_cluster
include
:
_template_yaml
task
:
blimp_nl__infinitival_argument_clause__bare_verb_cluster
lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_1.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
infinitival_argument_clause__bare_verb_type_1
include
:
_template_yaml
task
:
blimp_nl__infinitival_argument_clause__bare_verb_type_1
lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_2.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
infinitival_argument_clause__bare_verb_type_2
include
:
_template_yaml
task
:
blimp_nl__infinitival_argument_clause__bare_verb_type_2
lm_eval/tasks/blimp_nl/infinitival_argument_clause__bare_verb_type_3.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
infinitival_argument_clause__bare_verb_type_3
include
:
_template_yaml
task
:
blimp_nl__infinitival_argument_clause__bare_verb_type_3
lm_eval/tasks/blimp_nl/infinitival_argument_clause__om_te.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
infinitival_argument_clause__om_te
include
:
_template_yaml
task
:
blimp_nl__infinitival_argument_clause__om_te
lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_1.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
infinitival_argument_clause__te_om_te_difference_1
include
:
_template_yaml
task
:
blimp_nl__infinitival_argument_clause__te_om_te_difference_1
lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_om_te_difference_2.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
infinitival_argument_clause__te_om_te_difference_2
include
:
_template_yaml
task
:
blimp_nl__infinitival_argument_clause__te_om_te_difference_2
lm_eval/tasks/blimp_nl/infinitival_argument_clause__te_transparant_split.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
infinitival_argument_clause__te_transparant_split
include
:
_template_yaml
task
:
blimp_nl__infinitival_argument_clause__te_transparant_split
lm_eval/tasks/blimp_nl/infinitival_argument_clause__verb_type.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
infinitival_argument_clause__verb_type
include
:
_template_yaml
task
:
blimp_nl__infinitival_argument_clause__verb_type
lm_eval/tasks/blimp_nl/nominalization__type_inf_1.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
nominalization__type_inf_1
include
:
_template_yaml
task
:
blimp_nl__nominalization__type_inf_1
lm_eval/tasks/blimp_nl/nominalization__type_inf_2.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
nominalization__type_inf_2
include
:
_template_yaml
task
:
blimp_nl__nominalization__type_inf_2
lm_eval/tasks/blimp_nl/parasitic_gaps__scrambling.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
parasitic_gaps__scrambling
include
:
_template_yaml
task
:
blimp_nl__parasitic_gaps__scrambling
lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_1.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
parasitic_gaps__structure_type_1
include
:
_template_yaml
task
:
blimp_nl__parasitic_gaps__structure_type_1
lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_2.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
parasitic_gaps__structure_type_2
include
:
_template_yaml
task
:
blimp_nl__parasitic_gaps__structure_type_2
lm_eval/tasks/blimp_nl/parasitic_gaps__structure_type_3.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
parasitic_gaps__structure_type_3
include
:
_template_yaml
task
:
blimp_nl__parasitic_gaps__structure_type_3
lm_eval/tasks/blimp_nl/passive__aci.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
passive__aci
include
:
_template_yaml
task
:
blimp_nl__passive__aci
Prev
1
2
3
4
5
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment