Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
b0040ba0
Unverified
Commit
b0040ba0
authored
Aug 21, 2025
by
James A. Michaelov
Committed by
GitHub
Aug 21, 2025
Browse files
Add BLiMP-NL (#3221)
* add blimp_nl * add template yaml file
parent
1bd96448
Changes
88
Show whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
60 additions
and
0 deletions
+60
-0
lm_eval/tasks/blimp_nl/passive__ditransitive_1.yaml
lm_eval/tasks/blimp_nl/passive__ditransitive_1.yaml
+3
-0
lm_eval/tasks/blimp_nl/passive__ditransitive_2.yaml
lm_eval/tasks/blimp_nl/passive__ditransitive_2.yaml
+3
-0
lm_eval/tasks/blimp_nl/passive__impersonal.yaml
lm_eval/tasks/blimp_nl/passive__impersonal.yaml
+3
-0
lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_plural.yaml
...l/quantifiers__universal_difference_agreement_plural.yaml
+3
-0
lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_singular.yaml
...quantifiers__universal_difference_agreement_singular.yaml
+3
-0
lm_eval/tasks/blimp_nl/r_words__adverbial.yaml
lm_eval/tasks/blimp_nl/r_words__adverbial.yaml
+3
-0
lm_eval/tasks/blimp_nl/r_words__weak_proform.yaml
lm_eval/tasks/blimp_nl/r_words__weak_proform.yaml
+3
-0
lm_eval/tasks/blimp_nl/relativization__island.yaml
lm_eval/tasks/blimp_nl/relativization__island.yaml
+3
-0
lm_eval/tasks/blimp_nl/relativization__pied_piping.yaml
lm_eval/tasks/blimp_nl/relativization__pied_piping.yaml
+3
-0
lm_eval/tasks/blimp_nl/relativization__resumptive_prolepsis.yaml
.../tasks/blimp_nl/relativization__resumptive_prolepsis.yaml
+3
-0
lm_eval/tasks/blimp_nl/topicalization__island.yaml
lm_eval/tasks/blimp_nl/topicalization__island.yaml
+3
-0
lm_eval/tasks/blimp_nl/topicalization__question_similarity_1.yaml
...tasks/blimp_nl/topicalization__question_similarity_1.yaml
+3
-0
lm_eval/tasks/blimp_nl/topicalization__question_similarity_2.yaml
...tasks/blimp_nl/topicalization__question_similarity_2.yaml
+3
-0
lm_eval/tasks/blimp_nl/topicalization__resumptive_prolepsis.yaml
.../tasks/blimp_nl/topicalization__resumptive_prolepsis.yaml
+3
-0
lm_eval/tasks/blimp_nl/verb_second__order_embedded.yaml
lm_eval/tasks/blimp_nl/verb_second__order_embedded.yaml
+3
-0
lm_eval/tasks/blimp_nl/verb_second__order_main.yaml
lm_eval/tasks/blimp_nl/verb_second__order_main.yaml
+3
-0
lm_eval/tasks/blimp_nl/wh_movement__filler_effect_gap.yaml
lm_eval/tasks/blimp_nl/wh_movement__filler_effect_gap.yaml
+3
-0
lm_eval/tasks/blimp_nl/wh_movement__filler_effect_no_gap.yaml
...val/tasks/blimp_nl/wh_movement__filler_effect_no_gap.yaml
+3
-0
lm_eval/tasks/blimp_nl/wh_movement__hierarchy.yaml
lm_eval/tasks/blimp_nl/wh_movement__hierarchy.yaml
+3
-0
lm_eval/tasks/blimp_nl/wh_movement__question_formation.yaml
lm_eval/tasks/blimp_nl/wh_movement__question_formation.yaml
+3
-0
No files found.
lm_eval/tasks/blimp_nl/passive__ditransitive_1.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
passive__ditransitive_1
include
:
_template_yaml
task
:
blimp_nl__passive__ditransitive_1
lm_eval/tasks/blimp_nl/passive__ditransitive_2.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
passive__ditransitive_2
include
:
_template_yaml
task
:
blimp_nl__passive__ditransitive_2
lm_eval/tasks/blimp_nl/passive__impersonal.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
passive__impersonal
include
:
_template_yaml
task
:
blimp_nl__passive__impersonal
lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_plural.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
quantifiers__universal_difference_agreement_plural
include
:
_template_yaml
task
:
blimp_nl__quantifiers__universal_difference_agreement_plural
lm_eval/tasks/blimp_nl/quantifiers__universal_difference_agreement_singular.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
quantifiers__universal_difference_agreement_singular
include
:
_template_yaml
task
:
blimp_nl__quantifiers__universal_difference_agreement_singular
lm_eval/tasks/blimp_nl/r_words__adverbial.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
r_words__adverbial
include
:
_template_yaml
task
:
blimp_nl__r_words__adverbial
lm_eval/tasks/blimp_nl/r_words__weak_proform.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
r_words__weak_proform
include
:
_template_yaml
task
:
blimp_nl__r_words__weak_proform
lm_eval/tasks/blimp_nl/relativization__island.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
relativization__island
include
:
_template_yaml
task
:
blimp_nl__relativization__island
lm_eval/tasks/blimp_nl/relativization__pied_piping.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
relativization__pied_piping
include
:
_template_yaml
task
:
blimp_nl__relativization__pied_piping
lm_eval/tasks/blimp_nl/relativization__resumptive_prolepsis.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
relativization__resumptive_prolepsis
include
:
_template_yaml
task
:
blimp_nl__relativization__resumptive_prolepsis
lm_eval/tasks/blimp_nl/topicalization__island.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
topicalization__island
include
:
_template_yaml
task
:
blimp_nl__topicalization__island
lm_eval/tasks/blimp_nl/topicalization__question_similarity_1.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
topicalization__question_similarity_1
include
:
_template_yaml
task
:
blimp_nl__topicalization__question_similarity_1
lm_eval/tasks/blimp_nl/topicalization__question_similarity_2.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
topicalization__question_similarity_2
include
:
_template_yaml
task
:
blimp_nl__topicalization__question_similarity_2
lm_eval/tasks/blimp_nl/topicalization__resumptive_prolepsis.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
topicalization__resumptive_prolepsis
include
:
_template_yaml
task
:
blimp_nl__topicalization__resumptive_prolepsis
lm_eval/tasks/blimp_nl/verb_second__order_embedded.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
verb_second__order_embedded
include
:
_template_yaml
task
:
blimp_nl__verb_second__order_embedded
lm_eval/tasks/blimp_nl/verb_second__order_main.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
verb_second__order_main
include
:
_template_yaml
task
:
blimp_nl__verb_second__order_main
lm_eval/tasks/blimp_nl/wh_movement__filler_effect_gap.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
wh_movement__filler_effect_gap
include
:
_template_yaml
task
:
blimp_nl__wh_movement__filler_effect_gap
lm_eval/tasks/blimp_nl/wh_movement__filler_effect_no_gap.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
wh_movement__filler_effect_no_gap
include
:
_template_yaml
task
:
blimp_nl__wh_movement__filler_effect_no_gap
lm_eval/tasks/blimp_nl/wh_movement__hierarchy.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
wh_movement__hierarchy
include
:
_template_yaml
task
:
blimp_nl__wh_movement__hierarchy
lm_eval/tasks/blimp_nl/wh_movement__question_formation.yaml
0 → 100644
View file @
b0040ba0
dataset_name
:
wh_movement__question_formation
include
:
_template_yaml
task
:
blimp_nl__wh_movement__question_formation
Prev
1
2
3
4
5
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment