Add ZhoBLiMP benchmark (#3218)

* add zhoblimp files * correct group name * fix group * add normalized accuracy

Add ZhoBLiMP benchmark (#3218)
* add zhoblimp files * correct group name * fix group * add normalized accuracy
1bd96448 · James A. Michaelov · GitHub · 51d8a192 · 1bd96448 · 1bd96448
Unverified Commit 1bd96448 authored Aug 21, 2025 by James A. Michaelov Committed by GitHub Aug 21, 2025
20 changed files
--- a/lm_eval/tasks/zhoblimp/relative_operator_who.yaml
+++ b/lm_eval/tasks/zhoblimp/relative_operator_who.yaml
--- a/lm_eval/tasks/zhoblimp/relativization_movement_no_gap.yaml
+++ b/lm_eval/tasks/zhoblimp/relativization_movement_no_gap.yaml
--- a/lm_eval/tasks/zhoblimp/relativization_movement_when_where.yaml
+++ b/lm_eval/tasks/zhoblimp/relativization_movement_when_where.yaml
--- a/lm_eval/tasks/zhoblimp/renhe_no_episodic_sentences.yaml
+++ b/lm_eval/tasks/zhoblimp/renhe_no_episodic_sentences.yaml
--- a/lm_eval/tasks/zhoblimp/renhe_no_superordinate_negation.yaml
+++ b/lm_eval/tasks/zhoblimp/renhe_no_superordinate_negation.yaml
--- a/lm_eval/tasks/zhoblimp/renhe_non_factive_verb.yaml
+++ b/lm_eval/tasks/zhoblimp/renhe_non_factive_verb.yaml
--- a/lm_eval/tasks/zhoblimp/right_yijing_a.yaml
+++ b/lm_eval/tasks/zhoblimp/right_yijing_a.yaml
--- a/lm_eval/tasks/zhoblimp/right_yijing_b.yaml
+++ b/lm_eval/tasks/zhoblimp/right_yijing_b.yaml
--- a/lm_eval/tasks/zhoblimp/singular_PN_but_plural_pron.yaml
+++ b/lm_eval/tasks/zhoblimp/singular_PN_but_plural_pron.yaml
--- a/lm_eval/tasks/zhoblimp/superlative_quantifiers_1.yaml
+++ b/lm_eval/tasks/zhoblimp/superlative_quantifiers_1.yaml
--- a/lm_eval/tasks/zhoblimp/superlative_quantifiers_2.yaml
+++ b/lm_eval/tasks/zhoblimp/superlative_quantifiers_2.yaml
--- a/lm_eval/tasks/zhoblimp/topicalization_OSV.yaml
+++ b/lm_eval/tasks/zhoblimp/topicalization_OSV.yaml
--- a/lm_eval/tasks/zhoblimp/topicalization_OSV_mei.yaml
+++ b/lm_eval/tasks/zhoblimp/topicalization_OSV_mei.yaml
--- a/lm_eval/tasks/zhoblimp/topicalization_SOV.yaml
+++ b/lm_eval/tasks/zhoblimp/topicalization_SOV.yaml
--- a/lm_eval/tasks/zhoblimp/topicalization_SOV_mei.yaml
+++ b/lm_eval/tasks/zhoblimp/topicalization_SOV_mei.yaml
--- a/lm_eval/tasks/zhoblimp/verb_negation_particle.yaml
+++ b/lm_eval/tasks/zhoblimp/verb_negation_particle.yaml
--- a/lm_eval/tasks/zhoblimp/verb_phrase_left_adverbial.yaml
+++ b/lm_eval/tasks/zhoblimp/verb_phrase_left_adverbial.yaml
--- a/lm_eval/tasks/zhoblimp/verb_phrase_left_negation.yaml
+++ b/lm_eval/tasks/zhoblimp/verb_phrase_left_negation.yaml
--- a/lm_eval/tasks/zhoblimp/ya_insertion.yaml
+++ b/lm_eval/tasks/zhoblimp/ya_insertion.yaml
--- a/lm_eval/tasks/zhoblimp/you_quantifier_adj.yaml
+++ b/lm_eval/tasks/zhoblimp/you_quantifier_adj.yaml