Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
de4ce482
Unverified
Commit
de4ce482
authored
Jul 19, 2025
by
James A. Michaelov
Committed by
GitHub
Jul 19, 2025
Browse files
Add the MultiBLiMP benchmark (#3155)
* add multiblimp * run linter
parent
8c025369
Changes
103
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
60 additions
and
0 deletions
+60
-0
lm_eval/tasks/multiblimp/multiblimp_hbs.yaml
lm_eval/tasks/multiblimp/multiblimp_hbs.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_heb.yaml
lm_eval/tasks/multiblimp/multiblimp_heb.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_hin.yaml
lm_eval/tasks/multiblimp/multiblimp_hin.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_hit.yaml
lm_eval/tasks/multiblimp/multiblimp_hit.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_hsb.yaml
lm_eval/tasks/multiblimp/multiblimp_hsb.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_hun.yaml
lm_eval/tasks/multiblimp/multiblimp_hun.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_hye.yaml
lm_eval/tasks/multiblimp/multiblimp_hye.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_hyw.yaml
lm_eval/tasks/multiblimp/multiblimp_hyw.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_isl.yaml
lm_eval/tasks/multiblimp/multiblimp_isl.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_ita.yaml
lm_eval/tasks/multiblimp/multiblimp_ita.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_kat.yaml
lm_eval/tasks/multiblimp/multiblimp_kat.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_kaz.yaml
lm_eval/tasks/multiblimp/multiblimp_kaz.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_kir.yaml
lm_eval/tasks/multiblimp/multiblimp_kir.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_kmr.yaml
lm_eval/tasks/multiblimp/multiblimp_kmr.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_koi.yaml
lm_eval/tasks/multiblimp/multiblimp_koi.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_kpv.yaml
lm_eval/tasks/multiblimp/multiblimp_kpv.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_krl.yaml
lm_eval/tasks/multiblimp/multiblimp_krl.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_kxh.yaml
lm_eval/tasks/multiblimp/multiblimp_kxh.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_lat.yaml
lm_eval/tasks/multiblimp/multiblimp_lat.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_lav.yaml
lm_eval/tasks/multiblimp/multiblimp_lav.yaml
+3
-0
No files found.
lm_eval/tasks/multiblimp/multiblimp_hbs.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
hbs
task
:
multiblimp_hbs
lm_eval/tasks/multiblimp/multiblimp_heb.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
heb
task
:
multiblimp_heb
lm_eval/tasks/multiblimp/multiblimp_hin.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
hin
task
:
multiblimp_hin
lm_eval/tasks/multiblimp/multiblimp_hit.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
hit
task
:
multiblimp_hit
lm_eval/tasks/multiblimp/multiblimp_hsb.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
hsb
task
:
multiblimp_hsb
lm_eval/tasks/multiblimp/multiblimp_hun.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
hun
task
:
multiblimp_hun
lm_eval/tasks/multiblimp/multiblimp_hye.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
hye
task
:
multiblimp_hye
lm_eval/tasks/multiblimp/multiblimp_hyw.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
hyw
task
:
multiblimp_hyw
lm_eval/tasks/multiblimp/multiblimp_isl.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
isl
task
:
multiblimp_isl
lm_eval/tasks/multiblimp/multiblimp_ita.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
ita
task
:
multiblimp_ita
lm_eval/tasks/multiblimp/multiblimp_kat.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
kat
task
:
multiblimp_kat
lm_eval/tasks/multiblimp/multiblimp_kaz.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
kaz
task
:
multiblimp_kaz
lm_eval/tasks/multiblimp/multiblimp_kir.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
kir
task
:
multiblimp_kir
lm_eval/tasks/multiblimp/multiblimp_kmr.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
kmr
task
:
multiblimp_kmr
lm_eval/tasks/multiblimp/multiblimp_koi.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
koi
task
:
multiblimp_koi
lm_eval/tasks/multiblimp/multiblimp_kpv.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
kpv
task
:
multiblimp_kpv
lm_eval/tasks/multiblimp/multiblimp_krl.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
krl
task
:
multiblimp_krl
lm_eval/tasks/multiblimp/multiblimp_kxh.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
kxh
task
:
multiblimp_kxh
lm_eval/tasks/multiblimp/multiblimp_lat.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
lat
task
:
multiblimp_lat
lm_eval/tasks/multiblimp/multiblimp_lav.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
lav
task
:
multiblimp_lav
Prev
1
2
3
4
5
6
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment