Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
de4ce482
Unverified
Commit
de4ce482
authored
Jul 19, 2025
by
James A. Michaelov
Committed by
GitHub
Jul 19, 2025
Browse files
Add the MultiBLiMP benchmark (#3155)
* add multiblimp * run linter
parent
8c025369
Changes
103
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
60 additions
and
0 deletions
+60
-0
lm_eval/tasks/multiblimp/multiblimp_dan.yaml
lm_eval/tasks/multiblimp/multiblimp_dan.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_deu.yaml
lm_eval/tasks/multiblimp/multiblimp_deu.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_egy.yaml
lm_eval/tasks/multiblimp/multiblimp_egy.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_ell.yaml
lm_eval/tasks/multiblimp/multiblimp_ell.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_eng.yaml
lm_eval/tasks/multiblimp/multiblimp_eng.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_est.yaml
lm_eval/tasks/multiblimp/multiblimp_est.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_eus.yaml
lm_eval/tasks/multiblimp/multiblimp_eus.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_fao.yaml
lm_eval/tasks/multiblimp/multiblimp_fao.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_fas.yaml
lm_eval/tasks/multiblimp/multiblimp_fas.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_fin.yaml
lm_eval/tasks/multiblimp/multiblimp_fin.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_fra.yaml
lm_eval/tasks/multiblimp/multiblimp_fra.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_frm.yaml
lm_eval/tasks/multiblimp/multiblimp_frm.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_fro.yaml
lm_eval/tasks/multiblimp/multiblimp_fro.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_gla.yaml
lm_eval/tasks/multiblimp/multiblimp_gla.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_gle.yaml
lm_eval/tasks/multiblimp/multiblimp_gle.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_glg.yaml
lm_eval/tasks/multiblimp/multiblimp_glg.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_got.yaml
lm_eval/tasks/multiblimp/multiblimp_got.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_grc.yaml
lm_eval/tasks/multiblimp/multiblimp_grc.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_guj.yaml
lm_eval/tasks/multiblimp/multiblimp_guj.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_hbo.yaml
lm_eval/tasks/multiblimp/multiblimp_hbo.yaml
+3
-0
No files found.
lm_eval/tasks/multiblimp/multiblimp_dan.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
dan
task
:
multiblimp_dan
lm_eval/tasks/multiblimp/multiblimp_deu.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
deu
task
:
multiblimp_deu
lm_eval/tasks/multiblimp/multiblimp_egy.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
egy
task
:
multiblimp_egy
lm_eval/tasks/multiblimp/multiblimp_ell.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
ell
task
:
multiblimp_ell
lm_eval/tasks/multiblimp/multiblimp_eng.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
eng
task
:
multiblimp_eng
lm_eval/tasks/multiblimp/multiblimp_est.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
est
task
:
multiblimp_est
lm_eval/tasks/multiblimp/multiblimp_eus.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
eus
task
:
multiblimp_eus
lm_eval/tasks/multiblimp/multiblimp_fao.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
fao
task
:
multiblimp_fao
lm_eval/tasks/multiblimp/multiblimp_fas.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
fas
task
:
multiblimp_fas
lm_eval/tasks/multiblimp/multiblimp_fin.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
fin
task
:
multiblimp_fin
lm_eval/tasks/multiblimp/multiblimp_fra.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
fra
task
:
multiblimp_fra
lm_eval/tasks/multiblimp/multiblimp_frm.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
frm
task
:
multiblimp_frm
lm_eval/tasks/multiblimp/multiblimp_fro.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
fro
task
:
multiblimp_fro
lm_eval/tasks/multiblimp/multiblimp_gla.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
gla
task
:
multiblimp_gla
lm_eval/tasks/multiblimp/multiblimp_gle.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
gle
task
:
multiblimp_gle
lm_eval/tasks/multiblimp/multiblimp_glg.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
glg
task
:
multiblimp_glg
lm_eval/tasks/multiblimp/multiblimp_got.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
got
task
:
multiblimp_got
lm_eval/tasks/multiblimp/multiblimp_grc.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
grc
task
:
multiblimp_grc
lm_eval/tasks/multiblimp/multiblimp_guj.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
guj
task
:
multiblimp_guj
lm_eval/tasks/multiblimp/multiblimp_hbo.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
hbo
task
:
multiblimp_hbo
Prev
1
2
3
4
5
6
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment