Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
de4ce482
Unverified
Commit
de4ce482
authored
Jul 19, 2025
by
James A. Michaelov
Committed by
GitHub
Jul 19, 2025
Browse files
Add the MultiBLiMP benchmark (#3155)
* add multiblimp * run linter
parent
8c025369
Changes
103
Show whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
60 additions
and
0 deletions
+60
-0
lm_eval/tasks/multiblimp/multiblimp_lij.yaml
lm_eval/tasks/multiblimp/multiblimp_lij.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_lit.yaml
lm_eval/tasks/multiblimp/multiblimp_lit.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_mar.yaml
lm_eval/tasks/multiblimp/multiblimp_mar.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_mdf.yaml
lm_eval/tasks/multiblimp/multiblimp_mdf.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_mkd.yaml
lm_eval/tasks/multiblimp/multiblimp_mkd.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_myv.yaml
lm_eval/tasks/multiblimp/multiblimp_myv.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_nds.yaml
lm_eval/tasks/multiblimp/multiblimp_nds.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_nhi.yaml
lm_eval/tasks/multiblimp/multiblimp_nhi.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_nld.yaml
lm_eval/tasks/multiblimp/multiblimp_nld.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_olo.yaml
lm_eval/tasks/multiblimp/multiblimp_olo.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_orv.yaml
lm_eval/tasks/multiblimp/multiblimp_orv.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_ota.yaml
lm_eval/tasks/multiblimp/multiblimp_ota.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_pcm.yaml
lm_eval/tasks/multiblimp/multiblimp_pcm.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_pol.yaml
lm_eval/tasks/multiblimp/multiblimp_pol.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_por.yaml
lm_eval/tasks/multiblimp/multiblimp_por.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_quc.yaml
lm_eval/tasks/multiblimp/multiblimp_quc.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_ron.yaml
lm_eval/tasks/multiblimp/multiblimp_ron.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_rus.yaml
lm_eval/tasks/multiblimp/multiblimp_rus.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_sah.yaml
lm_eval/tasks/multiblimp/multiblimp_sah.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_san.yaml
lm_eval/tasks/multiblimp/multiblimp_san.yaml
+3
-0
No files found.
lm_eval/tasks/multiblimp/multiblimp_lij.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
lij
task
:
multiblimp_lij
lm_eval/tasks/multiblimp/multiblimp_lit.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
lit
task
:
multiblimp_lit
lm_eval/tasks/multiblimp/multiblimp_mar.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
mar
task
:
multiblimp_mar
lm_eval/tasks/multiblimp/multiblimp_mdf.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
mdf
task
:
multiblimp_mdf
lm_eval/tasks/multiblimp/multiblimp_mkd.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
mkd
task
:
multiblimp_mkd
lm_eval/tasks/multiblimp/multiblimp_myv.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
myv
task
:
multiblimp_myv
lm_eval/tasks/multiblimp/multiblimp_nds.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
nds
task
:
multiblimp_nds
lm_eval/tasks/multiblimp/multiblimp_nhi.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
nhi
task
:
multiblimp_nhi
lm_eval/tasks/multiblimp/multiblimp_nld.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
nld
task
:
multiblimp_nld
lm_eval/tasks/multiblimp/multiblimp_olo.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
olo
task
:
multiblimp_olo
lm_eval/tasks/multiblimp/multiblimp_orv.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
orv
task
:
multiblimp_orv
lm_eval/tasks/multiblimp/multiblimp_ota.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
ota
task
:
multiblimp_ota
lm_eval/tasks/multiblimp/multiblimp_pcm.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
pcm
task
:
multiblimp_pcm
lm_eval/tasks/multiblimp/multiblimp_pol.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
pol
task
:
multiblimp_pol
lm_eval/tasks/multiblimp/multiblimp_por.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
por
task
:
multiblimp_por
lm_eval/tasks/multiblimp/multiblimp_quc.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
quc
task
:
multiblimp_quc
lm_eval/tasks/multiblimp/multiblimp_ron.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
ron
task
:
multiblimp_ron
lm_eval/tasks/multiblimp/multiblimp_rus.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
rus
task
:
multiblimp_rus
lm_eval/tasks/multiblimp/multiblimp_sah.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
sah
task
:
multiblimp_sah
lm_eval/tasks/multiblimp/multiblimp_san.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
san
task
:
multiblimp_san
Prev
1
2
3
4
5
6
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment