Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
de4ce482
Unverified
Commit
de4ce482
authored
Jul 19, 2025
by
James A. Michaelov
Committed by
GitHub
Jul 19, 2025
Browse files
Add the MultiBLiMP benchmark (#3155)
* add multiblimp * run linter
parent
8c025369
Changes
103
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
60 additions
and
0 deletions
+60
-0
lm_eval/tasks/multiblimp/multiblimp_slk.yaml
lm_eval/tasks/multiblimp/multiblimp_slk.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_slv.yaml
lm_eval/tasks/multiblimp/multiblimp_slv.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_sme.yaml
lm_eval/tasks/multiblimp/multiblimp_sme.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_sms.yaml
lm_eval/tasks/multiblimp/multiblimp_sms.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_spa.yaml
lm_eval/tasks/multiblimp/multiblimp_spa.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_sqi.yaml
lm_eval/tasks/multiblimp/multiblimp_sqi.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_swe.yaml
lm_eval/tasks/multiblimp/multiblimp_swe.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_tam.yaml
lm_eval/tasks/multiblimp/multiblimp_tam.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_tpn.yaml
lm_eval/tasks/multiblimp/multiblimp_tpn.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_ttc.yaml
lm_eval/tasks/multiblimp/multiblimp_ttc.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_tur.yaml
lm_eval/tasks/multiblimp/multiblimp_tur.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_uig.yaml
lm_eval/tasks/multiblimp/multiblimp_uig.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_ukr.yaml
lm_eval/tasks/multiblimp/multiblimp_ukr.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_urb.yaml
lm_eval/tasks/multiblimp/multiblimp_urb.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_urd.yaml
lm_eval/tasks/multiblimp/multiblimp_urd.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_uzb.yaml
lm_eval/tasks/multiblimp/multiblimp_uzb.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_vep.yaml
lm_eval/tasks/multiblimp/multiblimp_vep.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_wbp.yaml
lm_eval/tasks/multiblimp/multiblimp_wbp.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_wol.yaml
lm_eval/tasks/multiblimp/multiblimp_wol.yaml
+3
-0
lm_eval/tasks/multiblimp/multiblimp_xcl.yaml
lm_eval/tasks/multiblimp/multiblimp_xcl.yaml
+3
-0
No files found.
lm_eval/tasks/multiblimp/multiblimp_slk.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
slk
task
:
multiblimp_slk
lm_eval/tasks/multiblimp/multiblimp_slv.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
slv
task
:
multiblimp_slv
lm_eval/tasks/multiblimp/multiblimp_sme.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
sme
task
:
multiblimp_sme
lm_eval/tasks/multiblimp/multiblimp_sms.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
sms
task
:
multiblimp_sms
lm_eval/tasks/multiblimp/multiblimp_spa.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
spa
task
:
multiblimp_spa
lm_eval/tasks/multiblimp/multiblimp_sqi.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
sqi
task
:
multiblimp_sqi
lm_eval/tasks/multiblimp/multiblimp_swe.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
swe
task
:
multiblimp_swe
lm_eval/tasks/multiblimp/multiblimp_tam.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
tam
task
:
multiblimp_tam
lm_eval/tasks/multiblimp/multiblimp_tpn.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
tpn
task
:
multiblimp_tpn
lm_eval/tasks/multiblimp/multiblimp_ttc.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
ttc
task
:
multiblimp_ttc
lm_eval/tasks/multiblimp/multiblimp_tur.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
tur
task
:
multiblimp_tur
lm_eval/tasks/multiblimp/multiblimp_uig.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
uig
task
:
multiblimp_uig
lm_eval/tasks/multiblimp/multiblimp_ukr.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
ukr
task
:
multiblimp_ukr
lm_eval/tasks/multiblimp/multiblimp_urb.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
urb
task
:
multiblimp_urb
lm_eval/tasks/multiblimp/multiblimp_urd.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
urd
task
:
multiblimp_urd
lm_eval/tasks/multiblimp/multiblimp_uzb.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
uzb
task
:
multiblimp_uzb
lm_eval/tasks/multiblimp/multiblimp_vep.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
vep
task
:
multiblimp_vep
lm_eval/tasks/multiblimp/multiblimp_wbp.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
wbp
task
:
multiblimp_wbp
lm_eval/tasks/multiblimp/multiblimp_wol.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
wol
task
:
multiblimp_wol
lm_eval/tasks/multiblimp/multiblimp_xcl.yaml
0 → 100644
View file @
de4ce482
include
:
_template_yaml
dataset_name
:
xcl
task
:
multiblimp_xcl
Prev
1
2
3
4
5
6
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment