Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
b69c67c1
Unverified
Commit
b69c67c1
authored
Feb 12, 2024
by
Alessandro Ercolani
Committed by
GitHub
Feb 13, 2024
Browse files
[m_mmul] added multilingual evaluation from alexandrainst/m_mmlu (#1358)
parent
7397b965
Changes
34
Hide whitespace changes
Inline
Side-by-side
Showing
14 changed files
with
56 additions
and
0 deletions
+56
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml
+4
-0
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml
+4
-0
No files found.
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_mr.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
mr
include
:
_default_yaml
task
:
m_mmlu_mr
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nb.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
nb
include
:
_default_yaml
task
:
m_mmlu_nb
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ne.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
ne
include
:
_default_yaml
task
:
m_mmlu_ne
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_nl.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
nl
include
:
_default_yaml
task
:
m_mmlu_nl
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_pt.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
pt
include
:
_default_yaml
task
:
m_mmlu_pt
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ro.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
ro
include
:
_default_yaml
task
:
m_mmlu_ro
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ru.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
ru
include
:
_default_yaml
task
:
m_mmlu_ru
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sr.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
sr
include
:
_default_yaml
task
:
m_mmlu_sr
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_sv.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
sv
include
:
_default_yaml
task
:
m_mmlu_sv
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_ta.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
ta
include
:
_default_yaml
task
:
m_mmlu_ta
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_te.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
te
include
:
_default_yaml
task
:
m_mmlu_te
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_uk.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
uk
include
:
_default_yaml
task
:
m_mmlu_uk
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_vi.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
vi
include
:
_default_yaml
task
:
m_mmlu_vi
lm_eval/tasks/okapi/mmlu_multilingual/m_mmlu_zh.yaml
0 → 100644
View file @
b69c67c1
# Generated by _generate_configs.py
dataset_name
:
zh
include
:
_default_yaml
task
:
m_mmlu_zh
Prev
1
2
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment