Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
da79e08b
Commit
da79e08b
authored
Jul 02, 2024
by
lintangsutawika
Browse files
add mmmu tasks
parent
566acef5
Changes
7
Hide whitespace changes
Inline
Side-by-side
Showing
7 changed files
with
147 additions
and
0 deletions
+147
-0
lm_eval/tasks/mmmu/art_and_design.yaml
lm_eval/tasks/mmmu/art_and_design.yaml
+19
-0
lm_eval/tasks/mmmu/business.yaml
lm_eval/tasks/mmmu/business.yaml
+23
-0
lm_eval/tasks/mmmu/health_and_medicine.yaml
lm_eval/tasks/mmmu/health_and_medicine.yaml
+23
-0
lm_eval/tasks/mmmu/humanities_and_social_sciences.yaml
lm_eval/tasks/mmmu/humanities_and_social_sciences.yaml
+19
-0
lm_eval/tasks/mmmu/mmmu.yaml
lm_eval/tasks/mmmu/mmmu.yaml
+8
-0
lm_eval/tasks/mmmu/science.yaml
lm_eval/tasks/mmmu/science.yaml
+23
-0
lm_eval/tasks/mmmu/tech_and_engineering.yaml
lm_eval/tasks/mmmu/tech_and_engineering.yaml
+32
-0
No files found.
lm_eval/tasks/mmmu/art_and_design.yaml
0 → 100644
View file @
da79e08b
group
:
mmmu_art_and_design
group_alias
:
Art and Design
task
:
-
task
:
mmmu_art
include
:
_template_yaml
task_alias
:
Art
dataset_name
:
Art
-
task
:
mmmu_art_theory
include
:
_template_yaml
task_alias
:
Art Theory
dataset_name
:
Art_Theory
-
task
:
mmmu_design
include
:
_template_yaml
task_alias
:
Design
dataset_name
:
Design
-
task
:
mmmu_music
include
:
_template_yaml
task_alias
:
Music
dataset_name
:
Music
lm_eval/tasks/mmmu/business.yaml
0 → 100644
View file @
da79e08b
group
:
mmmu_business
group_alias
:
Business
task
:
-
task
:
mmmu_accounting
include
:
_template_yaml
task_alias
:
Accounting
dataset_name
:
Accounting
-
task
:
mmmu_economics
include
:
_template_yaml
task_alias
:
Economics
dataset_name
:
Economics
-
task
:
mmmu_finance
include
:
_template_yaml
task_alias
:
Finance
dataset_name
:
Finance
-
task
:
mmmu_manage
include
:
_template_yaml
task_alias
:
Manage
dataset_name
:
Manage
-
task
:
mmmu_marketing
include
:
_template_yaml
task_alias
:
Marketing
dataset_name
:
Marketing
lm_eval/tasks/mmmu/health_and_medicine.yaml
0 → 100644
View file @
da79e08b
group
:
mmmu_health_and_medicine
group_alias
:
Health and Medicine
task
:
-
task
:
mmmu_basic_medical_science
include
:
_template_yaml
task_alias
:
Basic Medical Science
dataset_name
:
Basic_Medical_Science
-
task
:
mmmu_clinical_medicine
include
:
_template_yaml
task_alias
:
Clinical Medicine
dataset_name
:
Clinical_Medicine
-
task
:
mmmu_diagnostics_and_laboratory_medicine
include
:
_template_yaml
task_alias
:
Diagnostics and Laboratory Medicine
dataset_name
:
Diagnostics_and_Laboratory_Medicine
-
task
:
mmmu_pharmacy
include
:
_template_yaml
task_alias
:
Pharmacy
dataset_name
:
Pharmacy
-
task
:
mmmu_public_health
include
:
_template_yaml
task_alias
:
Public Health
dataset_name
:
Public_Health
lm_eval/tasks/mmmu/humanities_and_social_sciences.yaml
0 → 100644
View file @
da79e08b
group
:
mmmu_humanities_and_social_science
group_alias
:
Humanities and Social Science
task
:
-
task
:
mmmu_history
include
:
_template_yaml
task_alias
:
History
dataset_name
:
History
-
task
:
mmmu_literature
include
:
_template_yaml
task_alias
:
Literature
dataset_name
:
Literature
-
task
:
mmmu_sociology
include
:
_template_yaml
task_alias
:
Sociology
dataset_name
:
Sociology
-
task
:
mmmu_psychology
include
:
_template_yaml
task_alias
:
Psychology
dataset_name
:
Psychology
lm_eval/tasks/mmmu/mmmu.yaml
0 → 100644
View file @
da79e08b
group
:
mmmu
task
:
-
mmmu_art_and_design
-
mmmu_business
-
mmmu_health_and_medicine
-
mmmu_humanities_and_social_science
-
mmmu_science
-
mmmu_tech_and_engineering
lm_eval/tasks/mmmu/science.yaml
0 → 100644
View file @
da79e08b
group
:
mmmu_science
group_alias
:
Science
task
:
-
task
:
mmmu_biology
include
:
_template_yaml
task_alias
:
Biology
dataset_name
:
Biology
-
task
:
mmmu_chemistry
include
:
_template_yaml
task_alias
:
Chemistry
dataset_name
:
Chemistry
-
task
:
mmmu_geography
include
:
_template_yaml
task_alias
:
Geography
dataset_name
:
Geography
-
task
:
mmmu_math
include
:
_template_yaml
task_alias
:
Math
dataset_name
:
Math
-
task
:
mmmu_physics
include
:
_template_yaml
task_alias
:
Physics
dataset_name
:
Physics
lm_eval/tasks/mmmu/tech_and_engineering.yaml
0 → 100644
View file @
da79e08b
group
:
mmmu_tech_and_engineering
group_alias
:
Tech and Engineering
task
:
-
task
:
mmmu_agriculture
include
:
_template_yaml
task_alias
:
Agriculture
dataset_name
:
Agriculture
-
task
:
mmmu_architecture_and_engineering
include
:
_template_yaml
task_alias
:
Architecture and Engineering
dataset_name
:
Architecture_and_Engineering
-
task
:
mmmu_computer_science
include
:
_template_yaml
task_alias
:
Computer Science
dataset_name
:
Computer_Science
-
task
:
mmmu_electronics
include
:
_template_yaml
task_alias
:
Electronics
dataset_name
:
Electronics
-
task
:
mmmu_energy_and_power
include
:
_template_yaml
task_alias
:
Energy and Power
dataset_name
:
Energy_and_Power
-
task
:
mmmu_materials
include
:
_template_yaml
task_alias
:
Materials
dataset_name
:
Materials
-
task
:
mmmu_mechanical_engineering
include
:
_template_yaml
task_alias
:
Mechanical Engineering
dataset_name
:
Mechanical_Engineering
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment