Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
b2c090cc
Unverified
Commit
b2c090cc
authored
Jan 22, 2025
by
Minho Ryu
Committed by
GitHub
Jan 21, 2025
Browse files
aggregate by group (total and categories) (#2643)
parent
ed9c6fc8
Changes
204
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
0 deletions
+20
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml
...sks/kmmlu/direct/kmmlu_direct_information_technology.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml
...direct/kmmlu_direct_interior_architecture_and_design.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml
...direct/kmmlu_direct_machine_design_and_manufacturing.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml
...tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml
...asks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml
...sks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml
...sks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml
.../direct/kmmlu_direct_political_science_and_sociology.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml
...rect/kmmlu_direct_railway_and_automotive_engineering.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml
...ks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml
+1
-0
lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml
lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml
+1
-0
No files found.
lm_eval/tasks/kmmlu/direct/kmmlu_direct_information_technology.yaml
View file @
b2c090cc
dataset_name
:
Information-Technology
dataset_name
:
Information-Technology
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_information_technology
task
:
kmmlu_direct_information_technology
tag
:
kmmlu_direct_stem_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_interior_architecture_and_design.yaml
View file @
b2c090cc
dataset_name
:
Interior-Architecture-and-Design
dataset_name
:
Interior-Architecture-and-Design
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_interior_architecture_and_design
task
:
kmmlu_direct_interior_architecture_and_design
tag
:
kmmlu_direct_other_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_korean_history.yaml
View file @
b2c090cc
dataset_name
:
Korean-History
dataset_name
:
Korean-History
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_korean_history
task
:
kmmlu_direct_korean_history
tag
:
kmmlu_direct_humss_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_law.yaml
View file @
b2c090cc
dataset_name
:
Law
dataset_name
:
Law
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_law
task
:
kmmlu_direct_law
tag
:
kmmlu_direct_humss_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_machine_design_and_manufacturing.yaml
View file @
b2c090cc
dataset_name
:
Machine-Design-and-Manufacturing
dataset_name
:
Machine-Design-and-Manufacturing
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_machine_design_and_manufacturing
task
:
kmmlu_direct_machine_design_and_manufacturing
tag
:
kmmlu_direct_applied_science_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_management.yaml
View file @
b2c090cc
dataset_name
:
Management
dataset_name
:
Management
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_management
task
:
kmmlu_direct_management
tag
:
kmmlu_direct_humss_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_maritime_engineering.yaml
View file @
b2c090cc
dataset_name
:
Maritime-Engineering
dataset_name
:
Maritime-Engineering
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_maritime_engineering
task
:
kmmlu_direct_maritime_engineering
tag
:
kmmlu_direct_applied_science_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_marketing.yaml
View file @
b2c090cc
dataset_name
:
Marketing
dataset_name
:
Marketing
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_marketing
task
:
kmmlu_direct_marketing
tag
:
kmmlu_direct_other_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_materials_engineering.yaml
View file @
b2c090cc
dataset_name
:
Materials-Engineering
dataset_name
:
Materials-Engineering
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_materials_engineering
task
:
kmmlu_direct_materials_engineering
tag
:
kmmlu_direct_stem_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_math.yaml
View file @
b2c090cc
dataset_name
:
Math
dataset_name
:
Math
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_math
task
:
kmmlu_direct_math
tag
:
kmmlu_direct_stem_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_mechanical_engineering.yaml
View file @
b2c090cc
dataset_name
:
Mechanical-Engineering
dataset_name
:
Mechanical-Engineering
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_mechanical_engineering
task
:
kmmlu_direct_mechanical_engineering
tag
:
kmmlu_direct_stem_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_nondestructive_testing.yaml
View file @
b2c090cc
dataset_name
:
Nondestructive-Testing
dataset_name
:
Nondestructive-Testing
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_nondestructive_testing
task
:
kmmlu_direct_nondestructive_testing
tag
:
kmmlu_direct_applied_science_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_patent.yaml
View file @
b2c090cc
dataset_name
:
Patent
dataset_name
:
Patent
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_patent
task
:
kmmlu_direct_patent
tag
:
kmmlu_direct_other_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_political_science_and_sociology.yaml
View file @
b2c090cc
dataset_name
:
Political-Science-and-Sociology
dataset_name
:
Political-Science-and-Sociology
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_political_science_and_sociology
task
:
kmmlu_direct_political_science_and_sociology
tag
:
kmmlu_direct_humss_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_psychology.yaml
View file @
b2c090cc
dataset_name
:
Psychology
dataset_name
:
Psychology
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_psychology
task
:
kmmlu_direct_psychology
tag
:
kmmlu_direct_humss_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_public_safety.yaml
View file @
b2c090cc
dataset_name
:
Public-Safety
dataset_name
:
Public-Safety
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_public_safety
task
:
kmmlu_direct_public_safety
tag
:
kmmlu_direct_other_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_railway_and_automotive_engineering.yaml
View file @
b2c090cc
dataset_name
:
Railway-and-Automotive-Engineering
dataset_name
:
Railway-and-Automotive-Engineering
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_railway_and_automotive_engineering
task
:
kmmlu_direct_railway_and_automotive_engineering
tag
:
kmmlu_direct_applied_science_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_real_estate.yaml
View file @
b2c090cc
dataset_name
:
Real-Estate
dataset_name
:
Real-Estate
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_real_estate
task
:
kmmlu_direct_real_estate
tag
:
kmmlu_direct_other_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_refrigerating_machinery.yaml
View file @
b2c090cc
dataset_name
:
Refrigerating-Machinery
dataset_name
:
Refrigerating-Machinery
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_refrigerating_machinery
task
:
kmmlu_direct_refrigerating_machinery
tag
:
kmmlu_direct_other_tasks
lm_eval/tasks/kmmlu/direct/kmmlu_direct_social_welfare.yaml
View file @
b2c090cc
dataset_name
:
Social-Welfare
dataset_name
:
Social-Welfare
include
:
_direct_kmmlu_yaml
include
:
_direct_kmmlu_yaml
task
:
kmmlu_direct_social_welfare
task
:
kmmlu_direct_social_welfare
tag
:
kmmlu_direct_humss_tasks
Prev
1
2
3
4
5
6
7
8
9
…
11
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment