Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
543617fe
Unverified
Commit
543617fe
authored
Sep 05, 2024
by
Hailey Schoelkopf
Committed by
GitHub
Sep 05, 2024
Browse files
Bump version to v0.4.4 ; Fixes to TMMLUplus (#2280)
parent
7a1614eb
Changes
76
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
40 additions
and
60 deletions
+40
-60
lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml
lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml
lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml
lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml
...uplus/default/tmmluplus_education_(profession_level).yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml
...s/tmmluplus/default/tmmluplus_educational_psychology.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml
...l/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml
...al/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml
...tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml
lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml
...mmluplus/default/tmmluplus_general_principles_of_law.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml
...asks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml
...val/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml
.../tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml
...asks/tmmluplus/default/tmmluplus_introduction_to_law.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml
...val/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml
...l/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml
...asks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml
...l/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml
...asks/tmmluplus/default/tmmluplus_junior_science_exam.yaml
+2
-3
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml
...ks/tmmluplus/default/tmmluplus_junior_social_studies.yaml
+2
-3
No files found.
lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml
View file @
543617fe
"
dataset_name"
:
"
dentistry"
"
dataset_name"
:
"
dentistry"
"
description"
:
"
以下為牙醫學的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為牙醫學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
tag"
:
"
tmmluplus_other_tasks"
"
group_alias"
:
"
other"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_dentistry"
"
task"
:
"
tmmluplus_dentistry"
"
task_alias"
:
"
dentistry"
"
task_alias"
:
"
dentistry"
lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml
View file @
543617fe
"
dataset_name"
:
"
economics"
"
dataset_name"
:
"
economics"
"
description"
:
"
以下為經濟學的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為經濟學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
tag"
:
"
tmmluplus_social_sciences_tasks"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_economics"
"
task"
:
"
tmmluplus_economics"
"
task_alias"
:
"
economics"
"
task_alias"
:
"
economics"
lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml
View file @
543617fe
"
dataset_name"
:
"
education"
"
dataset_name"
:
"
education"
"
description"
:
"
以下為教育常識的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為教育常識的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
tag"
:
"
tmmluplus_social_sciences_tasks"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_education"
"
task"
:
"
tmmluplus_education"
"
task_alias"
:
"
education"
"
task_alias"
:
"
education"
lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml
View file @
543617fe
"
dataset_name"
:
"
education_(profession_level)"
"
dataset_name"
:
"
education_(profession_level)"
"
description"
:
"
以下為教育專業的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為教育專業的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
tag"
:
"
tmmluplus_social_sciences_tasks"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_education_(profession_level)"
"
task"
:
"
tmmluplus_education_(profession_level)"
"
task_alias"
:
"
education
(profession
level)"
"
task_alias"
:
"
education
(profession
level)"
lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml
View file @
543617fe
"
dataset_name"
:
"
educational_psychology"
"
dataset_name"
:
"
educational_psychology"
"
description"
:
"
以下為教育心理的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為教育心理的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
tag"
:
"
tmmluplus_social_sciences_tasks"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_educational_psychology"
"
task"
:
"
tmmluplus_educational_psychology"
"
task_alias"
:
"
educational
psychology"
"
task_alias"
:
"
educational
psychology"
lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml
View file @
543617fe
"
dataset_name"
:
"
engineering_math"
"
dataset_name"
:
"
engineering_math"
"
description"
:
"
以下為工程數學的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為工程數學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_STEM"
"
tag"
:
"
tmmluplus_STEM_tasks"
"
group_alias"
:
"
STEM"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_engineering_math"
"
task"
:
"
tmmluplus_engineering_math"
"
task_alias"
:
"
engineering
math"
"
task_alias"
:
"
engineering
math"
lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml
View file @
543617fe
"
dataset_name"
:
"
finance_banking"
"
dataset_name"
:
"
finance_banking"
"
description"
:
"
以下為金融與法規的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為金融與法規的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
tag"
:
"
tmmluplus_other_tasks"
"
group_alias"
:
"
other"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_finance_banking"
"
task"
:
"
tmmluplus_finance_banking"
"
task_alias"
:
"
finance
banking"
"
task_alias"
:
"
finance
banking"
lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml
View file @
543617fe
"
dataset_name"
:
"
financial_analysis"
"
dataset_name"
:
"
financial_analysis"
"
description"
:
"
以下為財務分析的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為財務分析的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
tag"
:
"
tmmluplus_other_tasks"
"
group_alias"
:
"
other"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_financial_analysis"
"
task"
:
"
tmmluplus_financial_analysis"
"
task_alias"
:
"
financial
analysis"
"
task_alias"
:
"
financial
analysis"
lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml
View file @
543617fe
"
dataset_name"
:
"
fire_science"
"
dataset_name"
:
"
fire_science"
"
description"
:
"
以下為火災學的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為火災學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
tag"
:
"
tmmluplus_other_tasks"
"
group_alias"
:
"
other"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_fire_science"
"
task"
:
"
tmmluplus_fire_science"
"
task_alias"
:
"
fire
science"
"
task_alias"
:
"
fire
science"
lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml
View file @
543617fe
"
dataset_name"
:
"
general_principles_of_law"
"
dataset_name"
:
"
general_principles_of_law"
"
description"
:
"
以下為法學大意的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為法學大意的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_humanities"
"
tag"
:
"
tmmluplus_humanities_tasks"
"
group_alias"
:
"
humanities"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_general_principles_of_law"
"
task"
:
"
tmmluplus_general_principles_of_law"
"
task_alias"
:
"
general
principles
of
law"
"
task_alias"
:
"
general
principles
of
law"
lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml
View file @
543617fe
"
dataset_name"
:
"
geography_of_taiwan"
"
dataset_name"
:
"
geography_of_taiwan"
"
description"
:
"
以下為台灣地理的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為台灣地理的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
tag"
:
"
tmmluplus_social_sciences_tasks"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_geography_of_taiwan"
"
task"
:
"
tmmluplus_geography_of_taiwan"
"
task_alias"
:
"
geography
of
taiwan"
"
task_alias"
:
"
geography
of
taiwan"
lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml
View file @
543617fe
"
dataset_name"
:
"
human_behavior"
"
dataset_name"
:
"
human_behavior"
"
description"
:
"
以下為人類行為與社會的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為人類行為與社會的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
tag"
:
"
tmmluplus_social_sciences_tasks"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_human_behavior"
"
task"
:
"
tmmluplus_human_behavior"
"
task_alias"
:
"
human
behavior"
"
task_alias"
:
"
human
behavior"
lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml
View file @
543617fe
"
dataset_name"
:
"
insurance_studies"
"
dataset_name"
:
"
insurance_studies"
"
description"
:
"
以下為保險學的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為保險學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
tag"
:
"
tmmluplus_other_tasks"
"
group_alias"
:
"
other"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_insurance_studies"
"
task"
:
"
tmmluplus_insurance_studies"
"
task_alias"
:
"
insurance
studies"
"
task_alias"
:
"
insurance
studies"
lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml
View file @
543617fe
"
dataset_name"
:
"
introduction_to_law"
"
dataset_name"
:
"
introduction_to_law"
"
description"
:
"
以下為法律概論的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為法律概論的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_humanities"
"
tag"
:
"
tmmluplus_humanities_tasks"
"
group_alias"
:
"
humanities"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_introduction_to_law"
"
task"
:
"
tmmluplus_introduction_to_law"
"
task_alias"
:
"
introduction
to
law"
"
task_alias"
:
"
introduction
to
law"
lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml
View file @
543617fe
"
dataset_name"
:
"
jce_humanities"
"
dataset_name"
:
"
jce_humanities"
"
description"
:
"
以下為指考人文科目的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為指考人文科目的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_humanities"
"
tag"
:
"
tmmluplus_humanities_tasks"
"
group_alias"
:
"
humanities"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_jce_humanities"
"
task"
:
"
tmmluplus_jce_humanities"
"
task_alias"
:
"
jce
humanities"
"
task_alias"
:
"
jce
humanities"
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml
View file @
543617fe
"
dataset_name"
:
"
junior_chemistry"
"
dataset_name"
:
"
junior_chemistry"
"
description"
:
"
以下為國中理化的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為國中理化的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_STEM"
"
tag"
:
"
tmmluplus_STEM_tasks"
"
group_alias"
:
"
STEM"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_junior_chemistry"
"
task"
:
"
tmmluplus_junior_chemistry"
"
task_alias"
:
"
junior
chemistry"
"
task_alias"
:
"
junior
chemistry"
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chinese_exam.yaml
View file @
543617fe
"
dataset_name"
:
"
junior_chinese_exam"
"
dataset_name"
:
"
junior_chinese_exam"
"
description"
:
"
以下為國中會考基測國文的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為國中會考基測國文的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
tag"
:
"
tmmluplus_social_sciences_tasks"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_junior_chinese_exam"
"
task"
:
"
tmmluplus_junior_chinese_exam"
"
task_alias"
:
"
junior
chinese
exam"
"
task_alias"
:
"
junior
chinese
exam"
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_math_exam.yaml
View file @
543617fe
"
dataset_name"
:
"
junior_math_exam"
"
dataset_name"
:
"
junior_math_exam"
"
description"
:
"
以下為國中會考基測數學科的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為國中會考基測數學科的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_STEM"
"
tag"
:
"
tmmluplus_STEM_tasks"
"
group_alias"
:
"
STEM"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_junior_math_exam"
"
task"
:
"
tmmluplus_junior_math_exam"
"
task_alias"
:
"
junior
math
exam"
"
task_alias"
:
"
junior
math
exam"
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_science_exam.yaml
View file @
543617fe
"
dataset_name"
:
"
junior_science_exam"
"
dataset_name"
:
"
junior_science_exam"
"
description"
:
"
以下為國中會考基測自然科的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為國中會考基測自然科的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_STEM"
"
tag"
:
"
tmmluplus_STEM_tasks"
"
group_alias"
:
"
STEM"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_junior_science_exam"
"
task"
:
"
tmmluplus_junior_science_exam"
"
task_alias"
:
"
junior
science
exam"
"
task_alias"
:
"
junior
science
exam"
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_social_studies.yaml
View file @
543617fe
"
dataset_name"
:
"
junior_social_studies"
"
dataset_name"
:
"
junior_social_studies"
"
description"
:
"
以下為國中會考基測社會科的單選題,請提供正確答案的選項。
\n\n
"
"
description"
:
"
以下為國中會考基測社會科的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
tag"
:
"
tmmluplus_other_tasks"
"
group_alias"
:
"
other"
"
include"
:
"
_tmmluplus_template_yaml"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_junior_social_studies"
"
task"
:
"
tmmluplus_junior_social_studies"
"
task_alias"
:
"
junior
social
studies"
"
task_alias"
:
"
junior
social
studies"
Prev
1
2
3
4
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment