Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
60c9c170
Commit
60c9c170
authored
May 29, 2024
by
haileyschoelkopf
Browse files
Merge branch 'main' into inverse-scaling-tasks
parents
4b2d565b
b4cd85d4
Changes
605
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
140 additions
and
0 deletions
+140
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml
...us/default/tmmluplus_chinese_language_and_literature.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml
...asks/tmmluplus/default/tmmluplus_clinical_psychology.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml
...l/tasks/tmmluplus/default/tmmluplus_computer_science.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml
...al/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml
lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml
lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml
lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml
...uplus/default/tmmluplus_education_(profession_level).yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml
...s/tmmluplus/default/tmmluplus_educational_psychology.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml
...l/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml
...al/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml
...tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml
lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml
...mmluplus/default/tmmluplus_general_principles_of_law.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml
...asks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml
...val/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml
.../tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml
...asks/tmmluplus/default/tmmluplus_introduction_to_law.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml
...val/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml
+7
-0
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml
...l/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml
+7
-0
No files found.
lm_eval/tasks/tmmluplus/default/tmmluplus_chinese_language_and_literature.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
chinese_language_and_literature"
"
description"
:
"
以下為國文的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_chinese_language_and_literature"
"
task_alias"
:
"
chinese
language
and
literature"
lm_eval/tasks/tmmluplus/default/tmmluplus_clinical_psychology.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
clinical_psychology"
"
description"
:
"
以下為臨床心理學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_clinical_psychology"
"
task_alias"
:
"
clinical
psychology"
lm_eval/tasks/tmmluplus/default/tmmluplus_computer_science.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
computer_science"
"
description"
:
"
以下為資訊工程的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_STEM"
"
group_alias"
:
"
STEM"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_computer_science"
"
task_alias"
:
"
computer
science"
lm_eval/tasks/tmmluplus/default/tmmluplus_culinary_skills.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
culinary_skills"
"
description"
:
"
以下為餐旅的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
group_alias"
:
"
other"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_culinary_skills"
"
task_alias"
:
"
culinary
skills"
lm_eval/tasks/tmmluplus/default/tmmluplus_dentistry.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
dentistry"
"
description"
:
"
以下為牙醫學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
group_alias"
:
"
other"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_dentistry"
"
task_alias"
:
"
dentistry"
lm_eval/tasks/tmmluplus/default/tmmluplus_economics.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
economics"
"
description"
:
"
以下為經濟學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_economics"
"
task_alias"
:
"
economics"
lm_eval/tasks/tmmluplus/default/tmmluplus_education.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
education"
"
description"
:
"
以下為教育常識的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_education"
"
task_alias"
:
"
education"
lm_eval/tasks/tmmluplus/default/tmmluplus_education_(profession_level).yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
education_(profession_level)"
"
description"
:
"
以下為教育專業的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_education_(profession_level)"
"
task_alias"
:
"
education
(profession
level)"
lm_eval/tasks/tmmluplus/default/tmmluplus_educational_psychology.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
educational_psychology"
"
description"
:
"
以下為教育心理的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_educational_psychology"
"
task_alias"
:
"
educational
psychology"
lm_eval/tasks/tmmluplus/default/tmmluplus_engineering_math.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
engineering_math"
"
description"
:
"
以下為工程數學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_STEM"
"
group_alias"
:
"
STEM"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_engineering_math"
"
task_alias"
:
"
engineering
math"
lm_eval/tasks/tmmluplus/default/tmmluplus_finance_banking.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
finance_banking"
"
description"
:
"
以下為金融與法規的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
group_alias"
:
"
other"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_finance_banking"
"
task_alias"
:
"
finance
banking"
lm_eval/tasks/tmmluplus/default/tmmluplus_financial_analysis.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
financial_analysis"
"
description"
:
"
以下為財務分析的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
group_alias"
:
"
other"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_financial_analysis"
"
task_alias"
:
"
financial
analysis"
lm_eval/tasks/tmmluplus/default/tmmluplus_fire_science.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
fire_science"
"
description"
:
"
以下為火災學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
group_alias"
:
"
other"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_fire_science"
"
task_alias"
:
"
fire
science"
lm_eval/tasks/tmmluplus/default/tmmluplus_general_principles_of_law.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
general_principles_of_law"
"
description"
:
"
以下為法學大意的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_humanities"
"
group_alias"
:
"
humanities"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_general_principles_of_law"
"
task_alias"
:
"
general
principles
of
law"
lm_eval/tasks/tmmluplus/default/tmmluplus_geography_of_taiwan.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
geography_of_taiwan"
"
description"
:
"
以下為台灣地理的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_geography_of_taiwan"
"
task_alias"
:
"
geography
of
taiwan"
lm_eval/tasks/tmmluplus/default/tmmluplus_human_behavior.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
human_behavior"
"
description"
:
"
以下為人類行為與社會的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_social_sciences"
"
group_alias"
:
"
social
sciences"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_human_behavior"
"
task_alias"
:
"
human
behavior"
lm_eval/tasks/tmmluplus/default/tmmluplus_insurance_studies.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
insurance_studies"
"
description"
:
"
以下為保險學的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_other"
"
group_alias"
:
"
other"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_insurance_studies"
"
task_alias"
:
"
insurance
studies"
lm_eval/tasks/tmmluplus/default/tmmluplus_introduction_to_law.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
introduction_to_law"
"
description"
:
"
以下為法律概論的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_humanities"
"
group_alias"
:
"
humanities"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_introduction_to_law"
"
task_alias"
:
"
introduction
to
law"
lm_eval/tasks/tmmluplus/default/tmmluplus_jce_humanities.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
jce_humanities"
"
description"
:
"
以下為指考人文科目的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_humanities"
"
group_alias"
:
"
humanities"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_jce_humanities"
"
task_alias"
:
"
jce
humanities"
lm_eval/tasks/tmmluplus/default/tmmluplus_junior_chemistry.yaml
0 → 100644
View file @
60c9c170
"
dataset_name"
:
"
junior_chemistry"
"
description"
:
"
以下為國中理化的單選題,請提供正確答案的選項。
\n\n
"
"
group"
:
"
tmmluplus_STEM"
"
group_alias"
:
"
STEM"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
tmmluplus_junior_chemistry"
"
task_alias"
:
"
junior
chemistry"
Prev
1
…
22
23
24
25
26
27
28
29
30
31
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment