Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
7d09b24c
Commit
7d09b24c
authored
Jul 03, 2024
by
haileyschoelkopf
Browse files
fix alllll the merge conflicts
parents
96dfe976
6348b947
Changes
395
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
87 additions
and
4 deletions
+87
-4
lm_eval/tasks/cmmlu/cmmlu_marketing.yaml
lm_eval/tasks/cmmlu/cmmlu_marketing.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml
lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_modern_chinese.yaml
lm_eval/tasks/cmmlu/cmmlu_modern_chinese.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml
lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml
lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_professional_accounting.yaml
lm_eval/tasks/cmmlu/cmmlu_professional_accounting.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_professional_law.yaml
lm_eval/tasks/cmmlu/cmmlu_professional_law.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_professional_medicine.yaml
lm_eval/tasks/cmmlu/cmmlu_professional_medicine.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_professional_psychology.yaml
lm_eval/tasks/cmmlu/cmmlu_professional_psychology.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_public_relations.yaml
lm_eval/tasks/cmmlu/cmmlu_public_relations.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_security_study.yaml
lm_eval/tasks/cmmlu/cmmlu_security_study.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_sociology.yaml
lm_eval/tasks/cmmlu/cmmlu_sociology.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_sports_science.yaml
lm_eval/tasks/cmmlu/cmmlu_sports_science.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_traditional_chinese_medicine.yaml
lm_eval/tasks/cmmlu/cmmlu_traditional_chinese_medicine.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_virology.yaml
lm_eval/tasks/cmmlu/cmmlu_virology.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_world_history.yaml
lm_eval/tasks/cmmlu/cmmlu_world_history.yaml
+4
-0
lm_eval/tasks/cmmlu/cmmlu_world_religions.yaml
lm_eval/tasks/cmmlu/cmmlu_world_religions.yaml
+4
-0
lm_eval/tasks/copal_id/standard.yaml
lm_eval/tasks/copal_id/standard.yaml
+1
-1
lm_eval/tasks/crows_pairs/crows_pairs_english.yaml
lm_eval/tasks/crows_pairs/crows_pairs_english.yaml
+1
-3
lm_eval/tasks/csatqa/_csatqa.yaml
lm_eval/tasks/csatqa/_csatqa.yaml
+17
-0
No files found.
lm_eval/tasks/cmmlu/cmmlu_marketing.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
marketing"
"
description"
:
"
以下是关于市场营销的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_marketing"
lm_eval/tasks/cmmlu/cmmlu_marxist_theory.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
marxist_theory"
"
description"
:
"
以下是关于马克思主义理论的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_marxist_theory"
lm_eval/tasks/cmmlu/cmmlu_modern_chinese.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
modern_chinese"
"
description"
:
"
以下是关于现代汉语的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_modern_chinese"
lm_eval/tasks/cmmlu/cmmlu_nutrition.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
nutrition"
"
description"
:
"
以下是关于营养学的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_nutrition"
lm_eval/tasks/cmmlu/cmmlu_philosophy.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
philosophy"
"
description"
:
"
以下是关于哲学的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_philosophy"
lm_eval/tasks/cmmlu/cmmlu_professional_accounting.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
professional_accounting"
"
description"
:
"
以下是关于专业会计的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_professional_accounting"
lm_eval/tasks/cmmlu/cmmlu_professional_law.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
professional_law"
"
description"
:
"
以下是关于专业法学的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_professional_law"
lm_eval/tasks/cmmlu/cmmlu_professional_medicine.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
professional_medicine"
"
description"
:
"
以下是关于专业医学的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_professional_medicine"
lm_eval/tasks/cmmlu/cmmlu_professional_psychology.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
professional_psychology"
"
description"
:
"
以下是关于专业心理学的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_professional_psychology"
lm_eval/tasks/cmmlu/cmmlu_public_relations.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
public_relations"
"
description"
:
"
以下是关于公共关系的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_public_relations"
lm_eval/tasks/cmmlu/cmmlu_security_study.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
security_study"
"
description"
:
"
以下是关于安全研究的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_security_study"
lm_eval/tasks/cmmlu/cmmlu_sociology.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
sociology"
"
description"
:
"
以下是关于社会学的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_sociology"
lm_eval/tasks/cmmlu/cmmlu_sports_science.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
sports_science"
"
description"
:
"
以下是关于体育学的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_sports_science"
lm_eval/tasks/cmmlu/cmmlu_traditional_chinese_medicine.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
traditional_chinese_medicine"
"
description"
:
"
以下是关于中医中药的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_traditional_chinese_medicine"
lm_eval/tasks/cmmlu/cmmlu_virology.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
virology"
"
description"
:
"
以下是关于病毒学的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_virology"
lm_eval/tasks/cmmlu/cmmlu_world_history.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
world_history"
"
description"
:
"
以下是关于世界历史的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_world_history"
lm_eval/tasks/cmmlu/cmmlu_world_religions.yaml
0 → 100644
View file @
7d09b24c
"
dataset_name"
:
"
world_religions"
"
description"
:
"
以下是关于世界宗教的单项选择题,请直接给出正确答案的选项。
\n\n
"
"
include"
:
"
_default_template_yaml"
"
task"
:
"
cmmlu_world_religions"
lm_eval/tasks/copal_id/standard.yaml
View file @
7d09b24c
group
:
copal_id
tag
:
copal_id
task
:
copal_id_standard
task
:
copal_id_standard
task_alias
:
standard
task_alias
:
standard
dataset_path
:
haryoaw/COPAL
dataset_path
:
haryoaw/COPAL
...
...
lm_eval/tasks/crows_pairs/crows_pairs_english.yaml
View file @
7d09b24c
group
:
tag
:
-
crows_pairs
-
crows_pairs
-
social_bias
-
loglikelihood
task
:
crows_pairs_english
task
:
crows_pairs_english
dataset_path
:
BigScienceBiasEval/crows_pairs_multilingual
dataset_path
:
BigScienceBiasEval/crows_pairs_multilingual
dataset_name
:
english
dataset_name
:
english
...
...
lm_eval/tasks/csatqa/_csatqa.yaml
0 → 100644
View file @
7d09b24c
group
:
csatqa
task
:
-
csatqa_gr
-
csatqa_li
-
csatqa_rch
-
csatqa_rcs
-
csatqa_rcss
-
csatqa_wr
aggregate_metric_list
:
-
metric
:
acc
aggregation
:
mean
weight_by_size
:
true
-
metric
:
acc_norm
aggregation
:
mean
weight_by_size
:
true
metadata
:
version
:
0.0
Prev
1
…
3
4
5
6
7
8
9
10
11
…
20
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment