Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
a382359c
Commit
a382359c
authored
Jun 21, 2024
by
haileyschoelkopf
Browse files
add groups for agieval, aexams, aclue
parent
f48d87ec
Changes
8
Hide whitespace changes
Inline
Side-by-side
Showing
8 changed files
with
124 additions
and
2 deletions
+124
-2
lm_eval/tasks/aclue/_aclue.yaml
lm_eval/tasks/aclue/_aclue.yaml
+26
-0
lm_eval/tasks/aclue/_default_template_yaml
lm_eval/tasks/aclue/_default_template_yaml
+0
-1
lm_eval/tasks/aexams/_aexams.yaml
lm_eval/tasks/aexams/_aexams.yaml
+16
-0
lm_eval/tasks/aexams/_default_template_yaml
lm_eval/tasks/aexams/_default_template_yaml
+0
-1
lm_eval/tasks/agieval/agieval.yaml
lm_eval/tasks/agieval/agieval.yaml
+29
-0
lm_eval/tasks/agieval/agieval_cn.yaml
lm_eval/tasks/agieval/agieval_cn.yaml
+19
-0
lm_eval/tasks/agieval/agieval_en.yaml
lm_eval/tasks/agieval/agieval_en.yaml
+18
-0
lm_eval/tasks/agieval/agieval_nous.yaml
lm_eval/tasks/agieval/agieval_nous.yaml
+16
-0
No files found.
lm_eval/tasks/aclue/_aclue.yaml
0 → 100644
View file @
a382359c
group
:
aclue
task
:
-
aclue_ancient_chinese_culture
-
aclue_ancient_literature
-
aclue_ancient_medical
-
aclue_ancient_phonetics
-
aclue_basic_ancient_chinese
-
aclue_couplet_prediction
-
aclue_homographic_character_resolution
-
aclue_named_entity_recognition
-
aclue_poetry_appreciate
-
aclue_poetry_context_prediction
-
aclue_poetry_quality_assessment
-
aclue_poetry_sentiment_analysis
-
aclue_polysemy_resolution
-
aclue_reading_comprehension
-
aclue_sentence_segmentation
aggregate_metric
:
-
metric
:
acc
aggregation
:
mean
weight_by_size
:
true
-
metric
:
acc_norm
aggregation
:
mean
weight_by_size
:
true
metadata
:
version
:
0.0
lm_eval/tasks/aclue/_default_template_yaml
View file @
a382359c
group: aclue
dataset_path: tyouisen/aclue
test_split: test
fewshot_split: dev
...
...
lm_eval/tasks/aexams/_aexams.yaml
0 → 100644
View file @
a382359c
group
:
aexams
task
:
-
aexams_Biology
-
aexams_IslamicStudies
-
aexams_Physics
-
aexams_Science
-
aexams_Social
aggregate_metric
:
-
metric
:
acc
aggregation
:
mean
weight_by_size
:
true
-
metric
:
acc_norm
aggregation
:
mean
weight_by_size
:
true
metadata
:
version
:
0.0
lm_eval/tasks/aexams/_default_template_yaml
View file @
a382359c
group: aexams
dataset_path: Hennara/aexams
test_split: test
fewshot_split: dev
...
...
lm_eval/tasks/agieval/agieval.yaml
0 → 100644
View file @
a382359c
group
:
agieval
task
:
-
agieval_gaokao_biology
-
agieval_gaokao_chemistry
-
agieval_gaokao_chinese
-
agieval_gaokao_geography
-
agieval_gaokao_history
-
agieval_gaokao_mathcloze
-
agieval_gaokao_mathqa
-
agieval_gaokao_physics
-
agieval_jec_qa_ca
-
agieval_jec_qa_kd
-
agieval_logiqa_zh
-
agieval_aqua_rat
-
agieval_gaokao_english
-
agieval_logiqa_en
-
agieval_lsat_ar
-
agieval_lsat_lr
-
agieval_lsat_rc
-
agieval_math
-
agieval_sat_en_without_passage
-
agieval_sat_en
-
agieval_sat_math
aggregate_metric
:
-
metric
:
acc
aggregation
:
mean
weight_by_size
:
true
metadata
:
version
:
0.0
lm_eval/tasks/agieval/agieval_cn.yaml
0 → 100644
View file @
a382359c
group
:
agieval_cn
task
:
-
agieval_gaokao_biology
-
agieval_gaokao_chemistry
-
agieval_gaokao_chinese
-
agieval_gaokao_geography
-
agieval_gaokao_history
-
agieval_gaokao_mathcloze
-
agieval_gaokao_mathqa
-
agieval_gaokao_physics
-
agieval_jec_qa_ca
-
agieval_jec_qa_kd
-
agieval_logiqa_zh
aggregate_metric
:
-
metric
:
acc
aggregation
:
mean
weight_by_size
:
true
metadata
:
version
:
0.0
lm_eval/tasks/agieval/agieval_en.yaml
0 → 100644
View file @
a382359c
group
:
agieval_en
task
:
-
agieval_aqua_rat
-
agieval_gaokao_english
-
agieval_logiqa_en
-
agieval_lsat_ar
-
agieval_lsat_lr
-
agieval_lsat_rc
-
agieval_math
-
agieval_sat_en_without_passage
-
agieval_sat_en
-
agieval_sat_math
aggregate_metric
:
-
metric
:
acc
aggregation
:
mean
weight_by_size
:
true
metadata
:
version
:
0.0
lm_eval/tasks/agieval/agieval_nous.yaml
0 → 100644
View file @
a382359c
group
:
agieval_nous
task
:
-
agieval_aqua_rat
-
agieval_logiqa_en
-
agieval_lsat_ar
-
agieval_lsat_lr
-
agieval_lsat_rc
-
agieval_sat_en_without_passage
-
agieval_sat_en
-
agieval_sat_math
aggregate_metric
:
-
metric
:
acc_norm
aggregation
:
mean
weight_by_size
:
true
metadata
:
version
:
0.0
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment