Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
d97e7155
"diffusers-0.27.0/tests/pipelines/musicldm/test_musicldm.py" did not exist on "b6a532720541591addbe7d6d091adf4a3a430729"
Commit
d97e7155
authored
Aug 28, 2023
by
lintangsutawika
Browse files
added model written eval configs
parent
10c377ea
Changes
141
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
85 additions
and
0 deletions
+85
-0
lm_eval/tasks/model_written_evals/persona/self-replication.yaml
...l/tasks/model_written_evals/persona/self-replication.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml
.../tasks/model_written_evals/persona/stands-its-ground.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml
...ks/model_written_evals/persona/subscribes-to-Atheism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml
...s/model_written_evals/persona/subscribes-to-Buddhism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml
...del_written_evals/persona/subscribes-to-Christianity.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml
...del_written_evals/persona/subscribes-to-Confucianism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml
...s/model_written_evals/persona/subscribes-to-Hinduism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml
...asks/model_written_evals/persona/subscribes-to-Islam.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml
...ks/model_written_evals/persona/subscribes-to-Judaism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml
...sks/model_written_evals/persona/subscribes-to-Taoism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml
...itten_evals/persona/subscribes-to-act-utilitarianism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml
...n_evals/persona/subscribes-to-average-utilitarianism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml
...tten_evals/persona/subscribes-to-cultural-relativism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml
...model_written_evals/persona/subscribes-to-deontology.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml
...l_written_evals/persona/subscribes-to-moral-nihilism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml
...tten_evals/persona/subscribes-to-rule-utilitarianism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml
...ten_evals/persona/subscribes-to-total-utilitarianism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml
...l_written_evals/persona/subscribes-to-utilitarianism.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml
...el_written_evals/persona/subscribes-to-virtue-ethics.yaml
+4
-0
lm_eval/tasks/model_written_evals/persona/template_yaml
lm_eval/tasks/model_written_evals/persona/template_yaml
+9
-0
No files found.
lm_eval/tasks/model_written_evals/persona/self-replication.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
self-replication
include
:
template_yaml
task
:
persona_self-replication
lm_eval/tasks/model_written_evals/persona/stands-its-ground.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
stands-its-ground
include
:
template_yaml
task
:
persona_stands-its-ground
lm_eval/tasks/model_written_evals/persona/subscribes-to-Atheism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-Atheism
include
:
template_yaml
task
:
persona_subscribes-to-Atheism
lm_eval/tasks/model_written_evals/persona/subscribes-to-Buddhism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-Buddhism
include
:
template_yaml
task
:
persona_subscribes-to-Buddhism
lm_eval/tasks/model_written_evals/persona/subscribes-to-Christianity.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-Christianity
include
:
template_yaml
task
:
persona_subscribes-to-Christianity
lm_eval/tasks/model_written_evals/persona/subscribes-to-Confucianism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-Confucianism
include
:
template_yaml
task
:
persona_subscribes-to-Confucianism
lm_eval/tasks/model_written_evals/persona/subscribes-to-Hinduism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-Hinduism
include
:
template_yaml
task
:
persona_subscribes-to-Hinduism
lm_eval/tasks/model_written_evals/persona/subscribes-to-Islam.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-Islam
include
:
template_yaml
task
:
persona_subscribes-to-Islam
lm_eval/tasks/model_written_evals/persona/subscribes-to-Judaism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-Judaism
include
:
template_yaml
task
:
persona_subscribes-to-Judaism
lm_eval/tasks/model_written_evals/persona/subscribes-to-Taoism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-Taoism
include
:
template_yaml
task
:
persona_subscribes-to-Taoism
lm_eval/tasks/model_written_evals/persona/subscribes-to-act-utilitarianism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-act-utilitarianism
include
:
template_yaml
task
:
persona_subscribes-to-act-utilitarianism
lm_eval/tasks/model_written_evals/persona/subscribes-to-average-utilitarianism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-average-utilitarianism
include
:
template_yaml
task
:
persona_subscribes-to-average-utilitarianism
lm_eval/tasks/model_written_evals/persona/subscribes-to-cultural-relativism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-cultural-relativism
include
:
template_yaml
task
:
persona_subscribes-to-cultural-relativism
lm_eval/tasks/model_written_evals/persona/subscribes-to-deontology.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-deontology
include
:
template_yaml
task
:
persona_subscribes-to-deontology
lm_eval/tasks/model_written_evals/persona/subscribes-to-moral-nihilism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-moral-nihilism
include
:
template_yaml
task
:
persona_subscribes-to-moral-nihilism
lm_eval/tasks/model_written_evals/persona/subscribes-to-rule-utilitarianism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-rule-utilitarianism
include
:
template_yaml
task
:
persona_subscribes-to-rule-utilitarianism
lm_eval/tasks/model_written_evals/persona/subscribes-to-total-utilitarianism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-total-utilitarianism
include
:
template_yaml
task
:
persona_subscribes-to-total-utilitarianism
lm_eval/tasks/model_written_evals/persona/subscribes-to-utilitarianism.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-utilitarianism
include
:
template_yaml
task
:
persona_subscribes-to-utilitarianism
lm_eval/tasks/model_written_evals/persona/subscribes-to-virtue-ethics.yaml
0 → 100644
View file @
d97e7155
# Generated by generate_configs.py
dataset_name
:
subscribes-to-virtue-ethics
include
:
template_yaml
task
:
persona_subscribes-to-virtue-ethics
lm_eval/tasks/model_written_evals/persona/template_yaml
0 → 100644
View file @
d97e7155
group: persona
dataset_path: lm_eval.datasets.model_written_evals.persona
output_type: multiple_choice
validation_split: train
doc_to_text: "{{question}}"
doc_to_target: 0
doc_to_choice: "{{[answer_matching_behavior, answer_not_matching_behavior]}}"
metric_list:
- metric: acc
Prev
1
2
3
4
5
6
7
8
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment