Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
7d09b24c
Commit
7d09b24c
authored
Jul 03, 2024
by
haileyschoelkopf
Browse files
fix alllll the merge conflicts
parents
96dfe976
6348b947
Changes
395
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
28 additions
and
31 deletions
+28
-31
lm_eval/tasks/translation/iwslt2017_en-ar.yaml
lm_eval/tasks/translation/iwslt2017_en-ar.yaml
+1
-2
lm_eval/tasks/translation/wmt14_en-fr.yaml
lm_eval/tasks/translation/wmt14_en-fr.yaml
+1
-2
lm_eval/tasks/translation/wmt14_fr-en.yaml
lm_eval/tasks/translation/wmt14_fr-en.yaml
+1
-2
lm_eval/tasks/translation/wmt16_de-en.yaml
lm_eval/tasks/translation/wmt16_de-en.yaml
+1
-2
lm_eval/tasks/translation/wmt16_en-de.yaml
lm_eval/tasks/translation/wmt16_en-de.yaml
+1
-2
lm_eval/tasks/translation/wmt16_en-ro.yaml
lm_eval/tasks/translation/wmt16_en-ro.yaml
+1
-2
lm_eval/tasks/translation/wmt16_ro-en.yaml
lm_eval/tasks/translation/wmt16_ro-en.yaml
+1
-2
lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml
lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml
+1
-1
lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml
lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml
+1
-1
lm_eval/tasks/unscramble/anagrams1.yaml
lm_eval/tasks/unscramble/anagrams1.yaml
+1
-1
lm_eval/tasks/unscramble/anagrams2.yaml
lm_eval/tasks/unscramble/anagrams2.yaml
+1
-1
lm_eval/tasks/unscramble/cycle_letters.yaml
lm_eval/tasks/unscramble/cycle_letters.yaml
+1
-1
lm_eval/tasks/unscramble/random_insertion.yaml
lm_eval/tasks/unscramble/random_insertion.yaml
+1
-1
lm_eval/tasks/unscramble/reversed_words.yaml
lm_eval/tasks/unscramble/reversed_words.yaml
+1
-1
lm_eval/tasks/webqs/webqs.yaml
lm_eval/tasks/webqs/webqs.yaml
+1
-1
lm_eval/tasks/wmdp/README.md
lm_eval/tasks/wmdp/README.md
+1
-1
lm_eval/tasks/wmdp/_default_template_yaml
lm_eval/tasks/wmdp/_default_template_yaml
+0
-1
lm_eval/tasks/wmdp/_wmdp.yaml
lm_eval/tasks/wmdp/_wmdp.yaml
+11
-0
lm_eval/tasks/wmt2016/README.md
lm_eval/tasks/wmt2016/README.md
+1
-5
lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml
lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml
+0
-2
No files found.
lm_eval/tasks/translation/iwslt2017_en-ar.yaml
View file @
7d09b24c
...
...
@@ -5,8 +5,7 @@ doc_to_target: ' {{translation["ar"]}}'
doc_to_text
:
'
English
phrase:
{{translation["en"]}}
Arabic
phrase:'
group
:
-
generate_until
tag
:
-
translation
-
iwslt2017
include
:
wmt_common_yaml
...
...
lm_eval/tasks/translation/wmt14_en-fr.yaml
View file @
7d09b24c
...
...
@@ -5,8 +5,7 @@ doc_to_target: ' {{translation["fr"]}}'
doc_to_text
:
'
English
phrase:
{{translation["en"]}}
French
phrase:'
group
:
-
generate_until
tag
:
-
translation
-
wmt14
-
gpt3_translation_benchmarks
...
...
lm_eval/tasks/translation/wmt14_fr-en.yaml
View file @
7d09b24c
...
...
@@ -5,8 +5,7 @@ doc_to_target: ' {{translation["en"]}}'
doc_to_text
:
'
French
phrase:
{{translation["fr"]}}
English
phrase:'
group
:
-
generate_until
tag
:
-
translation
-
wmt14
-
gpt3_translation_benchmarks
...
...
lm_eval/tasks/translation/wmt16_de-en.yaml
View file @
7d09b24c
...
...
@@ -5,8 +5,7 @@ doc_to_target: ' {{translation["en"]}}'
doc_to_text
:
'
German
phrase:
{{translation["de"]}}
English
phrase:'
group
:
-
generate_until
tag
:
-
translation
-
wmt16
-
gpt3_translation_benchmarks
...
...
lm_eval/tasks/translation/wmt16_en-de.yaml
View file @
7d09b24c
...
...
@@ -5,8 +5,7 @@ doc_to_target: ' {{translation["de"]}}'
doc_to_text
:
'
English
phrase:
{{translation["en"]}}
German
phrase:'
group
:
-
generate_until
tag
:
-
translation
-
wmt16
-
gpt3_translation_benchmarks
...
...
lm_eval/tasks/translation/wmt16_en-ro.yaml
View file @
7d09b24c
...
...
@@ -5,8 +5,7 @@ doc_to_target: ' {{translation["ro"]}}'
doc_to_text
:
'
English
phrase:
{{translation["en"]}}
Romanian
phrase:'
group
:
-
generate_until
tag
:
-
translation
-
wmt16
-
gpt3_translation_benchmarks
...
...
lm_eval/tasks/translation/wmt16_ro-en.yaml
View file @
7d09b24c
...
...
@@ -5,8 +5,7 @@ doc_to_target: ' {{translation["en"]}}'
doc_to_text
:
'
Romanian
phrase:
{{translation["ro"]}}
English
phrase:'
group
:
-
generate_until
tag
:
-
translation
-
wmt16
-
gpt3_translation_benchmarks
...
...
lm_eval/tasks/truthfulqa/truthfulqa_gen.yaml
View file @
7d09b24c
group
:
tag
:
-
truthfulqa
task
:
truthfulqa_gen
dataset_path
:
truthful_qa
...
...
lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml
View file @
7d09b24c
group
:
tag
:
-
truthfulqa
task
:
truthfulqa_mc1
dataset_path
:
truthful_qa
...
...
lm_eval/tasks/unscramble/anagrams1.yaml
View file @
7d09b24c
group
:
tag
:
-
unscramble
task
:
anagrams1
dataset_path
:
EleutherAI/unscramble
...
...
lm_eval/tasks/unscramble/anagrams2.yaml
View file @
7d09b24c
group
:
tag
:
-
unscramble
task
:
anagrams2
dataset_path
:
EleutherAI/unscramble
...
...
lm_eval/tasks/unscramble/cycle_letters.yaml
View file @
7d09b24c
group
:
tag
:
-
unscramble
task
:
cycle_letters
dataset_path
:
EleutherAI/unscramble
...
...
lm_eval/tasks/unscramble/random_insertion.yaml
View file @
7d09b24c
group
:
tag
:
-
unscramble
task
:
random_insertion
dataset_path
:
EleutherAI/unscramble
...
...
lm_eval/tasks/unscramble/reversed_words.yaml
View file @
7d09b24c
group
:
tag
:
-
unscramble
task
:
reversed_words
dataset_path
:
EleutherAI/unscramble
...
...
lm_eval/tasks/webqs/webqs.yaml
View file @
7d09b24c
group
:
tag
:
-
freebase
task
:
webqs
dataset_path
:
web_questions
...
...
lm_eval/tasks/wmdp/README.md
View file @
7d09b24c
...
...
@@ -24,7 +24,7 @@ Homepage: https://wmdp.ai
}
```
### Groups and Tasks
### Groups
, Tags,
and Tasks
#### Groups
...
...
lm_eval/tasks/wmdp/_default_template_yaml
View file @
7d09b24c
dataset_path: cais/wmdp
group: wmdp
test_split: test
training_split: null
validation_split: null
...
...
lm_eval/tasks/wmdp/_wmdp.yaml
0 → 100644
View file @
7d09b24c
group
:
wmdp
task
:
-
wmdp_bio
-
wmdp_chem
-
wmdp_cyber
aggregate_metric_list
:
-
metric
:
acc
aggregation
:
mean
weight_by_size
:
True
metadata
:
version
:
0
lm_eval/tasks/wmt2016/README.md
View file @
7d09b24c
...
...
@@ -27,11 +27,7 @@ Homepage: https://huggingface.co/datasets/wmt16
}
```
### Groups and Tasks
#### Groups
*
`wmt-t5-prompt`
: Group for all wmt tasks with prompt templates used for T5 (
`Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer`
)
### Groups, Tags, and Tasks
#### Tasks
...
...
lm_eval/tasks/wmt2016/ro_en-t5_prompt.yaml
View file @
7d09b24c
group
:
-
wmt-t5-prompt
task
:
wmt-ro-en-t5-prompt
dataset_path
:
wmt16
dataset_name
:
ro-en
...
...
Prev
1
…
15
16
17
18
19
20
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment