Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
223a63c4
Commit
223a63c4
authored
Aug 14, 2023
by
lintangsutawika
Browse files
edit lambada_multilingual
parent
66e05496
Changes
6
Hide whitespace changes
Inline
Side-by-side
Showing
6 changed files
with
7 additions
and
19 deletions
+7
-19
lm_eval/tasks/lambada_multilingual/README.md
lm_eval/tasks/lambada_multilingual/README.md
+7
-1
lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml
lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml
+0
-4
lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml
lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml
+0
-2
lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml
lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml
+0
-4
lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml
lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml
+0
-4
lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml
lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml
+0
-4
No files found.
lm_eval/tasks/lambada_multilingual/README.md
View file @
223a63c4
...
@@ -25,7 +25,13 @@ Homepage: https://zenodo.org/record/2630551#.X4Xzn5NKjUI
...
@@ -25,7 +25,13 @@ Homepage: https://zenodo.org/record/2630551#.X4Xzn5NKjUI
month={Aug}
month={Aug}
}
}
### Subtasks
### Groups and Tasks
#### Groups
*
`lambada_multilingual`
: Evaluates all
`lambada_mt_X`
tasks
#### Tasks
*
`lambada_mt_{en, fr, de, it, es}`
: Machine-translated versions of OpenAI's Lambada variant.
*
`lambada_mt_{en, fr, de, it, es}`
: Machine-translated versions of OpenAI's Lambada variant.
...
...
lm_eval/tasks/lambada_multilingual/lambada_mt_de.yaml
View file @
223a63c4
include
:
lambada_mt_en.yaml
include
:
lambada_mt_en.yaml
group
:
-
lambada_multilingual
-
loglikelihood
-
perplexity
task
:
lambada_openai_mt_de
task
:
lambada_openai_mt_de
dataset_name
:
de
dataset_name
:
de
lm_eval/tasks/lambada_multilingual/lambada_mt_en.yaml
View file @
223a63c4
group
:
group
:
-
lambada_multilingual
-
lambada_multilingual
-
loglikelihood
-
perplexity
task
:
lambada_openai_mt_en
task
:
lambada_openai_mt_en
dataset_path
:
EleutherAI/lambada_openai
dataset_path
:
EleutherAI/lambada_openai
dataset_name
:
en
dataset_name
:
en
...
...
lm_eval/tasks/lambada_multilingual/lambada_mt_es.yaml
View file @
223a63c4
include
:
lambada_mt_en.yaml
include
:
lambada_mt_en.yaml
group
:
-
lambada_multilingual
-
loglikelihood
-
perplexity
task
:
lambada_openai_mt_es
task
:
lambada_openai_mt_es
dataset_name
:
es
dataset_name
:
es
lm_eval/tasks/lambada_multilingual/lambada_mt_fr.yaml
View file @
223a63c4
include
:
lambada_mt_en.yaml
include
:
lambada_mt_en.yaml
group
:
-
lambada_multilingual
-
loglikelihood
-
perplexity
task
:
lambada_openai_mt_fr
task
:
lambada_openai_mt_fr
dataset_name
:
fr
dataset_name
:
fr
lm_eval/tasks/lambada_multilingual/lambada_mt_it.yaml
View file @
223a63c4
include
:
lambada_mt_en.yaml
include
:
lambada_mt_en.yaml
group
:
-
lambada_multilingual
-
loglikelihood
-
perplexity
task
:
lambada_openai_mt_it
task
:
lambada_openai_mt_it
dataset_name
:
it
dataset_name
:
it
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment