Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
219c81bd
Commit
219c81bd
authored
Aug 14, 2023
by
lintangsutawika
Browse files
add lambada
parent
986943e5
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
15 additions
and
6 deletions
+15
-6
lm_eval/tasks/lambada/README.md
lm_eval/tasks/lambada/README.md
+15
-2
lm_eval/tasks/lambada/lambada_openai.yaml
lm_eval/tasks/lambada/lambada_openai.yaml
+0
-2
lm_eval/tasks/lambada/lambada_standard.yaml
lm_eval/tasks/lambada/lambada_standard.yaml
+0
-2
No files found.
lm_eval/tasks/lambada/README.md
View file @
219c81bd
# LAMBADA
### Paper
The LAMBADA dataset: Word prediction requiring a broad discourse context
https://arxiv.org/pdf/1606.06031.pdf
Title:
`The LAMBADA dataset: Word prediction requiring a broad discourse context`
Abstract: https://arxiv.org/pdf/1606.06031.pdf
LAMBADA is a dataset to evaluate the capabilities of computational models for text
understanding by means of a word prediction task. LAMBADA is a collection of narrative
...
...
@@ -14,6 +15,18 @@ in the broader discourse.
Homepage: https://zenodo.org/record/2630551#.X4Xzn5NKjUI
### Groups and Tasks
#### Groups
-
`lambada`
#### Tasks
-
`lambada_openai`
-
`lambada_standard`
### Citation
@misc{
...
...
lm_eval/tasks/lambada/lambada_openai.yaml
View file @
219c81bd
group
:
-
lambada
-
loglikelihood
-
perplexity
task
:
lambada_openai
dataset_path
:
EleutherAI/lambada_openai
dataset_name
:
default
...
...
lm_eval/tasks/lambada/lambada_standard.yaml
View file @
219c81bd
group
:
-
lambada
-
loglikelihood
-
perplexity
task
:
lambada_standard
dataset_path
:
lambada
dataset_name
:
null
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment