Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
05a8191a
Commit
05a8191a
authored
Aug 14, 2023
by
lintangsutawika
Browse files
edit unscramble
parent
4c935519
Changes
6
Hide whitespace changes
Inline
Side-by-side
Showing
6 changed files
with
12 additions
and
6 deletions
+12
-6
lm_eval/tasks/unscramble/README.md
lm_eval/tasks/unscramble/README.md
+7
-1
lm_eval/tasks/unscramble/anagrams1.yaml
lm_eval/tasks/unscramble/anagrams1.yaml
+1
-1
lm_eval/tasks/unscramble/anagrams2.yaml
lm_eval/tasks/unscramble/anagrams2.yaml
+1
-1
lm_eval/tasks/unscramble/cycle_letters.yaml
lm_eval/tasks/unscramble/cycle_letters.yaml
+1
-1
lm_eval/tasks/unscramble/random_insertion.yaml
lm_eval/tasks/unscramble/random_insertion.yaml
+1
-1
lm_eval/tasks/unscramble/reversed_words.yaml
lm_eval/tasks/unscramble/reversed_words.yaml
+1
-1
No files found.
lm_eval/tasks/unscramble/README.md
View file @
05a8191a
...
@@ -28,7 +28,13 @@ Homepage: https://github.com/openai/gpt-3/tree/master/data
...
@@ -28,7 +28,13 @@ Homepage: https://github.com/openai/gpt-3/tree/master/data
}
}
```
```
### Subtasks
### Groups and Tasks
#### Groups
*
`unscramble`
#### Tasks
*
`anagrams1`
- Anagrams of all but the first and last letter.
*
`anagrams1`
- Anagrams of all but the first and last letter.
*
`anagrams2`
- Anagrams of all but the first and last 2 letters.
*
`anagrams2`
- Anagrams of all but the first and last 2 letters.
...
...
lm_eval/tasks/unscramble/anagrams1.yaml
View file @
05a8191a
group
:
group
:
-
greedy_until
-
unscramble
task
:
anagrams1
task
:
anagrams1
dataset_path
:
EleutherAI/unscramble
dataset_path
:
EleutherAI/unscramble
dataset_name
:
mid_word_1_anagrams
dataset_name
:
mid_word_1_anagrams
...
...
lm_eval/tasks/unscramble/anagrams2.yaml
View file @
05a8191a
group
:
group
:
-
greedy_until
-
unscramble
task
:
anagrams2
task
:
anagrams2
dataset_path
:
EleutherAI/unscramble
dataset_path
:
EleutherAI/unscramble
dataset_name
:
mid_word_2_anagrams
dataset_name
:
mid_word_2_anagrams
...
...
lm_eval/tasks/unscramble/cycle_letters.yaml
View file @
05a8191a
group
:
group
:
-
greedy_until
-
unscramble
task
:
cycle_letters
task
:
cycle_letters
dataset_path
:
EleutherAI/unscramble
dataset_path
:
EleutherAI/unscramble
dataset_name
:
cycle_letters_in_word
dataset_name
:
cycle_letters_in_word
...
...
lm_eval/tasks/unscramble/random_insertion.yaml
View file @
05a8191a
group
:
group
:
-
greedy_until
-
unscramble
task
:
random_insertion
task
:
random_insertion
dataset_path
:
EleutherAI/unscramble
dataset_path
:
EleutherAI/unscramble
dataset_name
:
random_insertion_in_word
dataset_name
:
random_insertion_in_word
...
...
lm_eval/tasks/unscramble/reversed_words.yaml
View file @
05a8191a
group
:
group
:
-
greedy_until
-
unscramble
task
:
reversed_words
task
:
reversed_words
dataset_path
:
EleutherAI/unscramble
dataset_path
:
EleutherAI/unscramble
dataset_name
:
reversed_words
dataset_name
:
reversed_words
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment