Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
605787a9
Commit
605787a9
authored
Sep 13, 2023
by
lintangsutawika
Browse files
add bigbench
parent
b8d1cef9
Changes
338
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
80 additions
and
0 deletions
+80
-0
lm_eval/tasks/bigbench/greedy_until/codenames.yaml
lm_eval/tasks/bigbench/greedy_until/codenames.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/color.yaml
lm_eval/tasks/bigbench/greedy_until/color.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/common_morpheme.yaml
lm_eval/tasks/bigbench/greedy_until/common_morpheme.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/conceptual_combinations.yaml
.../tasks/bigbench/greedy_until/conceptual_combinations.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/conlang_translation.yaml
lm_eval/tasks/bigbench/greedy_until/conlang_translation.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/contextual_parametric_knowledge_conflicts.yaml
...eedy_until/contextual_parametric_knowledge_conflicts.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/crash_blossom.yaml
lm_eval/tasks/bigbench/greedy_until/crash_blossom.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/crass_ai.yaml
lm_eval/tasks/bigbench/greedy_until/crass_ai.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/cryobiology_spanish.yaml
lm_eval/tasks/bigbench/greedy_until/cryobiology_spanish.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/cryptonite.yaml
lm_eval/tasks/bigbench/greedy_until/cryptonite.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/cs_algorithms.yaml
lm_eval/tasks/bigbench/greedy_until/cs_algorithms.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/dark_humor_detection.yaml
...val/tasks/bigbench/greedy_until/dark_humor_detection.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/date_understanding.yaml
lm_eval/tasks/bigbench/greedy_until/date_understanding.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/disambiguation_qa.yaml
lm_eval/tasks/bigbench/greedy_until/disambiguation_qa.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/discourse_marker_prediction.yaml
...ks/bigbench/greedy_until/discourse_marker_prediction.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/disfl_qa.yaml
lm_eval/tasks/bigbench/greedy_until/disfl_qa.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/dyck_languages.yaml
lm_eval/tasks/bigbench/greedy_until/dyck_languages.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/elementary_math_qa.yaml
lm_eval/tasks/bigbench/greedy_until/elementary_math_qa.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/emoji_movie.yaml
lm_eval/tasks/bigbench/greedy_until/emoji_movie.yaml
+4
-0
lm_eval/tasks/bigbench/greedy_until/emojis_emotion_prediction.yaml
...asks/bigbench/greedy_until/emojis_emotion_prediction.yaml
+4
-0
No files found.
lm_eval/tasks/bigbench/greedy_until/codenames.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
codenames
include
:
../greedy_until_template_yaml
task
:
bigbench_codenames_greedy_until
lm_eval/tasks/bigbench/greedy_until/color.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
color
include
:
../greedy_until_template_yaml
task
:
bigbench_color_greedy_until
lm_eval/tasks/bigbench/greedy_until/common_morpheme.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
common_morpheme
include
:
../greedy_until_template_yaml
task
:
bigbench_common_morpheme_greedy_until
lm_eval/tasks/bigbench/greedy_until/conceptual_combinations.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
conceptual_combinations
include
:
../greedy_until_template_yaml
task
:
bigbench_conceptual_combinations_greedy_until
lm_eval/tasks/bigbench/greedy_until/conlang_translation.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
conlang_translation
include
:
../greedy_until_template_yaml
task
:
bigbench_conlang_translation_greedy_until
lm_eval/tasks/bigbench/greedy_until/contextual_parametric_knowledge_conflicts.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
contextual_parametric_knowledge_conflicts
include
:
../greedy_until_template_yaml
task
:
bigbench_contextual_parametric_knowledge_conflicts_greedy_until
lm_eval/tasks/bigbench/greedy_until/crash_blossom.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
crash_blossom
include
:
../greedy_until_template_yaml
task
:
bigbench_crash_blossom_greedy_until
lm_eval/tasks/bigbench/greedy_until/crass_ai.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
crass_ai
include
:
../greedy_until_template_yaml
task
:
bigbench_crass_ai_greedy_until
lm_eval/tasks/bigbench/greedy_until/cryobiology_spanish.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
cryobiology_spanish
include
:
../greedy_until_template_yaml
task
:
bigbench_cryobiology_spanish_greedy_until
lm_eval/tasks/bigbench/greedy_until/cryptonite.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
cryptonite
include
:
../greedy_until_template_yaml
task
:
bigbench_cryptonite_greedy_until
lm_eval/tasks/bigbench/greedy_until/cs_algorithms.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
cs_algorithms
include
:
../greedy_until_template_yaml
task
:
bigbench_cs_algorithms_greedy_until
lm_eval/tasks/bigbench/greedy_until/dark_humor_detection.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
dark_humor_detection
include
:
../greedy_until_template_yaml
task
:
bigbench_dark_humor_detection_greedy_until
lm_eval/tasks/bigbench/greedy_until/date_understanding.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
date_understanding
include
:
../greedy_until_template_yaml
task
:
bigbench_date_understanding_greedy_until
lm_eval/tasks/bigbench/greedy_until/disambiguation_qa.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
disambiguation_qa
include
:
../greedy_until_template_yaml
task
:
bigbench_disambiguation_qa_greedy_until
lm_eval/tasks/bigbench/greedy_until/discourse_marker_prediction.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
discourse_marker_prediction
include
:
../greedy_until_template_yaml
task
:
bigbench_discourse_marker_prediction_greedy_until
lm_eval/tasks/bigbench/greedy_until/disfl_qa.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
disfl_qa
include
:
../greedy_until_template_yaml
task
:
bigbench_disfl_qa_greedy_until
lm_eval/tasks/bigbench/greedy_until/dyck_languages.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
dyck_languages
include
:
../greedy_until_template_yaml
task
:
bigbench_dyck_languages_greedy_until
lm_eval/tasks/bigbench/greedy_until/elementary_math_qa.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
elementary_math_qa
include
:
../greedy_until_template_yaml
task
:
bigbench_elementary_math_qa_greedy_until
lm_eval/tasks/bigbench/greedy_until/emoji_movie.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
emoji_movie
include
:
../greedy_until_template_yaml
task
:
bigbench_emoji_movie_greedy_until
lm_eval/tasks/bigbench/greedy_until/emojis_emotion_prediction.yaml
0 → 100644
View file @
605787a9
# Generated by utils.py
dataset_name
:
emojis_emotion_prediction
include
:
../greedy_until_template_yaml
task
:
bigbench_emojis_emotion_prediction_greedy_until
Prev
1
2
3
4
5
6
…
17
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment