Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
4f5b72bc
Commit
4f5b72bc
authored
Sep 04, 2023
by
lintangsutawika
Browse files
fixed template
parent
4a752867
Changes
109
Hide whitespace changes
Inline
Side-by-side
Showing
9 changed files
with
10 additions
and
10 deletions
+10
-10
lm_eval/tasks/bbh/flan_zeroshot/salient_translation_error_detection.yaml
...bh/flan_zeroshot/salient_translation_error_detection.yaml
+1
-1
lm_eval/tasks/bbh/flan_zeroshot/snarks.yaml
lm_eval/tasks/bbh/flan_zeroshot/snarks.yaml
+2
-2
lm_eval/tasks/bbh/flan_zeroshot/sports_understanding.yaml
lm_eval/tasks/bbh/flan_zeroshot/sports_understanding.yaml
+1
-1
lm_eval/tasks/bbh/flan_zeroshot/temporal_sequences.yaml
lm_eval/tasks/bbh/flan_zeroshot/temporal_sequences.yaml
+1
-1
lm_eval/tasks/bbh/flan_zeroshot/tracking_shuffled_objects_five_objects.yaml
...flan_zeroshot/tracking_shuffled_objects_five_objects.yaml
+1
-1
lm_eval/tasks/bbh/flan_zeroshot/tracking_shuffled_objects_seven_objects.yaml
...lan_zeroshot/tracking_shuffled_objects_seven_objects.yaml
+1
-1
lm_eval/tasks/bbh/flan_zeroshot/tracking_shuffled_objects_three_objects.yaml
...lan_zeroshot/tracking_shuffled_objects_three_objects.yaml
+1
-1
lm_eval/tasks/bbh/flan_zeroshot/web_of_lies.yaml
lm_eval/tasks/bbh/flan_zeroshot/web_of_lies.yaml
+1
-1
lm_eval/tasks/bbh/flan_zeroshot/word_sorting.yaml
lm_eval/tasks/bbh/flan_zeroshot/word_sorting.yaml
+1
-1
No files found.
lm_eval/tasks/bbh/flan_zeroshot/salient_translation_error_detection.yaml
View file @
4f5b72bc
"
dataset_name"
:
"
salient_translation_error_detection"
"
description"
:
"
Detect
the
type
of
error
in
an
English
translation
of
a
German
source
sentence.
\n\n
"
"
doc_to_text"
:
"
Q:
{{input}}
\n
A:"
"
include"
:
"
_template_yaml"
"
include"
:
"
_
flan_zeroshot_
template_yaml"
"
task"
:
"
bbh_flan_zeroshot_salient_translation_error_detection"
lm_eval/tasks/bbh/flan_zeroshot/snarks.yaml
View file @
4f5b72bc
"
dataset_name"
:
"
snarks"
"
description"
:
"
Determine
which
of
two
sentences
is
sarcastic.
\n\n
"
"
description"
:
"
Determine
which
of
two
sentences
is
sarcastic.
\n\
n
According
to
Cambridge
University
Dictionary,
sarcasm
is
\"
the
use
of
remarks
that
clearly
mean
the
opposite
of
what
they
say,
made
in
order
to
hurt
someone's
feelings
or
to
criticize
something
in
a
humorous
way.
\"
Sarcastic
sentences
often
contain
satirical
or
ironic
utterances,
hyperboles,
ambivalent
or
witty
remarks.
\n\
n
"
"
doc_to_text"
:
"
Q:
{{input}}
\n
A:"
"
include"
:
"
_template_yaml"
"
include"
:
"
_
flan_zeroshot_
template_yaml"
"
task"
:
"
bbh_flan_zeroshot_snarks"
lm_eval/tasks/bbh/flan_zeroshot/sports_understanding.yaml
View file @
4f5b72bc
"
dataset_name"
:
"
sports_understanding"
"
description"
:
"
Determine
whether
an
artificially
constructed
sentence
relating
to
sports
is
plausible
or
not.
\n\n
"
"
doc_to_text"
:
"
Q:
{{input}}
\n
A:"
"
include"
:
"
_template_yaml"
"
include"
:
"
_
flan_zeroshot_
template_yaml"
"
task"
:
"
bbh_flan_zeroshot_sports_understanding"
lm_eval/tasks/bbh/flan_zeroshot/temporal_sequences.yaml
View file @
4f5b72bc
"
dataset_name"
:
"
temporal_sequences"
"
description"
:
"
Task
description:
Answer
questions
about
which
times
certain
events
could
have
occurred.
\n\n
"
"
doc_to_text"
:
"
Q:
{{input}}
\n
A:"
"
include"
:
"
_template_yaml"
"
include"
:
"
_
flan_zeroshot_
template_yaml"
"
task"
:
"
bbh_flan_zeroshot_temporal_sequences"
lm_eval/tasks/bbh/flan_zeroshot/tracking_shuffled_objects_five_objects.yaml
View file @
4f5b72bc
"
dataset_name"
:
"
tracking_shuffled_objects_five_objects"
"
description"
:
"
A
task
requiring
determining
the
final
positions
of
a
set
of
objects
given
their
initial
positions
and
a
description
of
a
sequence
of
swaps.
\n\n
"
"
doc_to_text"
:
"
Q:
{{input}}
\n
A:"
"
include"
:
"
_template_yaml"
"
include"
:
"
_
flan_zeroshot_
template_yaml"
"
task"
:
"
bbh_flan_zeroshot_tracking_shuffled_objects_five_objects"
lm_eval/tasks/bbh/flan_zeroshot/tracking_shuffled_objects_seven_objects.yaml
View file @
4f5b72bc
"
dataset_name"
:
"
tracking_shuffled_objects_seven_objects"
"
description"
:
"
A
task
requiring
determining
the
final
positions
of
a
set
of
objects
given
their
initial
positions
and
a
description
of
a
sequence
of
swaps.
\n\n
"
"
doc_to_text"
:
"
Q:
{{input}}
\n
A:"
"
include"
:
"
_template_yaml"
"
include"
:
"
_
flan_zeroshot_
template_yaml"
"
task"
:
"
bbh_flan_zeroshot_tracking_shuffled_objects_seven_objects"
lm_eval/tasks/bbh/flan_zeroshot/tracking_shuffled_objects_three_objects.yaml
View file @
4f5b72bc
"
dataset_name"
:
"
tracking_shuffled_objects_three_objects"
"
description"
:
"
A
task
requiring
determining
the
final
positions
of
a
set
of
objects
given
their
initial
positions
and
a
description
of
a
sequence
of
swaps.
\n\n
"
"
doc_to_text"
:
"
Q:
{{input}}
\n
A:"
"
include"
:
"
_template_yaml"
"
include"
:
"
_
flan_zeroshot_
template_yaml"
"
task"
:
"
bbh_flan_zeroshot_tracking_shuffled_objects_three_objects"
lm_eval/tasks/bbh/flan_zeroshot/web_of_lies.yaml
View file @
4f5b72bc
"
dataset_name"
:
"
web_of_lies"
"
description"
:
"
Evaluate
a
random
boolean
function
expressed
as
a
word
problem.
\n\n
"
"
doc_to_text"
:
"
Q:
{{input}}
\n
A:"
"
include"
:
"
_template_yaml"
"
include"
:
"
_
flan_zeroshot_
template_yaml"
"
task"
:
"
bbh_flan_zeroshot_web_of_lies"
lm_eval/tasks/bbh/flan_zeroshot/word_sorting.yaml
View file @
4f5b72bc
"
dataset_name"
:
"
word_sorting"
"
description"
:
"
Sort
a
list
of
words.
\n\n
"
"
doc_to_text"
:
"
Q:
{{input}}
\n
A:"
"
include"
:
"
_template_yaml"
"
include"
:
"
_
flan_zeroshot_
template_yaml"
"
task"
:
"
bbh_flan_zeroshot_word_sorting"
Prev
1
2
3
4
5
6
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment