Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
da211969
Unverified
Commit
da211969
authored
Jun 28, 2024
by
Jess
Committed by
GitHub
Jun 28, 2024
Browse files
Merge branch 'EleutherAI:main' into main
parents
1b97e487
801322e0
Changes
654
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
1433 additions
and
1100 deletions
+1433
-1100
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml
...u/flan_cot_fewshot/mmlu_high_school_european_history.yaml
+199
-169
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml
...sks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml
+53
-37
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml
...cot_fewshot/mmlu_high_school_government_and_politics.yaml
+61
-43
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml
...mlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml
+53
-37
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml
...s/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml
+51
-37
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml
...mlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml
+56
-40
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml
...tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml
+50
-39
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml
...ks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml
+64
-48
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml
...ks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml
+81
-58
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml
...ks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml
+156
-134
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml
...mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml
+100
-82
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml
+42
-28
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml
...val/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml
+50
-34
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml
...l/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml
+70
-54
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml
+59
-45
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml
...l/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml
+61
-45
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml
...al/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml
+74
-60
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml
+46
-33
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml
+56
-40
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml
...al/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml
+51
-37
No files found.
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_european_history"
dataset_name
:
high_school_european_history
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
european
history.
\n\n
Q:
This
question
refers
to
the
following
information.
\n\
school european history.
Albeit
the
king's
Majesty
justly
and
rightfully
is
and
ought
to
be
the
supreme
head
\
fewshot_config
:
\
of
the
Church
of
England,
and
so
is
recognized
by
the
clergy
of
this
realm
in
\
sampler
:
first_n
\
their
convocations,
yet
nevertheless,
for
corroboration
and
confirmation
thereof,
\
samples
:
\
and
for
increase
of
virtue
in
Christ's
religion
within
this
realm
of
England,
\
-
question
:
'
This
question
refers
to
the
following
information.
\
and
to
repress
and
extirpate
all
errors,
heresies,
and
other
enormities
and
abuses
\
\
heretofore
used
in
the
same,
be
it
enacted,
by
authority
of
this
present
Parliament,
\
Albeit
the
king'
'
s
Majesty
justly
and
rightfully
is
and
ought
to
be
the
supreme
\
that
the
king,
our
sovereign
lord,
his
heirs
and
successors,
kings
of
this
realm,
\
head
of
the
Church
of
England,
and
so
is
recognized
by
the
clergy
of
this
realm
\
shall
be
taken,
accepted,
and
reputed
the
only
supreme
head
in
earth
of
the
Church
\
in
their
convocations,
yet
nevertheless,
for
corroboration
and
confirmation
\
of
England,
called
Anglicans
Ecclesia;
and
shall
have
and
enjoy,
annexed
and
united
\
thereof,
and
for
increase
of
virtue
in
Christ'
'
s
religion
within
this
realm
\
to
the
imperial
crown
of
this
realm,
as
well
the
title
and
style
thereof,
as
all
\
of
England,
and
to
repress
and
extirpate
all
errors,
heresies,
and
other
enormities
\
honors,
dignities,
preeminences,
jurisdictions,
privileges,
authorities,
immunities,
\
and
abuses
heretofore
used
in
the
same,
be
it
enacted,
by
authority
of
this
\
profits,
and
commodities
to
the
said
dignity
of
the
supreme
head
of
the
same
Church
\
present
Parliament,
that
the
king,
our
sovereign
lord,
his
heirs
and
successors,
\
belonging
and
appertaining;
and
that
our
said
sovereign
lord,
his
heirs
and
successors,
\
kings
of
this
realm,
shall
be
taken,
accepted,
and
reputed
the
only
supreme
\
kings
of
this
realm,
shall
have
full
power
and
authority
from
time
to
time
to
\
head
in
earth
of
the
Church
of
England,
called
Anglicans
Ecclesia;
and
shall
\
visit,
repress,
redress,
record,
order,
correct,
restrain,
and
amend
all
such
\
have
and
enjoy,
annexed
and
united
to
the
imperial
crown
of
this
realm,
as
well
\
errors,
heresies,
abuses,
offenses,
contempts,
and
enormities,
whatsoever
they
\
the
title
and
style
thereof,
as
all
honors,
dignities,
preeminences,
jurisdictions,
\
be,
which
by
any
manner
of
spiritual
authority
or
jurisdiction
ought
or
may
lawfully
\
privileges,
authorities,
immunities,
profits,
and
commodities
to
the
said
dignity
\
be
reformed,
repressed,
ordered,
redressed,
corrected,
restrained,
or
amended,
\
of
the
supreme
head
of
the
same
Church
belonging
and
appertaining;
and
that
\
most
to
the
pleasure
of
Almighty
God,
the
increase
of
virtue
in
Christ's
religion,
\
our
said
sovereign
lord,
his
heirs
and
successors,
kings
of
this
realm,
shall
\
and
for
the
conservation
of
the
peace,
unity,
and
tranquility
of
this
realm;
any
\
have
full
power
and
authority
from
time
to
time
to
visit,
repress,
redress,
\
usage,
foreign
land,
foreign
authority,
prescription,
or
any
other
thing
or
things
\
record,
order,
correct,
restrain,
and
amend
all
such
errors,
heresies,
abuses,
\
to
the
contrary
hereof
notwithstanding.
\n
English
Parliament,
Act
of
Supremacy,
\
offenses,
contempts,
and
enormities,
whatsoever
they
be,
which
by
any
manner
\
1534
\n
From
the
passage,
one
may
infer
that
the
English
Parliament
wished
to
argue
\
of
spiritual
authority
or
jurisdiction
ought
or
may
lawfully
be
reformed,
repressed,
\
that
the
Act
of
Supremacy
would
\n
(A)
give
the
English
king
a
new
position
of
authority
\
ordered,
redressed,
corrected,
restrained,
or
amended,
most
to
the
pleasure
\
(B)
give
the
position
of
head
of
the
Church
of
England
to
Henry
VIII
alone
and
\
of
Almighty
God,
the
increase
of
virtue
in
Christ'
'
s
religion,
and
for
the
conservation
\
exclude
his
heirs
(C)
establish
Calvinism
as
the
one
true
theology
in
England
\
of
the
peace,
unity,
and
tranquility
of
this
realm;
any
usage,
foreign
land,
\
(D)
end
various
forms
of
corruption
plaguing
the
Church
in
England
\n
A:
Let's
think
\
foreign
authority,
prescription,
or
any
other
thing
or
things
to
the
contrary
\
step
by
step.
We
refer
to
Wikipedia
articles
on
european
history
for
help.
The
\
hereof
notwithstanding.
\
Act
of
Supremacy
states
that
it
grants
authority
to
the
king
\"
to
repress
and
\
\
extirpate
all
errors,
heresies,
and
other
enormities
and
abuses
\"
,
referring
to
\
English
Parliament,
Act
of
Supremacy,
1534
\
the
corruption
in
the
Church
of
England.
The
answer
is
(D).
\n\n
Q:
This
question
\
\
refers
to
the
following
information.
\n
Read
the
following
excerpt.
\n
The
revolutionary
\
From
the
passage,
one
may
infer
that
the
English
Parliament
wished
to
argue
\
seed
had
penetrated
into
every
country
and
spread
more
or
less.
It
was
greatly
\
that
the
Act
of
Supremacy
would
\
developed
under
the
régime
of
the
military
despotism
of
Bonaparte.
His
conquests
\
\
displaced
a
number
of
laws,
institutions,
and
customs;
broke
through
bonds
sacred
\
(A)
give
the
English
king
a
new
position
of
authority
(B)
give
the
position
\
among
all
nations,
strong
enough
to
resist
time
itself;
which
is
more
than
can
\
of
head
of
the
Church
of
England
to
Henry
VIII
alone
and
exclude
his
heirs
(C)
\
be
said
of
certain
benefits
conferred
by
these
innovators.
\n
The
monarchs
will
\
establish
Calvinism
as
the
one
true
theology
in
England
(D)
end
various
forms
\
fulfil
the
duties
imposed
upon
them
by
Him
who,
by
entrusting
them
with
power,
\
of
corruption
plaguing
the
Church
in
England'
\
has
charged
them
to
watch
over
the
maintenance
of
justice,
and
the
rights
of
all,
\
target
:
Let's think step by step. We refer to Wikipedia articles on european history
\
to
avoid
the
paths
of
error,
and
tread
firmly
in
the
way
of
truth.
Placed
beyond
\
for help. The Act of Supremacy states that it grants authority to the king "to
\
the
passions
which
agitate
society,
it
is
in
days
of
trial
chiefly
that
they
are
\
repress and extirpate all errors, heresies, and other enormities and abuses",
\
called
upon
to
despoil
realities
of
their
false
appearances,
and
to
show
themselves
\
referring to the corruption in the Church of England. The answer is (D).
\
as
they
are,
fathers
invested
with
the
authority
belonging
by
right
to
the
heads
\
-
question
:
"
This
question
refers
to
the
following
information.
\n
Read
the
following
\
\
of
families,
to
prove
that,
in
days
of
mourning,
they
know
how
to
be
just,
wise,
\
\
excerpt.
\n
The
revolutionary
seed
had
penetrated
into
every
country
and
spread
\
\
and
therefore
strong,
and
that
they
will
not
abandon
the
people
whom
they
ought
\
\
more
or
less.
It
was
greatly
developed
under
the
r
\xE9
gime
of
the
military
\
\
to
govern
to
be
the
sport
of
factions,
to
error
and
its
consequences,
which
must
\
\
despotism
of
Bonaparte.
His
conquests
displaced
a
number
of
laws,
institutions,
\
\
involve
the
loss
of
society.
\n
Union
between
the
monarchs
is
the
basis
of
the
policy
\
\
and
customs;
broke
through
bonds
sacred
among
all
nations,
strong
enough
to
\
\
which
must
now
be
followed
to
save
society
from
total
ruin.
.
.
.
\n
Let
them
not
\
\
resist
time
itself;
which
is
more
than
can
be
said
of
certain
benefits
conferred
\
\
confound
concessions
made
to
parties
with
the
good
they
ought
to
do
for
their
\
\
by
these
innovators.
\n
The
monarchs
will
fulfil
the
duties
imposed
upon
them
\
\
people,
in
modifying,
according
to
their
recognized
needs,
such
branches
of
the
\
\
by
Him
who,
by
entrusting
them
with
power,
has
charged
them
to
watch
over
\
\
administration
as
require
it.
\n
Let
them
be
just,
but
strong;
beneficent,
but
strict.
\n\
\
the
maintenance
of
justice,
and
the
rights
of
all,
to
avoid
the
paths
of
error,
\
Let
them
maintain
religious
principles
in
all
their
purity,
and
not
allow
the
faith
\
\
and
tread
firmly
in
the
way
of
truth.
Placed
beyond
the
passions
which
agitate
\
\
to
be
attacked
and
morality
interpreted
according
to
the
social
contract
or
the
\
\
society,
it
is
in
days
of
trial
chiefly
that
they
are
called
upon
to
despoil
\
\
visions
of
foolish
sectarians.
\n
Let
them
suppress
Secret
Societies;
that
gangrene
\
\
realities
of
their
false
appearances,
and
to
show
themselves
as
they
are,
\
\
of
society.
\n
—Klemens
von
Metternich,
Political
Confession
of
Faith,
1820
\n
Which
\
\
fathers
invested
with
the
authority
belonging
by
right
to
the
heads
of
families,
\
\
of
the
following
was
the
greatest
cause
of
the
fears
expressed
by
Metternich
in
\
\
to
prove
that,
in
days
of
mourning,
they
know
how
to
be
just,
wise,
and
therefore
\
\
the
document
above?
\n
(A)
The
ideas
of
personal
liberty
and
nationalism
conceived
\
\
strong,
and
that
they
will
not
abandon
the
people
whom
they
ought
to
govern
\
\
during
the
Enlightenment
resulted
in
radical
revolutions
that
could
spread
throughout
\
\
to
be
the
sport
of
factions,
to
error
and
its
consequences,
which
must
involve
\
\
Europe.
(B)
The
conquest
of
Europe
by
Napoleon
led
to
the
creation
of
new
factions
\
\
the
loss
of
society.
\n
Union
between
the
monarchs
is
the
basis
of
the
policy
\
\
and
shifted
the
European
balance
of
power.
(C)
The
power
of
monarchs
had
grown
\
\
which
must
now
be
followed
to
save
society
from
total
ruin.
.
.
.
\n
Let
them
\
\
to
the
point
where
it
needed
to
be
checked
by
other
powers
within
each
nation
\
\
not
confound
concessions
made
to
parties
with
the
good
they
ought
to
do
for
\
\
or
domination
of
civilians
would
occur.
(D)
The
rising
and
falling
economic
cycle
\
\
their
people,
in
modifying,
according
to
their
recognized
needs,
such
branches
\
\
of
the
newly
emerging
capitalist
economy
could
lead
to
civilian
unrest
that
must
\
\
of
the
administration
as
require
it.
\n
Let
them
be
just,
but
strong;
beneficent,
\
\
be
suppressed.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
\
\
but
strict.
\n
Let
them
maintain
religious
principles
in
all
their
purity,
and
\
\
european
history
for
help.
The
fears
of
revolution
in
early
19th
century
Europe
\
\
not
allow
the
faith
to
be
attacked
and
morality
interpreted
according
to
the
\
\
expressed
by
Klemens
von
Metternich,
a
conservative
Austrian
statesman,
were
a
\
\
social
contract
or
the
visions
of
foolish
sectarians.
\n
Let
them
suppress
Secret
\
\
direct
result
of
the
age
of
Enlightenment,
a
period
of
European
history
where
\
\
Societies;
that
gangrene
of
society.
\n\u2014
Klemens
von
Metternich,
Political
\
\
the
absolute
power
of
the
monarchy
was
challenged
with
ideas
of
individual
liberty
\
\
Confession
of
Faith,
1820
\n
Which
of
the
following
was
the
greatest
cause
of
\
\
and
nationalism,
leading
to
the
French
revolution
and
its
effects
all
over
Europe.
\
\
the
fears
expressed
by
Metternich
in
the
document
above?
\n
(A)
The
ideas
of
\
\
The
answer
is
(A).
\n\n
Q:
This
question
refers
to
the
following
information.
\n\
\
personal
liberty
and
nationalism
conceived
during
the
Enlightenment
resulted
\
The
excerpts
below
are
from
the
Navigation
Acts
of
1651.
\n
[A]fter
the
first
day
\
\
in
radical
revolutions
that
could
spread
throughout
Europe.
(B)
The
conquest
\
\
of
December,
one
thousand
six
hundred
fifty
and
one,
and
from
thence
forwards,
\
\
of
Europe
by
Napoleon
led
to
the
creation
of
new
factions
and
shifted
the
\
\
no
goods
or
commodities
whatsoever
of
the
growth,
production
or
manufacture
of
\
\
European
balance
of
power.
(C)
The
power
of
monarchs
had
grown
to
the
point
\
\
Asia,
Africa
or
America,
or
of
any
part
thereof;
or
of
any
islands
belonging
to
\
\
where
it
needed
to
be
checked
by
other
powers
within
each
nation
or
domination
\
\
them,
or
which
are
described
or
laid
down
in
the
usual
maps
or
cards
of
those
\
\
of
civilians
would
occur.
(D)
The
rising
and
falling
economic
cycle
of
the
\
\
places,
as
well
of
the
English
plantations
as
others,
shall
be
imported
or
brought
\
\
newly
emerging
capitalist
economy
could
lead
to
civilian
unrest
that
must
\
\
into
this
Commonwealth
of
England,
or
into
Ireland,
or
any
other
lands,
islands,
\
\
be
suppressed."
\
plantations,
or
territories
to
this
Commonwealth
belonging,
or
in
their
possession,
\
target
:
Let's think step by step. We refer to Wikipedia articles on european history
\
in
any
other
ship
or
ships,
vessel
or
vessels
whatsoever,
but
only
in
such
as
\
for help. The fears of revolution in early 19th century Europe expressed by
\
do
truly
and
without
fraud
belong
only
to
the
people
of
this
Commonwealth,
or
\
Klemens von Metternich, a conservative Austrian statesman, were a direct result
\
the
plantations
thereof,
as
the
proprietors
or
right
owners
thereof;
and
whereof
\
of the age of Enlightenment, a period of European history where the absolute
\
the
master
and
mariners
are
also
of
the
people
of
this
Commonwealth,
under
the
\
power of the monarchy was challenged with ideas of individual liberty and nationalism,
\
penalty
of
the
forfeiture
and
loss
of
all
the
goods
that
shall
be
imported
contrary
\
leading to the French revolution and its effects all over Europe. The answer
\
to
this
act,
,
,
,
\n
[N]o
goods
or
commodities
of
the
growth,
production,
or
manufacture
\
is (A).
\
of
Europe,
or
of
any
part
thereof,
shall
after
the
first
day
of
December,
one
\
-
question
:
'
This
question
refers
to
the
following
information.
\
thousand
six
hundred
fifty
and
one,
be
imported
or
brought
into
this
Commonwealth
\
\
of
England,
or
any
other
lands
or
territories
to
this
Commonwealth
belonging,
\
The
excerpts
below
are
from
the
Navigation
Acts
of
1651.
\
or
in
their
possession,
in
any
ship
or
ships,
vessel
or
vessels
whatsoever,
but
\
\
in
such
as
do
truly
and
without
fraud
belong
only
to
the
people
of
this
Commonwealth,
\
[A]fter
the
first
day
of
December,
one
thousand
six
hundred
fifty
and
one,
and
\
and
in
no
other,
except
only
such
foreign
ships
and
vessels
as
do
truly
and
properly
\
from
thence
forwards,
no
goods
or
commodities
whatsoever
of
the
growth,
production
\
belong
to
the
people
of
that
country
or
place,
of
which
the
said
goods
are
the
\
or
manufacture
of
Asia,
Africa
or
America,
or
of
any
part
thereof;
or
of
any
\
growth,
production
or
manufacture.
\n
Which
of
the
following
best
describes
the
\
islands
belonging
to
them,
or
which
are
described
or
laid
down
in
the
usual
\
outcome
of
the
Navigation
Acts
of
1651?
\n
(A)
They
served
as
a
catalyst
for
the
\
maps
or
cards
of
those
places,
as
well
of
the
English
plantations
as
others,
\
growth
of
English
shipping
and
overseas
trade,
but
did
little
to
limit
the
prospects
\
shall
be
imported
or
brought
into
this
Commonwealth
of
England,
or
into
Ireland,
\
of
the
Dutch
in
the
seventeenth
century.
(B)
They
brought
about
almost
immediate
\
or
any
other
lands,
islands,
plantations,
or
territories
to
this
Commonwealth
\
hardships
for
the
Dutch
economy
as
their
dominance
of
overseas
trade
quickly
ended.
\
belonging,
or
in
their
possession,
in
any
other
ship
or
ships,
vessel
or
vessels
\
(C)
They
were
rescinded
during
the
restoration
of
the
Stuarts
as
they
sought
normal
\
whatsoever,
but
only
in
such
as
do
truly
and
without
fraud
belong
only
to
the
\
diplomatic
relations
with
the
Dutch
so
not
as
to
need
Parliament's
financial
support
\
people
of
this
Commonwealth,
or
the
plantations
thereof,
as
the
proprietors
\
for
war.
(D)
They
led
to
nearly
a
century
of
recurrent
war
between
England
and
\
or
right
owners
thereof;
and
whereof
the
master
and
mariners
are
also
of
the
\
the
Netherlands,
which
would
not
end
until
after
American
independence.
\n
A:
Let's
\
people
of
this
Commonwealth,
under
the
penalty
of
the
forfeiture
and
loss
of
\
think
step
by
step.
We
refer
to
Wikipedia
articles
on
european
history
for
help.
\
all
the
goods
that
shall
be
imported
contrary
to
this
act,
,
,
,
\
The
Navigation
Acts
of
1651
helped
English
shipping
by
restricting
the
ability
\
\
of
ships
from
other
European
countries,
especially
the
Dutch,
to
transport
goods
\
[N]o
goods
or
commodities
of
the
growth,
production,
or
manufacture
of
Europe,
\
from
colonies
in
Asia
and
Africa
into
England.
The
answer
is
(A).
\n\n
Q:
This
question
\
or
of
any
part
thereof,
shall
after
the
first
day
of
December,
one
thousand
\
refers
to
the
following
information.
\n
In
Russia
there
was
nothing
going
on
well,
\
six
hundred
fifty
and
one,
be
imported
or
brought
into
this
Commonwealth
of
\
and
[Souvarine]
was
in
despair
over
the
news
he
had
received.
His
old
companions
\
England,
or
any
other
lands
or
territories
to
this
Commonwealth
belonging,
or
\
were
all
turning
to
the
politicians;
the
famous
Nihilists
who
made
Europe
tremble-sons
\
in
their
possession,
in
any
ship
or
ships,
vessel
or
vessels
whatsoever,
but
\
of
village
priests,
of
the
lower
middle
class,
of
tradesmen-could
not
rise
above
\
in
such
as
do
truly
and
without
fraud
belong
only
to
the
people
of
this
Commonwealth,
\
the
idea
of
national
liberation,
and
seemed
to
believe
that
the
world
would
be
\
and
in
no
other,
except
only
such
foreign
ships
and
vessels
as
do
truly
and
\
delivered-when
they
had
killed
their
despot&…
\n\"
Foolery!
They'll
never
get
out
\
properly
belong
to
the
people
of
that
country
or
place,
of
which
the
said
goods
\
of
it
with
their
foolery.
\"\n
Then,
lowering
his
voice
still
more,
in
a
few
bitter
\
are
the
growth,
production
or
manufacture.
\
words
he
described
his
old
dream
of
fraternity.
He
had
renounced
his
rank
and
\
\
his
fortune;
he
had
gone
among
workmen,
only
in
the
hope
of
seeing
at
last
the
\
Which
of
the
following
best
describes
the
outcome
of
the
Navigation
Acts
of
\
foundation
of
a
new
society
of
labour
in
common.
All
the
sous
in
his
pockets
had
\
1651?
\
long
gone
to
the
urchins
of
the
settlement;
he
had
been
as
tender
as
a
brother
\
\
with
the
colliers,
smiling
at
their
suspicion,
winning
them
over
by
his
quiet
\
(A)
They
served
as
a
catalyst
for
the
growth
of
English
shipping
and
overseas
\
workmanlike
ways
and
his
dislike
of
chattering.
But
decidedly
the
fusion
had
not
\
trade,
but
did
little
to
limit
the
prospects
of
the
Dutch
in
the
seventeenth
\
taken
place.
\n
His
voice
changed,
his
eyes
grew
bright,
he
fixed
them
on
étienne,
\
century.
(B)
They
brought
about
almost
immediate
hardships
for
the
Dutch
economy
\
directly
addressing
him:
\n\"
Now,
do
you
understand
that?
These
hatworkers
at
Marseilles
\
as
their
dominance
of
overseas
trade
quickly
ended.
(C)
They
were
rescinded
\
who
have
won
the
great
lottery
prize
of
a
hundred
thousand
francs
have
gone
off
\
during
the
restoration
of
the
Stuarts
as
they
sought
normal
diplomatic
relations
\
at
once
and
invested
it,
declaring
that
they
are
going
to
live
without
doing
anything!
\
with
the
Dutch
so
not
as
to
need
Parliament'
'
s
financial
support
for
war.
(D)
\
Yes,
that
is
your
idea,
all
of
you
French
workmen;
you
want
to
unearth
a
treasure
\
They
led
to
nearly
a
century
of
recurrent
war
between
England
and
the
Netherlands,
\
in
order
to
devour
it
alone
afterwards
in
some
lazy,
selfish
corner.
You
may
cry
\
which
would
not
end
until
after
American
independence.'
\
out
as
much
as
you
like
against
the
rich,
you
haven't
got
courage
enough
to
give
\
target
:
Let's think step by step. We refer to Wikipedia articles on european history
\
back
to
the
poor
the
money
that
luck
brings
you.
You
will
never
be
worthy
of
happiness
\
for help. The Navigation Acts of 1651 helped English shipping by restricting
\
as
long
as
you
own
anything,
and
your
hatred
of
the
bourgeois
proceeds
solely
\
the ability of ships from other European countries, especially the Dutch, to
\
from
an
angry
desire
to
be
bourgeois
yourselves
in
their
place.
\"\n
émile
Zola,
\
transport goods from colonies in Asia and Africa into England. The answer is
\
French
writer,
Germinal,
1885
\n
The
passage
displays
the
direct
concern
for
the
\
(A).
\
welfare
of
the
working
classes
that
was
typically
a
part
of
which
movement?
\n\
-
question
:
"
This
question
refers
to
the
following
information.
\n
In
Russia
there
was
\
(A)
Capitalist
(B)
Scientific
(C)
Communist
(D)
Existentialist
\n
A:
Let's
think
step
\
\
nothing
going
on
well,
and
[Souvarine]
was
in
despair
over
the
news
he
had
\
\
by
step.
We
refer
to
Wikipedia
articles
on
european
history
for
help.
The
modern
\
\
received.
His
old
companions
were
all
turning
to
the
politicians;
the
famous
\
\
Communist
movement
aims
to
establish
a
classless
society
based
on
communal
ownership
\
\
Nihilists
who
made
Europe
tremble-sons
of
village
priests,
of
the
lower
middle
\
\
and
distribution
of
property
and
means
of
production,
thereby
especially
benefiting
\
\
class,
of
tradesmen-could
not
rise
above
the
idea
of
national
liberation,
\
\
the
working
classes.
The
answer
is
(C).
\n\n
Q:
This
question
refers
to
the
following
\
\
and
seemed
to
believe
that
the
world
would
be
delivered-when
they
had
killed
\
\
information.
\n
The
following
excerpt
is
from
a
pamphlet.
\n
You
will
do
me
the
justice
\
\
their
despot&
\u2026\n\"
Foolery!
They'll
never
get
out
of
it
with
their
foolery.
\"\
\
to
remember,
that
I
have
always
strenuously
supported
the
Right
of
every
man
to
\
\n
Then,
lowering
his
voice
still
more,
in
a
few
bitter
words
he
described
his
\
\
his
own
opinion,
however
different
that
opinion
might
be
to
mine.
He
who
denies
\
\
old
dream
of
fraternity.
He
had
renounced
his
rank
and
his
fortune;
he
had
\
\
to
another
this
right,
makes
a
slave
of
himself
to
his
present
opinion,
because
\
\
gone
among
workmen,
only
in
the
hope
of
seeing
at
last
the
foundation
of
a
\
\
he
precludes
himself
the
right
of
changing
it.
\n
The
most
formidable
weapon
against
\
\
new
society
of
labour
in
common.
All
the
sous
in
his
pockets
had
long
gone
\
\
errors
of
every
kind
is
Reason.
I
have
never
used
any
other,
and
I
trust
I
never
\
\
to
the
urchins
of
the
settlement;
he
had
been
as
tender
as
a
brother
with
\
\
shall.
\n
The
circumstance
that
has
now
taken
place
in
France
of
the
total
abolition
\
\
the
colliers,
smiling
at
their
suspicion,
winning
them
over
by
his
quiet
workmanlike
\
\
of
the
whole
national
order
of
priesthood,
and
of
everything
appertaining
to
compulsive
\
\
ways
and
his
dislike
of
chattering.
But
decidedly
the
fusion
had
not
taken
\
\
systems
of
religion,
and
compulsive
articles
of
faith,
has
not
only
precipitated
\
\
place.
\n
His
voice
changed,
his
eyes
grew
bright,
he
fixed
them
on
\xE9
tienne,
\
\
my
intention,
but
rendered
a
work
of
this
kind
exceedingly
necessary,
lest
in
\
\
directly
addressing
him:
\n\"
Now,
do
you
understand
that?
These
hatworkers
\
\
the
general
wreck
of
superstition,
of
false
systems
of
government,
and
false
theology,
\
\
at
Marseilles
who
have
won
the
great
lottery
prize
of
a
hundred
thousand
francs
\
\
we
lose
sight
of
morality,
of
humanity,
and
of
the
theology
that
is
true.
\n
I
believe
\
\
have
gone
off
at
once
and
invested
it,
declaring
that
they
are
going
to
live
\
\
in
one
God,
and
no
more;
and
I
hope
for
happiness
beyond
this
life.
\n
I
believe
\
\
without
doing
anything!
Yes,
that
is
your
idea,
all
of
you
French
workmen;
\
\
in
the
equality
of
man;
and
I
believe
that
religious
duties
consist
in
doing
justice,
\
\
you
want
to
unearth
a
treasure
in
order
to
devour
it
alone
afterwards
in
some
\
\
loving
mercy,
and
endeavoring
to
make
our
fellow-creatures
happy.
\n
I
do
not
believe
\
\
lazy,
selfish
corner.
You
may
cry
out
as
much
as
you
like
against
the
rich,
\
\
in
the
creed
professed
by
the
Jewish
church,
by
the
Roman
church,
by
the
Greek
\
\
you
haven't
got
courage
enough
to
give
back
to
the
poor
the
money
that
luck
\
\
church,
by
the
Turkish
church,
by
the
Protestant
church,
nor
by
any
church
that
\
\
brings
you.
You
will
never
be
worthy
of
happiness
as
long
as
you
own
anything,
\
\
I
know
of.
My
own
mind
is
my
own
church.
\n
All
national
institutions
of
churches,
\
\
and
your
hatred
of
the
bourgeois
proceeds
solely
from
an
angry
desire
to
be
\
\
whether
Jewish,
Christian
or
Turkish,
appear
to
me
no
other
than
human
inventions,
\
\
bourgeois
yourselves
in
their
place.
\"\n\xE9
mile
Zola,
French
writer,
Germinal,
\
\
set
up
to
terrify
and
enslave
mankind,
and
monopolize
power
and
profit.
\n
I
do
\
\
1885
\n
The
passage
displays
the
direct
concern
for
the
welfare
of
the
working
\
\
not
mean
by
this
declaration
to
condemn
those
who
believe
otherwise;
they
have
\
\
classes
that
was
typically
a
part
of
which
movement?
\n
(A)
Capitalist
(B)
Scientific
\
\
the
same
right
to
their
belief
as
I
have
to
mine.
\n
—Thomas
Paine,
The
Age
of
Reason,
\
\
(C)
Communist
(D)
Existentialist"
\
1794–1795
\n
Which
of
the
following
Enlightenment
philosophes
designed
a
system
\
target
:
Let's think step by step. We refer to Wikipedia articles on european history
\
of
checks
and
balances
for
government
to
avoid
abuses
of
power?
\n
(A)
Jean
Jacques
\
for help. The modern Communist movement aims to establish a classless society
\
Rousseau
(B)
Baron
Montesquieu
(C)
Mary
Wollstonecraft
(D)
Adam
Smith
\n
A:
Let's
\
based on communal ownership and distribution of property and means of production,
\
think
step
by
step.
We
refer
to
Wikipedia
articles
on
european
history
for
help.
\
thereby especially benefiting the working classes. The answer is (C).
\
Baron
Montesquieu
was
a
18th
centrury
French
philsopher
who
wrote
extensively
\
-
question
:
"
This
question
refers
to
the
following
information.
\n
The
following
excerpt
\
\
against
the
monoplization
of
power
and
advocated
for
a
system
of
checks
and
balances
\
\
is
from
a
pamphlet.
\n
You
will
do
me
the
justice
to
remember,
that
I
have
always
\
\
in
government
to
prevent
the
rise
of
despotism.
The
answer
is
(B).
\n\n
"
\
strenuously
supported
the
Right
of
every
man
to
his
own
opinion,
however
different
\
"
group"
:
"
mmlu_flan_cot_fewshot_humanities"
\
that
opinion
might
be
to
mine.
He
who
denies
to
another
this
right,
makes
\
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
\
a
slave
of
himself
to
his
present
opinion,
because
he
precludes
himself
the
\
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_european_history"
\
right
of
changing
it.
\n
The
most
formidable
weapon
against
errors
of
every
\
\
kind
is
Reason.
I
have
never
used
any
other,
and
I
trust
I
never
shall.
\n\
The
circumstance
that
has
now
taken
place
in
France
of
the
total
abolition
of
\
\
the
whole
national
order
of
priesthood,
and
of
everything
appertaining
to
\
\
compulsive
systems
of
religion,
and
compulsive
articles
of
faith,
has
not
\
\
only
precipitated
my
intention,
but
rendered
a
work
of
this
kind
exceedingly
\
\
necessary,
lest
in
the
general
wreck
of
superstition,
of
false
systems
of
\
\
government,
and
false
theology,
we
lose
sight
of
morality,
of
humanity,
and
\
\
of
the
theology
that
is
true.
\n
I
believe
in
one
God,
and
no
more;
and
I
hope
\
\
for
happiness
beyond
this
life.
\n
I
believe
in
the
equality
of
man;
and
I
believe
\
\
that
religious
duties
consist
in
doing
justice,
loving
mercy,
and
endeavoring
\
\
to
make
our
fellow-creatures
happy.
\n
I
do
not
believe
in
the
creed
professed
\
\
by
the
Jewish
church,
by
the
Roman
church,
by
the
Greek
church,
by
the
Turkish
\
\
church,
by
the
Protestant
church,
nor
by
any
church
that
I
know
of.
My
own
\
\
mind
is
my
own
church.
\n
All
national
institutions
of
churches,
whether
Jewish,
\
\
Christian
or
Turkish,
appear
to
me
no
other
than
human
inventions,
set
up
\
\
to
terrify
and
enslave
mankind,
and
monopolize
power
and
profit.
\n
I
do
not
\
\
mean
by
this
declaration
to
condemn
those
who
believe
otherwise;
they
have
\
\
the
same
right
to
their
belief
as
I
have
to
mine.
\n\u2014
Thomas
Paine,
The
\
\
Age
of
Reason,
1794
\u2013
1795
\n
Which
of
the
following
Enlightenment
philosophes
\
\
designed
a
system
of
checks
and
balances
for
government
to
avoid
abuses
of
\
\
power?
\n
(A)
Jean
Jacques
Rousseau
(B)
Baron
Montesquieu
(C)
Mary
Wollstonecraft
\
\
(D)
Adam
Smith"
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
european
history
for
help.
Baron
Montesquieu
was
a
18th
centrury
French
philsopher
who
wrote
extensively
against
the
monoplization
of
power
and
advocated
for
a
system
of
checks
and
balances
in
government
to
prevent
the
rise
of
despotism.
The
answer
is
(B).'
group
:
mmlu_flan_cot_fewshot_humanities
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_european_history
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_geography"
dataset_name
:
high_school_geography
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
geography.
\n\n
Q:
Which
one
of
the
following
items
is
an
example
of
nonmaterial
\
school geography.
\
culture?
\n
(A)
Dove
soap
(B)
Dove
candy
bar
(C)
Dove
symbol
(D)
A
dove
(bird).
\n\
fewshot_config
:
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
geography
for
help.
\
sampler
:
first_n
\
Nonmaterial
culture
consists
of
cultural
ideas,
beliefs
or
symbols
that
are
not
\
samples
:
\
physical
objects.
The
answer
is
(C).
\n\n
Q:
During
the
third
stage
of
the
demographic
\
-
question
:
'
Which
one
of
the
following
items
is
an
example
of
nonmaterial
culture?
\
transition
model,
which
of
the
following
is
true?
\n
(A)
Birth
rates
increase
and
\
\
population
growth
rate
is
less
rapid.
(B)
Birth
rates
decline
and
population
growth
\
(A)
Dove
soap
(B)
Dove
candy
bar
(C)
Dove
symbol
(D)
A
dove
(bird).'
\
rate
is
less
rapid.
(C)
Birth
rates
increase
and
population
growth
rate
increases.
\
target
:
Let's think step by step. We refer to Wikipedia articles on geography
\
(D)
Birth
rates
decrease
and
population
growth
rate
increases.
\n
A:
Let's
think
\
for help. Nonmaterial culture consists of cultural ideas, beliefs or symbols
\
step
by
step.
We
refer
to
Wikipedia
articles
on
geography
for
help.
The
demographic
\
that are not physical objects. The answer is (C).
\
transition
model
models
the
five
different
stages
of
population
growth
as
a
country
\
-
question
:
'
During
the
third
stage
of
the
demographic
transition
model,
which
of
the
\
goes
through
economic
development,
where
the
third
stage
refers
to
a
period
of
\
following
is
true?
\
declining
birth
rates
and
lower
population
growth.
The
answer
is
(B).
\n\n
Q:
The
\
\
practice
of
hiring
a
foreign
third-party
service
provider
to
run
an
operation
\
(A)
Birth
rates
increase
and
population
growth
rate
is
less
rapid.
(B)
Birth
\
is
called
\n
(A)
outsourcing.
(B)
offshoring.
(C)
maquiladoras.
(D)
locational
interdependence.
\n\
rates
decline
and
population
growth
rate
is
less
rapid.
(C)
Birth
rates
increase
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
geography
for
help.
\
and
population
growth
rate
increases.
(D)
Birth
rates
decrease
and
population
\ \"
Offshoring
\"
literally
means
to
move
or
base
some
of
the
activities
or
processes
\
growth
rate
increases.'
\
of
a
company
to
a
foreign
country.
The
answer
is
(B).
\n\n
Q:
Which
of
the
following
\
target
:
Let's think step by step. We refer to Wikipedia articles on geography
\
statements
is
NOT
accurate
regarding
the
services
provided
by
local
governments
\
for help. The demographic transition model models the five different stages
\
in
the
United
States?
\n
(A)
Duplication
of
efforts
occurs
often.
(B)
Social
problems
\
of population growth as a country goes through economic development, where the
\
of
the
central
city
spill
over
into
the
surrounding
residential
suburbs.
(C)
Inefficiency
\
third stage refers to a period of declining birth rates and lower population
\
in
providing
services
occurs
often.
(D)
One
neighborhood's
efforts
to
reduce
pollution
\
growth. The answer is (B).
\
are
always
supported
by
neighboring
communities.
\n
A:
Let's
think
step
by
step.
\
-
question
:
'
The
practice
of
hiring
a
foreign
third-party
service
provider
to
run
an
\
We
refer
to
Wikipedia
articles
on
geography
for
help.
There
may
be
economic,
social
\
operation
is
called
\
or
political
reasons
for
two
neighboring
communities
and
their
local
governments
\
\
not
agreeing
to
pollution
reduction
efforts
initiated
by
one
of
them.
The
answer
\
(A)
outsourcing.
(B)
offshoring.
(C)
maquiladoras.
(D)
locational
interdependence.'
\
is
(D).
\n\n
Q:
The
rate
of
natural
increase
of
a
population
is
found
by
subtracting
\
target
:
Let's think step by step. We refer to Wikipedia articles on geography
\
the
\n
(A)
crude
death
rate
from
the
crude
birth
date.
(B)
crude
birth
rate
from
\
for help. "Offshoring" literally means to move or base some of the activities
\
the
crude
death
rate.
(C)
doubling
time
from
the
crude
birth
rate.
(D)
fertility
\
or processes of a company to a foreign country. The answer is (B).
\
rate
from
the
crude
death
rate.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
-
question
:
'
Which
of
the
following
statements
is
NOT
accurate
regarding
the
services
\
articles
on
geography
for
help.
The
difference
between
number
of
births
and
deaths
\
provided
by
local
governments
in
the
United
States?
\
gives
the
population
increase
at
any
given
time.
The
answer
is
(A).
\n\n
"
"
group"
:
"
mmlu_flan_cot_fewshot_social_sciences"
(A)
Duplication
of
efforts
occurs
often.
(B)
Social
problems
of
the
central
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
city
spill
over
into
the
surrounding
residential
suburbs.
(C)
Inefficiency
in
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_geography"
providing
services
occurs
often.
(D)
One
neighborhood'
'
s
efforts
to
reduce
pollution
are
always
supported
by
neighboring
communities.'
target
:
Let's think step by step. We refer to Wikipedia articles on geography
for help. There may be economic, social or political reasons for two neighboring
communities and their local governments not agreeing to pollution reduction
efforts initiated by one of them. The answer is (D).
-
question
:
'
The
rate
of
natural
increase
of
a
population
is
found
by
subtracting
the
(A)
crude
death
rate
from
the
crude
birth
date.
(B)
crude
birth
rate
from
the
crude
death
rate.
(C)
doubling
time
from
the
crude
birth
rate.
(D)
fertility
rate
from
the
crude
death
rate.'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
geography
for
help.
The
difference
between
number
of
births
and
deaths
gives
the
population
increase
at
any
given
time.
The
answer
is
(A).'
group
:
mmlu_flan_cot_fewshot_social_sciences
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_geography
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_government_and_politics.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_government_and_politics"
dataset_name
:
high_school_government_and_politics
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
government
and
politics.
\n\n
Q:
Which
of
the
following
best
states
an
argument
\
school government and politics.
\
made
by
James
Madison
in
The
Federalist
number
10?
\n
(A)
Honest
politicians
can
\
fewshot_config
:
\
prevent
factions
from
developing.
(B)
Factions
are
more
likely
to
occur
in
large
\
sampler
:
first_n
\
republics
than
in
small
ones.
(C)
The
negative
effects
of
factionalism
can
be
\
samples
:
\
reduced
by
a
republican
government.
(D)
Free
elections
are
the
people's
best
defense
\
-
question
:
'
Which
of
the
following
best
states
an
argument
made
by
James
Madison
in
\
against
factionalism.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
\
The
Federalist
number
10?
\
on
government
and
politics
for
help.
In
the
Federalist
number
10,
James
Madison
\
\
advocated
for
a
representative
republican
form
of
government
to
guard
against
\
(A)
Honest
politicians
can
prevent
factions
from
developing.
(B)
Factions
are
\
factionalism.
The
answer
is
(C).
\n\n
Q:
The
term
\"
budget
deficit
\"
refers
to
the
\n\
more
likely
to
occur
in
large
republics
than
in
small
ones.
(C)
The
negative
(A)
annual
increase
in
federal
spending
on
the
military
(B)
amount
of
interest
on
\
effects
of
factionalism
can
be
reduced
by
a
republican
government.
(D)
Free
\
the
national
debt
(C)
difference
between
the
initial
budget
proposals
made
by
\
elections
are
the
people'
'
s
best
defense
against
factionalism.'
\
the
president
and
Congress
(D)
amount
the
government
spends
in
excess
of
its
revenues
\n\
target
:
Let's think step by step. We refer to Wikipedia articles on government
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
government
and
politics
\
and politics for help. In the Federalist number 10, James Madison advocated
\
for
help.
When
the
goverment
spends
more
than
it
earns,
their
difference
is
the
\
for a representative republican form of government to guard against factionalism.
\
budget
deficit.
The
answer
is
(D).
\n\n
Q:
Which
of
the
following
statements
about
\
The answer is (C).
\
cabinet
departments
is
FALSE?
\n
(A)
They
are
established
by
the
legislative
branch.
\
-
question
:
'
The
term
"budget
deficit"
refers
to
the
\
(B)
Their
members
often
don't
have
much
influence
over
presidential
decisions.
\
\
(C)
They
cannot
all
be
run
by
leaders
who
belong
to
the
same
political
party
the
\
(A)
annual
increase
in
federal
spending
on
the
military
(B)
amount
of
interest
\
president
does.
(D)
Not
every
federal
agency
is
a
cabinet
department.
\n
A:
Let's
\
on
the
national
debt
(C)
difference
between
the
initial
budget
proposals
made
\
think
step
by
step.
We
refer
to
Wikipedia
articles
on
government
and
politics
\
by
the
president
and
Congress
(D)
amount
the
government
spends
in
excess
of
\
for
help.
There
is
no
law
stipulating
that
some
cabinet
department
leaders
have
\
its
revenues'
\
to
belong
to
a
political
party
different
from
that
of
the
president.
The
answer
\
target
:
Let's think step by step. We refer to Wikipedia articles on government
\
is
(C).
\n\n
Q:
Which
of
the
following
cases
established
the
precedent
that
a
defendant
\
and politics for help. When the goverment spends more than it earns, their difference
\
must
be
informed
of
the
right
to
remain
silent,
the
right
to
a
lawyer,
and
protection
\
is the budget deficit. The answer is (D).
\
from
self-incrimination?
\n
(A)
Weeks
v.
United
States
(B)
Betts
v.
Brady
(C)
Mapp
\
-
question
:
'
Which
of
the
following
statements
about
cabinet
departments
is
FALSE?
\
v.
Ohio
(D)
Miranda
v.
Arizona
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
\
articles
on
government
and
politics
for
help.
In
the
landmark
Miranda
v.
Arizona
\
(A)
They
are
established
by
the
legislative
branch.
(B)
Their
members
often
\
in
1966,
the
US
Supreme
Court,
based
on
the
Fifth
and
Sixth
Amendment
of
the
US
\
don'
'
t
have
much
influence
over
presidential
decisions.
(C)
They
cannot
all
\
Constitution,
guaranteed
a
defendant's
right
to
an
attorney
and
protection
from
\
be
run
by
leaders
who
belong
to
the
same
political
party
the
president
does.
\
self-incrimination.
The
answer
is
(D).
\n\n
Q:
Uncertainty
over
the
limits
to
presidential
\
(D)
Not
every
federal
agency
is
a
cabinet
department.'
\
power
is
caused
primarily
by
the
fact
that
\n
(A)
the
constitutional
definition
\
target
:
Let's think step by step. We refer to Wikipedia articles on government
\
of
those
powers
is
broad
and
unspecific
(B)
most
people
agree
that
the
Constitution
\
and politics for help. There is no law stipulating that some cabinet department
\
places
too
many
limits
on
presidential
power
(C)
the
Supreme
Court
consistently
\
leaders have to belong to a political party different from that of the president.
\
refuses
to
rule
on
cases
concerning
presidential
powers
(D)
constitutional
amendments
\
The answer is (C).
\
have
greatly
increased
presidential
powers
\n
A:
Let's
think
step
by
step.
We
refer
\
-
question
:
'
Which
of
the
following
cases
established
the
precedent
that
a
defendant
\
to
Wikipedia
articles
on
government
and
politics
for
help.
The
US
Constitution
\
must
be
informed
of
the
right
to
remain
silent,
the
right
to
a
lawyer,
and
protection
\
is
not
very
specific
about
the
powers
of
the
president,
leading
to
uncertainty
\
from
self-incrimination?
\
over
its
limits.
The
answer
is
(A).
\n\n
"
"
group"
:
"
mmlu_flan_cot_fewshot_social_sciences"
(A)
Weeks
v.
United
States
(B)
Betts
v.
Brady
(C)
Mapp
v.
Ohio
(D)
Miranda
v.
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
Arizona'
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_government_and_politics"
target
:
Let's think step by step. We refer to Wikipedia articles on government
and politics for help. In the landmark Miranda v. Arizona in 1966, the US Supreme
Court, based on the Fifth and Sixth Amendment of the US Constitution, guaranteed
a defendant's right to an attorney and protection from self-incrimination. The
answer is (D).
-
question
:
'
Uncertainty
over
the
limits
to
presidential
power
is
caused
primarily
by
the
fact
that
(A)
the
constitutional
definition
of
those
powers
is
broad
and
unspecific
(B)
most
people
agree
that
the
Constitution
places
too
many
limits
on
presidential
power
(C)
the
Supreme
Court
consistently
refuses
to
rule
on
cases
concerning
presidential
powers
(D)
constitutional
amendments
have
greatly
increased
presidential
powers'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
government
and
politics
for
help.
The
US
Constitution
is
not
very
specific
about
the
powers
of
the
president,
leading
to
uncertainty
over
its
limits.
The
answer
is
(A).'
group
:
mmlu_flan_cot_fewshot_social_sciences
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_government_and_politics
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_macroeconomics.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_macroeconomics"
dataset_name
:
high_school_macroeconomics
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
macroeconomics.
\n\n
Q:
Which
of
the
following
policies
best
describes
supply-side
\
school macroeconomics.
\
fiscal
policy?
\n
(A)
An
increase
in
the
money
supply
(B)
Increased
government
spending
\
fewshot_config
:
\
(C)
Lower
taxes
on
research
and
development
of
new
technology
(D)
Higher
taxes
\
sampler
:
first_n
\
on
household
income
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
\
samples
:
\
on
macroeconomics
for
help.
Supply-side
fiscal
policy
stimulates
the
economy
by
\
-
question
:
'
Which
of
the
following
policies
best
describes
supply-side
fiscal
policy?
\
encouraging
more
production
of
goods
and
services
through
reduction
in
taxes
and
\
\
deregulation.
The
answer
is
(C).
\n\n
Q:
The
short-run
Phillips
curve
indicates
\
(A)
An
increase
in
the
money
supply
(B)
Increased
government
spending
(C)
Lower
\
a
\n
(A)
direct
relation
between
unemployment
and
inflation
(B)
direct
relation
\
taxes
on
research
and
development
of
new
technology
(D)
Higher
taxes
on
household
\
between
price
and
quantity
demanded
(C)
inverse
relation
between
price
and
quantity
\
income'
\
demanded
(D)
inverse
relation
between
unemployment
and
inflation
\n
A:
Let's
think
\
target
:
Let's think step by step. We refer to Wikipedia articles on macroeconomics
\
step
by
step.
We
refer
to
Wikipedia
articles
on
macroeconomics
for
help.
The
short-run
\
for help. Supply-side fiscal policy stimulates the economy by encouraging more
\
Phillips
curve
shows
that
whenever
unemployment
decreases
below
a
natural
level,
\
production of goods and services through reduction in taxes and deregulation.
\
the
inflation
starts
increasing,
and
vice-versa.
The
answer
is
(D).
\n\n
Q:
Holding
\
The answer is (C).
\
all
else
equal
which
of
the
following
monetary
policies
would
be
used
to
boost
\
-
question
:
'
The
short-run
Phillips
curve
indicates
a
\
U.S.
exports?
\n
(A)
Increasing
the
discount
rate
(B)
Increasing
the
reserve
ratio
\
\
(C)
Buying
government
securities
(D)
Lowering
tariffs
\n
A:
Let's
think
step
by
\
(A)
direct
relation
between
unemployment
and
inflation
(B)
direct
relation
between
\
step.
We
refer
to
Wikipedia
articles
on
macroeconomics
for
help.
Buying
government
\
price
and
quantity
demanded
(C)
inverse
relation
between
price
and
quantity
\
securities
leads
to
reduction
in
demand
for
US
dollars
from
foreign
buyers,
thereby
\
demanded
(D)
inverse
relation
between
unemployment
and
inflation'
\
making
it
cheaper
and
hence
making
US
exports
more
attractive.
The
answer
is
(C).
\n\
target
:
Let's think step by step. We refer to Wikipedia articles on macroeconomics
\n
Q:
A
federal
deficit
occurs
when
\n
(A)
exports
exceed
imports.
(B)
imports
exceed
\
for help. The short-run Phillips curve shows that whenever unemployment decreases
\
exports.
(C)
federal
tax
collections
exceed
spending.
(D)
federal
spending
exceeds
\
below a natural level, the inflation starts increasing, and vice-versa. The
\
federal
tax
revenues.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
\
answer is (D).
\
on
macroeconomics
for
help.
A
federal
deficit
occurs
when
federal
spending
exceeds
\
-
question
:
'
Holding
all
else
equal
which
of
the
following
monetary
policies
would
\
federal
income
which
is
primarily
from
tax
revenues.
The
answer
is
(D).
\n\n
Q:
\
be
used
to
boost
U.S.
exports?
\
Which
of
the
following
is
not
included
in
the
U.S.
GDP?
\n
(A)
The
U.S.
military
\
\
opens
a
new
base
in
a
foreign
country
with
1000
U.S.
personnel.
(B)
Japanese
consumers
\
(A)
Increasing
the
discount
rate
(B)
Increasing
the
reserve
ratio
(C)
Buying
\
buy
thousands
of
CDs
produced
in
the
United
States.
(C)
An
American
pop
singer
\
government
securities
(D)
Lowering
tariffs'
\
performs
a
sold-out
concert
in
Paris.
(D)
A
French
theatrical
production
tours
\
target
:
Let's think step by step. We refer to Wikipedia articles on macroeconomics
\
dozens
of
American
cities.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
for help. Buying government securities leads to reduction in demand for US dollars
\
articles
on
macroeconomics
for
help.
The
economic
transactions
related
to
the
\
from foreign buyers, thereby making it cheaper and hence making US exports more
\
performance
of
the
American
pop-singer
in
Paris
happens
entirely
outside
the
U.S.
\
attractive. The answer is (C).
\
and
hence
is
not
included
in
the
GDP
numbers.
The
answer
is
(C).
\n\n
"
-
question
:
'
A
federal
deficit
occurs
when
"
group"
:
"
mmlu_flan_cot_fewshot_social_sciences"
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
(A)
exports
exceed
imports.
(B)
imports
exceed
exports.
(C)
federal
tax
collections
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_macroeconomics"
exceed
spending.
(D)
federal
spending
exceeds
federal
tax
revenues.'
target
:
Let's think step by step. We refer to Wikipedia articles on macroeconomics
for help. A federal deficit occurs when federal spending exceeds federal income
which is primarily from tax revenues. The answer is (D).
-
question
:
'
Which
of
the
following
is
not
included
in
the
U.S.
GDP?
(A)
The
U.S.
military
opens
a
new
base
in
a
foreign
country
with
1000
U.S.
personnel.
(B)
Japanese
consumers
buy
thousands
of
CDs
produced
in
the
United
States.
(C)
An
American
pop
singer
performs
a
sold-out
concert
in
Paris.
(D)
A
French
theatrical
production
tours
dozens
of
American
cities.'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
macroeconomics
for
help.
The
economic
transactions
related
to
the
performance
of
the
American
pop-singer
in
Paris
happens
entirely
outside
the
U.S.
and
hence
is
not
included
in
the
GDP
numbers.
The
answer
is
(C).'
group
:
mmlu_flan_cot_fewshot_social_sciences
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_macroeconomics
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_mathematics.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_mathematics"
dataset_name
:
high_school_mathematics
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
mathematics.
\n\n
Q:
Simplify
and
write
the
result
with
a
rational
denominator:
\
school mathematics.
\
$$
\\
sqrt{
\\
sqrt[3]{
\\
sqrt{
\\
frac{1}{729}}}}$$
\n
(A)
\\
frac{3
\\
sqrt{3}}{3}
(B)
\\\
fewshot_config
:
frac{1}{3}
(C)
\\
sqrt{3}
(D)
\\
frac{
\\
sqrt{3}}{3}
\n
A:
Let's
think
step
by
step.
\
sampler
:
first_n
\
Factoring
$729=3^6$
and
combining
the
roots
$
\\
frac{1}{2}
\\
frac{1}{3}
\\
frac{1}{2}=
\\\
samples
:
frac{1}{12}$,
we
get
that
$
\\
sqrt{
\\
sqrt[3]{
\\
sqrt{
\\
frac{1}{729}}}}=
\\
left(
\\
frac{1}{3^6}
\\\
-
question
:
'
Simplify
and
write
the
result
with
a
rational
denominator:
$$\sqrt{\sqrt[3]{\sqrt{\frac{1}{729}}}}$$
right)^{
\\
frac{1}{12}}=
\\
frac{1}{3^{
\\
frac{1}{2}}}=
\\
frac{3}{
\\
sqrt{3}}$
The
answer
\
\
is
(D).
\n\n
Q:
Five
thousand
dollars
compounded
annually
at
an
$x
\\
%$
interest
\
(A)
\frac{3\sqrt{3}}{3}
(B)
\frac{1}{3}
(C)
\sqrt{3}
(D)
\frac{\sqrt{3}}{3}'
\
rate
takes
six
years
to
double.
At
the
same
interest
rate,
how
many
years
will
\
target
:
Let's think step by step. Factoring $729=3^6$ and combining the roots
\
it
take
$
\\
$300$
to
grow
to
$
\\
$9600$?
\n
(A)
12
(B)
1
(C)
30
(D)
5
\n
A:
Let's
think
\
$\frac{1}{2}\frac{1}{3}\frac{1}{2}=\frac{1}{12}$, we get that $\sqrt{\sqrt[3]{\sqrt{\frac{1}{729}}}}=\left(\frac{1}{3^6}\right)^{\frac{1}{12}}=\frac{1}{3^{\frac{1}{2}}}=\frac{3}{\sqrt{3}}$
\
step
by
step.
To
go
from
$
\\
$300$
to
$
\\
$9600$,
the
value
must
go
up
by
a
factor
\
The answer is (D).
\
of
$9600/300=32=2^5$.
Since
at
this
interest
rate
it
takes
six
years
for
it
to
\
-
question
:
'
Five
thousand
dollars
compounded
annually
at
an
$x\%$
interest
rate
takes
\
double,
it
will
take
$5*6=30$
years
to
grow
to
$
\\
$9600$.
The
answer
is
(C).
\n\
six
years
to
double.
At
the
same
interest
rate,
how
many
years
will
it
take
\n
Q:
Ten
students
take
a
biology
test
and
receive
the
following
scores:
45,
55,
\
$\$300$
to
grow
to
$\$9600$?
\
50,
70,
65,
80,
40,
90,
70,
85.
What
is
the
mean
of
the
students’
test
scores?
\n\
(A)
55
(B)
60
(C)
62
(D)
65
\n
A:
Let's
think
step
by
step.
There
are
10
students
\
(A)
12
(B)
1
(C)
30
(D)
5'
\
and
the
sum
of
their
scores
is
$45
+
55
+
50
+
70
+
65
+
80
+
40
+
90
+
70
+
85
\
target
:
Let's think step by step. To go from $\$300$ to $\$9600$, the value must
\
=
650$,
the
mean
is
$650/10=65$.
The
answer
is
(D).
\n\n
Q:
The
variable
$x$
varies
\
go up by a factor of $9600/300=32=2^5$. Since at this interest rate it takes
\
directly
as
the
square
of
$y$,
and
$y$
varies
directly
as
the
cube
of
$z$.
If
\
six years for it to double, it will take $5*6=30$ years to grow to $\$9600$.
\
$x$
equals
$-16$
when
$z$
equals
2,
what
is
the
value
of
$x$
when
$z$
equals
$
\\\
The answer is (C).
frac{1}{2}$?
\n
(A)
-1
(B)
16
(C)
-
\\
frac{1}{256}
(D)
\\
frac{1}{16}
\n
A:
Let's
think
\
-
question
:
"
Ten
students
take
a
biology
test
and
receive
the
following
scores:
45,
\
\
step
by
step.
We
know
that
$x
\\
propto
y^2$
and
$y
\\
propto
z^3$,
so
$x
=
k
z^6$
\
\
55,
50,
70,
65,
80,
40,
90,
70,
85.
What
is
the
mean
of
the
students
\u2019\
\
for
some
constant
$k$.
Plugging
in
for
$x=-16$
and
$z=2$,
the
constant
value
is
\
\
test
scores?
\n
(A)
55
(B)
60
(C)
62
(D)
65"
\
$k=
\\
frac{x}{z^6}=
\\
frac{-16}{64}=-
\\
frac{1}{4}$.
So,
when
$z=
\\
frac{1}{2}$,
the
\
target
:
Let's think step by step. There are 10 students and the sum of their scores
\
value
of
$x$
is
$x=kz^6=-
\\
frac{1}{4}
\\
frac{1}{2^6}=-
\\
frac{1}{256}$.
The
answer
\
is $45 + 55 + 50 + 70 + 65 + 80 + 40 + 90 + 70 + 85 = 650$, the mean is $650/10=65$.
\
is
(C).
\n\n
Q:
Joe
was
in
charge
of
lights
for
a
dance.
The
red
light
blinks
every
\
The answer is (D).
\
two
seconds,
the
yellow
light
every
three
seconds,
and
the
blue
light
every
five
\
-
question
:
'
The
variable
$x$
varies
directly
as
the
square
of
$y$,
and
$y$
varies
\
seconds.
If
we
include
the
very
beginning
and
very
end
of
the
dance,
how
many
\
directly
as
the
cube
of
$z$.
If
$x$
equals
$-16$
when
$z$
equals
2,
what
is
\
times
during
a
seven
minute
dance
will
all
the
lights
come
on
at
the
same
time?
\
the
value
of
$x$
when
$z$
equals
$\frac{1}{2}$?
\
(Assume
that
all
three
lights
blink
simultaneously
at
the
very
beginning
of
the
\
\
dance.)
\n
(A)
3
(B)
15
(C)
6
(D)
5
\n
A:
Let's
think
step
by
step.
The
least
common
\
(A)
-1
(B)
16
(C)
-\frac{1}{256}
(D)
\frac{1}{16}'
\
multiple
of
2,
3
and
5
is
30,
so
during
a
7
minute
dance,
all
the
three
lights
\
target
:
Let's think step by step. We know that $x \propto y^2$ and $y \propto
\
will
come
on
at
the
same
time
$2*7+1=15$
times.
The
answer
is
(B).
\n\n
"
z^3$, so $x = k z^6$ for some constant $k$. Plugging in for $x=-16$ and $z=2$,
"
group"
:
"
mmlu_flan_cot_fewshot_stem"
the constant value is $k=\frac{x}{z^6}=\frac{-16}{64}=-\frac{1}{4}$. So, when
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
$z=\frac{1}{2}$, the value of $x$ is $x=kz^6=-\frac{1}{4}\frac{1}{2^6}=-\frac{1}{256}$.
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_mathematics"
The answer is (C).
-
question
:
'
Joe
was
in
charge
of
lights
for
a
dance.
The
red
light
blinks
every
two
seconds,
the
yellow
light
every
three
seconds,
and
the
blue
light
every
five
seconds.
If
we
include
the
very
beginning
and
very
end
of
the
dance,
how
many
times
during
a
seven
minute
dance
will
all
the
lights
come
on
at
the
same
time?
(Assume
that
all
three
lights
blink
simultaneously
at
the
very
beginning
of
the
dance.)
(A)
3
(B)
15
(C)
6
(D)
5'
target
:
'
Let'
'
s
think
step
by
step.
The
least
common
multiple
of
2,
3
and
5
is
30,
so
during
a
7
minute
dance,
all
the
three
lights
will
come
on
at
the
same
time
$2*7+1=15$
times.
The
answer
is
(B).'
group
:
mmlu_flan_cot_fewshot_stem
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_mathematics
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_microeconomics.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_microeconomics"
dataset_name
:
high_school_microeconomics
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
microeconomics.
\n\n
Q:
Which
of
the
following
is
necessarily
a
characteristic
\
school microeconomics.
\
of
oligopoly?
\n
(A)
Free
entry
into
and
exit
from
the
market
(B)
A
few
large
producers
\
fewshot_config
:
\
(C)
One
producer
of
a
good
with
no
close
substitutes
(D)
A
homogenous
product
\n\
sampler
:
first_n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
microeconomics
for
\
samples
:
\
help.
An
oligopoly
is
when
a
market
is
dominated
by
just
one
or
a
few
number
of
\
-
question
:
'
Which
of
the
following
is
necessarily
a
characteristic
of
oligopoly?
\
sellers
or
producers.
To
get
oligopoly,
the
market
should
have
high
barriers
to
\
\
new
entry,
and
the
product
has
differentiation.
The
answer
is
(B).
\n\n
Q:
If
the
\
(A)
Free
entry
into
and
exit
from
the
market
(B)
A
few
large
producers
(C)
One
\
government
subsidizes
producers
in
a
perfectly
competitive
market,
then
\n
(A)
the
\
producer
of
a
good
with
no
close
substitutes
(D)
A
homogenous
product'
\
demand
for
the
product
will
increase
(B)
the
demand
for
the
product
will
decrease
\
target
:
Let's think step by step. We refer to Wikipedia articles on microeconomics
\
(C)
the
consumer
surplus
will
increase
(D)
the
consumer
surplus
will
decrease
\n\
for help. An oligopoly is when a market is dominated by just one or a few number
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
microeconomics
for
\
of sellers or producers. To get oligopoly, the market should have high barriers
\
help.
(A)
and
(B)
are
wrong
because
the
demand
curve
does
not
change
at
all.
If
\
to new entry, and the product has differentiation. The answer is (B).
\
the
government
subsidizes
producers,
the
supply
will
increase,
and
thus
the
consumer
\
-
question
:
'
If
the
government
subsidizes
producers
in
a
perfectly
competitive
market,
\
surplus
also
increases.
The
answer
is
(C).
\n\n
Q:
Which
of
the
following
is
true
\
then
\
of
a
price
floor?
\n
(A)
The
price
floor
shifts
the
demand
curve
to
the
left.
(B)
\
\
An
effective
floor
creates
a
shortage
of
the
good.
(C)
The
price
floor
shifts
\
(A)
the
demand
for
the
product
will
increase
(B)
the
demand
for
the
product
\
the
supply
curve
of
the
good
to
the
right.
(D)
To
be
an
effective
floor,
it
must
\
will
decrease
(C)
the
consumer
surplus
will
increase
(D)
the
consumer
surplus
\
be
set
above
the
equilibrium
price.
\n
A:
Let's
think
step
by
step.
We
refer
to
\
will
decrease'
\
Wikipedia
articles
on
microeconomics
for
help.
Price
floor
does
not
shift
the
\
target
:
Let's think step by step. We refer to Wikipedia articles on microeconomics
\
demand
or
shift
curve.
An
effective
price
floor
should
be
set
above
the
equilibrium
\
for help. (A) and (B) are wrong because the demand curve does not change at
\
price,
otherwise
the
market
bears
and
the
floor
does
not
have
effective
effect.
\
all. If the government subsidizes producers, the supply will increase, and thus
\
The
answer
is
(D).
\n\n
Q:
The
concentration
ratio
for
a
monopoly
is
\n
(A)
0
(B)
\
the consumer surplus also increases. The answer is (C).
\
5
(C)
10
(D)
100
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
\
-
question
:
'
Which
of
the
following
is
true
of
a
price
floor?
\
on
microeconomics
for
help.
The
concentration
ratio
is
calculated
as
the
sum
of
\
\
market
share
of
a
specific
number
of
largest
companies.
Monopoly
means
one
company
\
(A)
The
price
floor
shifts
the
demand
curve
to
the
left.
(B)
An
effective
floor
\
or
entity
controls
the
entire
market,
therefore,
the
concentration
ratio
is
100
\
creates
a
shortage
of
the
good.
(C)
The
price
floor
shifts
the
supply
curve
\
percent.
The
answer
is
(D).
\n\n
Q:
In
a
competitive
labor
market
for
housepainters,
\
of
the
good
to
the
right.
(D)
To
be
an
effective
floor,
it
must
be
set
above
\
which
of
the
following
would
increase
the
demand
for
housepainters?
\n
(A)
An
effective
\
the
equilibrium
price.'
\
minimum
wage
imposed
on
this
labor
market.
(B)
An
increase
in
the
price
of
gallons
\
target
:
Let's think step by step. We refer to Wikipedia articles on microeconomics
\
of
paint.
(C)
An
increase
in
the
construction
of
new
houses.
(D)
An
increase
in
\
for help. Price floor does not shift the demand or shift curve. An effective
\
the
price
of
mechanical
painters
so
long
as
the
output
effect
exceeds
the
substitution
\
price floor should be set above the equilibrium price, otherwise the market
\
effect.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
microeconomics
\
bears and the floor does not have effective effect. The answer is (D).
\
for
help.
An
increase
in
the
construction
of
new
houses
means
an
increase
demand
\
-
question
:
'
The
concentration
ratio
for
a
monopoly
is
\
of
in-house
painting,
thus
increases
the
demand
for
housepainters.
The
answer
\
\
is
(C).
\n\n
"
(A)
0
(B)
5
(C)
10
(D)
100'
"
group"
:
"
mmlu_flan_cot_fewshot_social_sciences"
target
:
Let's think step by step. We refer to Wikipedia articles on microeconomics
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
for help. The concentration ratio is calculated as the sum of market share of
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_microeconomics"
a specific number of largest companies. Monopoly means one company or entity
controls the entire market, therefore, the concentration ratio is 100 percent.
The answer is (D).
-
question
:
'
In
a
competitive
labor
market
for
housepainters,
which
of
the
following
would
increase
the
demand
for
housepainters?
(A)
An
effective
minimum
wage
imposed
on
this
labor
market.
(B)
An
increase
in
the
price
of
gallons
of
paint.
(C)
An
increase
in
the
construction
of
new
houses.
(D)
An
increase
in
the
price
of
mechanical
painters
so
long
as
the
output
effect
exceeds
the
substitution
effect.'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
microeconomics
for
help.
An
increase
in
the
construction
of
new
houses
means
an
increase
demand
of
in-house
painting,
thus
increases
the
demand
for
housepainters.
The
answer
is
(C).'
group
:
mmlu_flan_cot_fewshot_social_sciences
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_microeconomics
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_physics.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_physics"
dataset_name
:
high_school_physics
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
physics.
\n\n
Q:
A
microwave
oven
is
connected
to
an
outlet,
120
V,
and
draws
\
school physics.
\
a
current
of
2
amps.
At
what
rate
is
energy
being
used
by
the
microwave
oven?
\n\
fewshot_config
:
(A)
10
W
(B)
30
W
(C)
60
W
(D)
240
W
\n
A:
Let's
think
step
by
step.
Rate
of
energy
\
sampler
:
first_n
\
usage
is
known
as
power;
in
an
dissipative
electrical
circuit,
power
is
given
\
samples
:
\
by
voltage
times
current.
So
in
our
case,
the
power
is
120
V
times
2
amps,
or
\
-
question
:
'
A
microwave
oven
is
connected
to
an
outlet,
120
V,
and
draws
a
current
\
240
W.
The
answer
is
(D).
\n\n
Q:
A
point
charge,
Q
=
+1
mC,
is
fixed
at
the
origin.
\
of
2
amps.
At
what
rate
is
energy
being
used
by
the
microwave
oven?
\
How
much
work
is
required
to
move
a
charge,
Q
=
+8
µC,
from
the
point
(0,
4
meters)
\
\
to
the
point
(3
meters,
0)?
\n
(A)
3.5
J
(B)
6.0
J
(C)
22.5
J
(D)
40
J
\n
A:
Let's
\
(A)
10
W
(B)
30
W
(C)
60
W
(D)
240
W'
\
think
step
by
step.
To
calculate
the
work
required
to
move
a
charge
from
one
location
\
target
:
Let's think step by step. Rate of energy usage is known as power; in an
\
to
another
in
a
fixed
electric
field,
it
is
enough
to
calculate
the
potential
\
dissipative electrical circuit, power is given by voltage times current. So
\
difference
between
the
two
locations.
Here,
the
potential
only
depends
on
the
\
in our case, the power is 120 V times 2 amps, or 240 W. The answer is (D).
\
distance
between
the
charges;
it’s
$k
q_1
q_2
/
r$,
where
$k$
is
Coulomb’s
constant.
\
-
question
:
"
A
point
charge,
Q
=
+1
mC,
is
fixed
at
the
origin.
How
much
work
is
required
\
\
Plugging
in
values
$q_1
=
$
1
mC,
$q_2
=
8
\\
mu$
C,
gives
the
answer
as
5.992
\
\
to
move
a
charge,
Q
=
+8
\xB5
C,
from
the
point
(0,
4
meters)
to
the
point
\
\
J,
which
rounds
to
6
J.
The
answer
is
(B).
\n\n
Q:
Which
of
the
following
conditions
\
\
(3
meters,
0)?
\n
(A)
3.5
J
(B)
6.0
J
(C)
22.5
J
(D)
40
J"
\
will
ensure
that
angular
momentum
is
conserved?
I.
Conservation
of
linear
momentum
\
target
:
"
Let's
think
step
by
step.
To
calculate
the
work
required
to
move
a
charge
\
\
II.
Zero
net
external
force
III.
Zero
net
external
torque
\n
(A)
I
and
II
only
(B)
\
\
from
one
location
to
another
in
a
fixed
electric
field,
it
is
enough
to
calculate
\
\
I
and
III
only
(C)
II
and
III
only
(D)
III
only
\n
A:
Let's
think
step
by
step.
\
\
the
potential
difference
between
the
two
locations.
Here,
the
potential
only
\
\
Torque
is
defined
as
the
change
in
angular
momentum;
if
there
is
zero
external
\
\
depends
on
the
distance
between
the
charges;
it
\u2019
s
$k
q_1
q_2
/
r$,
where
\
\
torque,
angular
momentum
is
conserved.
The
answer
is
(D).
\n\n
Q:
A
photocell
of
\
\
$k$
is
Coulomb
\u2019
s
constant.
Plugging
in
values
$q_1
=
$
1
mC,
$q_2
=
8
\
\
work
function
ϕ
=
2eV
is
connected
to
a
resistor
in
series.
Light
of
frequency
\
\ \\
mu$
C,
gives
the
answer
as
5.992
J,
which
rounds
to
6
J.
The
answer
is
(B)."
\
f
=
1
×
10^15
Hz
hits
a
metal
plate
of
the
photocell.
If
the
power
of
the
light
\
-
question
:
'
Which
of
the
following
conditions
will
ensure
that
angular
momentum
is
\
is
P
=
100
W,
what
is
the
current
through
the
resistor?
\n
(A)
2:00
AM
(B)
6:00
\
conserved?
I.
Conservation
of
linear
momentum
II.
Zero
net
external
force
III.
\
AM
(C)
12:00
AM
(D)
24
A
\n
A:
Let's
think
step
by
step.
The
only
answer
above
which
\
Zero
net
external
torque
\
has
units
of
current
is
D,
24
A.
The
answer
is
(D).
\n\n
Q:
A
pipe
full
of
air
is
\
\
closed
at
one
end.
A
standing
wave
is
produced
in
the
pipe,
causing
the
pipe
to
\
(A)
I
and
II
only
(B)
I
and
III
only
(C)
II
and
III
only
(D)
III
only'
\
sound
a
note.
Which
of
the
following
is
a
correct
statement
about
the
wave’s
properties
\
target
:
Let's think step by step. Torque is defined as the change in angular momentum;
\
at
the
closed
end
of
the
pipe?
\n
(A)
The
pressure
is
at
a
node,
but
the
particle
\
if there is zero external torque, angular momentum is conserved. The answer
\
displacement
is
at
an
antinode.
(B)
The
pressure
is
at
an
antinode,
but
the
particle
\
is (D).
\
displacement
is
at
a
node.
(C)
The
pressure
and
the
particle
displacement
are
\
-
question
:
"
A
photocell
of
work
function
\u03D5
=
2eV
is
connected
to
a
resistor
in
\
\
both
at
nodes.
(D)
The
pressure
and
the
particle
displacement
are
both
at
antinodes.
\n\
\
series.
Light
of
frequency
f
=
1
\xD7
10^15
Hz
hits
a
metal
plate
of
the
photocell.
\
A:
Let's
think
step
by
step.
At
the
closed
end
of
the
pipe,
the
particles
cannot
\
\
If
the
power
of
the
light
is
P
=
100
W,
what
is
the
current
through
the
resistor?
\n\
\
have
any
net
displacement
because
the
pipe
closure
stops
them.
So
the
particle
\
(A)
2:00
AM
(B)
6:00
AM
(C)
12:00
AM
(D)
24
A"
\
displacement
is
at
a
node.
This
closure
also
causes
the
pressure
to
be
maximal,
\
target
:
Let's think step by step. The only answer above which has units of current
\
i.e.
an
antinode.
The
answer
is
(B).
\n\n
"
is D, 24 A. The answer is (D).
"
group"
:
"
mmlu_flan_cot_fewshot_stem"
-
question
:
"
A
pipe
full
of
air
is
closed
at
one
end.
A
standing
wave
is
produced
in
\
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
\
the
pipe,
causing
the
pipe
to
sound
a
note.
Which
of
the
following
is
a
correct
\
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_physics"
\
statement
about
the
wave
\u2019
s
properties
at
the
closed
end
of
the
pipe?
\n\
(A)
The
pressure
is
at
a
node,
but
the
particle
displacement
is
at
an
antinode.
\
\
(B)
The
pressure
is
at
an
antinode,
but
the
particle
displacement
is
at
a
\
\
node.
(C)
The
pressure
and
the
particle
displacement
are
both
at
nodes.
(D)
\
\
The
pressure
and
the
particle
displacement
are
both
at
antinodes."
target
:
'
Let'
'
s
think
step
by
step.
At
the
closed
end
of
the
pipe,
the
particles
cannot
have
any
net
displacement
because
the
pipe
closure
stops
them.
So
the
particle
displacement
is
at
a
node.
This
closure
also
causes
the
pressure
to
be
maximal,
i.e.
an
antinode.
The
answer
is
(B).'
group
:
mmlu_flan_cot_fewshot_stem
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_physics
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_psychology.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_psychology"
dataset_name
:
high_school_psychology
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
psychology.
\n\n
Q:
Pascale
is
interested
in
the
processing
strategies
children
\
school psychology.
\
use
to
learn
new
information.
Pascale
would
best
be
classified
as
what
type
of
\
fewshot_config
:
\
psychologist?
\n
(A)
sociocultural
(B)
clinical
(C)
cognitive
(D)
behaviorist
\n\
sampler
:
first_n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
psychology
for
help.
\
samples
:
\
Sociocultural
psychologist
focuses
on
the
effect
of
societal
factors
on
people.
\
-
question
:
'
Pascale
is
interested
in
the
processing
strategies
children
use
to
learn
\
Clinical
psychologist
focuses
on
people
with
mental
issues.
Cognitive
psychologist
\
new
information.
Pascale
would
best
be
classified
as
what
type
of
psychologist?
\
focuses
on
how
people
think
and
learn,
including
the
processing
strategies.
Behaviorist
\
\
focuses
more
on
the
environment
and
experience
effect
on
people.
The
answer
is
\
(A)
sociocultural
(B)
clinical
(C)
cognitive
(D)
behaviorist'
\
(C).
\n\n
Q:
According
to
Caplan's
model
of
consultee-centered
case
consultation,
\
target
:
Let's think step by step. We refer to Wikipedia articles on psychology
\
the
consultant
is
primarily
interested
in
\n
(A)
identifying
the
causes
and
solutions
\
for help. Sociocultural psychologist focuses on the effect of societal factors
\
of
the
client's
presenting
problems
(B)
identifying
and
eliminating
the
causes
\
on people. Clinical psychologist focuses on people with mental issues. Cognitive
\
of
the
consultee's
difficulties
in
handling
a
problem
(C)
establishing
a
hierarchy
\
psychologist focuses on how people think and learn, including the processing
\
of
authority
to
enable
effective
decision
making
(D)
presenting
a
single,
well-defined
\
strategies. Behaviorist focuses more on the environment and experience effect
\
and
unambiguous
course
of
action
for
the
consultant
to
overcome
skills
deficits
\n\
on people. The answer is (C).
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
psychology
for
help.
\
-
question
:
'
According
to
Caplan'
'
s
model
of
consultee-centered
case
consultation,
\
Caplan
defines
two
type
of
consultation.
Client-centered
case
consultation
aims
\
the
consultant
is
primarily
interested
in
\
to
handle
client's
problems,
while
consultee-centered
case
consultation
aims
to
\
\
identify
the
reason
of
client's
difficulty
to
solve
problems.
The
answer
is
(B).
\n\
(A)
identifying
the
causes
and
solutions
of
the
client'
'
s
presenting
problems
\n
Q:
According
to
the
Individuals
with
Disabilities
Education
Improvement
Act,
which
\
(B)
identifying
and
eliminating
the
causes
of
the
consultee'
'
s
difficulties
\
of
the
following
must
an
educational
agency
do
before
it
changes
the
educational
\
in
handling
a
problem
(C)
establishing
a
hierarchy
of
authority
to
enable
effective
\
placement
of
a
student
with
a
disability?
\n
(A)
Give
the
child
a
trial
period
in
\
decision
making
(D)
presenting
a
single,
well-defined
and
unambiguous
course
\
the
new
environment
(B)
Notify
the
parents
in
writing
(C)
Obtain
school
board
\
of
action
for
the
consultant
to
overcome
skills
deficits'
\
approval
(D)
Obtain
parental
consent
\n
A:
Let's
think
step
by
step.
We
refer
to
\
target
:
Let's think step by step. We refer to Wikipedia articles on psychology
\
Wikipedia
articles
on
psychology
for
help.
When
the
decision
to
change
the
educational
\
for help. Caplan defines two type of consultation. Client-centered case consultation
\
placement
of
a
student
with
a
disability
is
made,
the
educational
agency
must
\
aims to handle client's problems, while consultee-centered case consultation
\
notify
the
parents
in
writing
on
that
date.
The
answer
is
(B).
\n\n
Q:
While
swimming
\
aims to identify the reason of client's difficulty to solve problems. The answer
\
in
the
ocean,
Ivan
is
frightened
by
a
dark
shadow
in
the
water
even
before
he
\
is (B).
\
has
the
chance
to
identify
what
the
shadow
is.
The
synaptic
connections
taking
\
-
question
:
'
According
to
the
Individuals
with
Disabilities
Education
Improvement
Act,
\
place
during
this
incident
of
fright
are
best
described
by
which
of
the
following?
\n\
which
of
the
following
must
an
educational
agency
do
before
it
changes
the
educational
(A)
Messages
are
sent
from
the
thalamus
directly
to
the
amygdala.
(B)
Messages
are
\
placement
of
a
student
with
a
disability?
\
sent
from
the
thalamus
to
the
\"
what
\"
and
\"
where
\"
pathways.
(C)
Messages
are
\
\
sent
from
the
parasympathetic
nervous
system
to
the
cerebral
cortex.
(D)
Messages
\
(A)
Give
the
child
a
trial
period
in
the
new
environment
(B)
Notify
the
parents
\
are
sent
from
the
frontal
lobes
to
the
pituitary
gland.
\n
A:
Let's
think
step
by
\
in
writing
(C)
Obtain
school
board
approval
(D)
Obtain
parental
consent'
\
step.
We
refer
to
Wikipedia
articles
on
psychology
for
help.
Our
neural
system
\
target
:
Let's think step by step. We refer to Wikipedia articles on psychology
\
has
a
mechanism
that
can
respond
immediate
emotional
signal
before
going
to
the
\
for help. When the decision to change the educational placement of a student
\
thought
center.
In
the
Ivan's
case,
messages
travel
directly
from
thalamus
to
\
with a disability is made, the educational agency must notify the parents in
\
amygdala.
The
answer
is
(A).
\n\n
Q:
Ani
believes
that
her
attitudes
and
behavior
\
writing on that date. The answer is (B).
\
play
a
central
role
in
what
happens
to
her.
Such
a
belief
is
likely
to
be
associated
\
-
question
:
'
While
swimming
in
the
ocean,
Ivan
is
frightened
by
a
dark
shadow
in
the
\
with
\n
(A)
a
strong
superego.
(B)
low
self-esteem.
(C)
low
self-efficacy.
(D)
an
\
water
even
before
he
has
the
chance
to
identify
what
the
shadow
is.
The
synaptic
\
internal
locus
of
control.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
connections
taking
place
during
this
incident
of
fright
are
best
described
by
\
articles
on
psychology
for
help.
People
with
an
external
locus
of
control
believes
\
which
of
the
following?
\
fate
and
luck
play
an
important
role
in
their
lives,
while
people
with
an
internal
\
\
locus
of
control
believes
they
control
their
lives.
The
answer
is
(D).
\n\n
"
(A)
Messages
are
sent
from
the
thalamus
directly
to
the
amygdala.
(B)
Messages
"
group"
:
"
mmlu_flan_cot_fewshot_social_sciences"
are
sent
from
the
thalamus
to
the
"what"
and
"where"
pathways.
(C)
Messages
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
are
sent
from
the
parasympathetic
nervous
system
to
the
cerebral
cortex.
(D)
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_psychology"
Messages
are
sent
from
the
frontal
lobes
to
the
pituitary
gland.'
target
:
Let's think step by step. We refer to Wikipedia articles on psychology
for help. Our neural system has a mechanism that can respond immediate emotional
signal before going to the thought center. In the Ivan's case, messages travel
directly from thalamus to amygdala. The answer is (A).
-
question
:
'
Ani
believes
that
her
attitudes
and
behavior
play
a
central
role
in
what
happens
to
her.
Such
a
belief
is
likely
to
be
associated
with
(A)
a
strong
superego.
(B)
low
self-esteem.
(C)
low
self-efficacy.
(D)
an
internal
locus
of
control.'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
psychology
for
help.
People
with
an
external
locus
of
control
believes
fate
and
luck
play
an
important
role
in
their
lives,
while
people
with
an
internal
locus
of
control
believes
they
control
their
lives.
The
answer
is
(D).'
group
:
mmlu_flan_cot_fewshot_social_sciences
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_psychology
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_statistics.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_statistics"
dataset_name
:
high_school_statistics
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
statistics.
\n\n
Q:
A
new
smartwatch
is
manufactured
in
one
part
of
a
factory,
\
school statistics.
\
then
secured
for
shipping
in
another,
independent
part
of
the
factory.
The
weight
\
fewshot_config
:
\
of
the
smartwatch
has
a
mean
of
62
grams
and
a
standard
deviation
of
1.0
grams.
\
sampler
:
first_n
\
The
weight
of
the
packaging
(box,
user's
guide,
bubble
wrap,
etc.)
has
a
mean
\
samples
:
\
of
456
grams
and
a
standard
deviation
of
6
grams.
Together,
the
distribution
of
\
-
question
:
'
A
new
smartwatch
is
manufactured
in
one
part
of
a
factory,
then
secured
\
the
weight
of
the
smartwatch
and
its
packaging
would
have
the
following
mean
and
\
for
shipping
in
another,
independent
part
of
the
factory.
The
weight
of
the
\
standard
deviation:
\n
(A)
Mean
518
grams;
standard
deviation
7.0
grams
(B)
Mean
\
smartwatch
has
a
mean
of
62
grams
and
a
standard
deviation
of
1.0
grams.
The
\
518
grams;
standard
deviation
3.5
grams
(C)
Mean
518
grams;
standard
deviation
\
weight
of
the
packaging
(box,
user'
'
s
guide,
bubble
wrap,
etc.)
has
a
mean
of
\
6.1
grams
(D)
Mean
394
grams;
standard
deviation
6.1
grams
\n
A:
Let's
think
step
\
456
grams
and
a
standard
deviation
of
6
grams.
Together,
the
distribution
of
\
by
step.
Since
the
weight
of
the
watch
and
the
weight
of
the
packaging
are
independent
\
the
weight
of
the
smartwatch
and
its
packaging
would
have
the
following
mean
\
random
variables,
the
mean
and
variance
of
their
sum
is
equal
to
the
sum
of
their
\
and
standard
deviation:
\
individual
means
and
variances.
So
the
mean
is
62
+
456
=
518
grams,
and
the
variances
\
\
is
1.0^2
+
6.0^2
=
37,
leading
to
a
standard
deviation
of
6.1
grams.
The
answer
\
(A)
Mean
518
grams;
standard
deviation
7.0
grams
(B)
Mean
518
grams;
standard
\
is
(C).
\n\n
Q:
After
a
frost
warning
was
issued,
the
owner
of
a
large
orange
grove
\
deviation
3.5
grams
(C)
Mean
518
grams;
standard
deviation
6.1
grams
(D)
Mean
\
asked
his
workers
to
spray
all
his
trees
with
water.
The
water
was
supposed
to
\
394
grams;
standard
deviation
6.1
grams'
\
freeze
and
form
a
protective
covering
of
ice
around
the
orange
blossom.
Nevertheless,
\
target
:
Let's think step by step. Since the weight of the watch and the weight
\
the
owner
suspected
that
some
trees
suffered
considerable
damage
due
to
the
frost.
\
of the packaging are independent random variables, the mean and variance of
\
To
estimate
the
proportion
of
trees
that
suffered
more
than
50
percent
damage
\
their sum is equal to the sum of their individual means and variances. So the
\
due
to
the
frost,
he
took
a
random
sample
of
100
trees
from
his
grove.
What
is
\
mean is 62 + 456 = 518 grams, and the variances is 1.0^2 + 6.0^2 = 37, leading
\
the
response
variable
in
this
experiment?
\n
(A)
The
proportion
of
trees
that
suffered
\
to a standard deviation of 6.1 grams. The answer is (C).
\
more
than
50
percent
damage
due
to
frost.
(B)
The
number
of
trees
affected
by
\
-
question
:
'
After
a
frost
warning
was
issued,
the
owner
of
a
large
orange
grove
asked
\
the
frost.
(C)
The
number
of
trees
sampled
from
the
grove.
(D)
For
each
sampled
\
his
workers
to
spray
all
his
trees
with
water.
The
water
was
supposed
to
freeze
\
tree,
whether
it
suffered
more
than
50
percent
damage
or
at
most
50
percent
damage.
\n\
and
form
a
protective
covering
of
ice
around
the
orange
blossom.
Nevertheless,
A:
Let's
think
step
by
step.
In
this
experiment,
the
response
variable
is
what
is
\
the
owner
suspected
that
some
trees
suffered
considerable
damage
due
to
the
\
measured.
For
each
tree,
what
is
measured
is
whether
or
not
it
suffered
more
than
\
frost.
To
estimate
the
proportion
of
trees
that
suffered
more
than
50
percent
\
50
percent
damage
due
to
the
frost.
The
answer
is
(D).
\n\n
Q:
Suppose
X
and
Y
are
\
damage
due
to
the
frost,
he
took
a
random
sample
of
100
trees
from
his
grove.
\
random
variables
with
E(X)
=
37,
var(X)
=
5,
E(Y)
=
62,
and
var(Y)
=
12.
What
\
What
is
the
response
variable
in
this
experiment?
\
are
the
expected
value
and
variance
of
the
random
variable
X
+
Y?
\n
(A)
E(X
+
Y)
\
\
=
99,
var(X
+
Y)
=
8.5
(B)
E(X
+
Y)
=
99,
var(X
+
Y)
=
13
(C)
E(X
+
Y)
=
99,
var(X
\
(A)
The
proportion
of
trees
that
suffered
more
than
50
percent
damage
due
to
\
+
Y)
=
17
(D)
There
is
insufficient
information
to
answer
this
question.
\n
A:
Let's
\
frost.
(B)
The
number
of
trees
affected
by
the
frost.
(C)
The
number
of
trees
\
think
step
by
step.
While
means
of
sums
of
random
variables
add
(regardless
of
\
sampled
from
the
grove.
(D)
For
each
sampled
tree,
whether
it
suffered
more
\
whether
the
variables
are
independent)
in
order
to
determine
the
variance
of
a
\
than
50
percent
damage
or
at
most
50
percent
damage.'
\
sum
of
random
variables,
we
need
to
know
not
just
their
individual
variances
but
\
target
:
Let's think step by step. In this experiment, the response variable is
\
the
covariance
of
the
two
variables,
which
is
not
given
in
this
problem.
The
answer
\
what is measured. For each tree, what is measured is whether or not it suffered
\
is
(D).
\n\n
Q:
Which
of
the
following
sets
has
the
smallest
standard
deviation?
\
more than 50 percent damage due to the frost. The answer is (D).
\
Which
has
the
largest?
\n
I:
{1,2,3}
\n
II:
{-10,10}
\n
III:
{100}
\n
(A)
I,
II
(B)
II,
\
-
question
:
'
Suppose
X
and
Y
are
random
variables
with
E(X)
=
37,
var(X)
=
5,
E(Y)
\
III
(C)
III,
I
(D)
III,
II
\n
A:
Let's
think
step
by
step.
The
variance
of
distribution
\
=
62,
and
var(Y)
=
12.
What
are
the
expected
value
and
variance
of
the
random
\
I
is
the
expected
squared
deviation
from
its
mean
(which
is
2),
so
the
variance
\
variable
X
+
Y?
\
is
2/3
.
The
variance
of
distribution
II
is
10^2
(because
both
elements
are
10
\
\
away
from
the
mean
of
zero).
The
variance
of
distribution
III
is
0,
since
it
has
\
(A)
E(X
+
Y)
=
99,
var(X
+
Y)
=
8.5
(B)
E(X
+
Y)
=
99,
var(X
+
Y)
=
13
(C)
E(X
\
a
single
entry.
So
distribution
III
has
the
smallest
standard
deviation
and
distribution
\
+
Y)
=
99,
var(X
+
Y)
=
17
(D)
There
is
insufficient
information
to
answer
this
\
II
has
the
largest.
The
answer
is
(D).
\n\n
Q:
Which
of
the
following
is
a
correct
\
question.'
\
statement
about
correlation?
\n
(A)
If
the
slope
of
the
regression
line
is
exactly
\
target
:
Let's think step by step. While means of sums of random variables add
\
1,
then
the
correlation
is
exactly
1.
(B)
If
the
correlation
is
0,
then
the
slope
\
(regardless of whether the variables are independent) in order to determine
\
of
the
regression
line
is
undefined.
(C)
Switching
which
variable
is
called
x
\
the variance of a sum of random variables, we need to know not just their individual
\
and
which
is
called
y
changes
the
sign
of
the
correlation.
(D)
The
correlation
\
variances but the covariance of the two variables, which is not given in this
\
r
is
equal
to
the
slope
of
the
regression
line
when
z-scores
for
the
y-variable
\
problem. The answer is (D).
\
are
plotted
against
z-scores
for
the
x-variable.
\n
A:
Let's
think
step
by
step.
\
-
question
:
'
Which
of
the
following
sets
has
the
smallest
standard
deviation?
Which
\
Statement
A
is
false
because
the
slope
of
the
regression
line
being
exactly
1
\
has
the
largest?
\
can
occur
even
when
the
two
variables
are
not
perfectly
correlated.
Statement
\
\
B
is
false
because
uncorrelated
variables
regression
lines
can
have
slope
zero.
\
I:
{1,2,3}
\
Statement
C
is
false
because
correlation
is
symmetric
in
the
two
random
variables.
\
\
The
answer
is
(D).
\n\n
"
II:
{-10,10}
"
group"
:
"
mmlu_flan_cot_fewshot_stem"
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
III:
{100}
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_statistics"
(A)
I,
II
(B)
II,
III
(C)
III,
I
(D)
III,
II'
target
:
Let's think step by step. The variance of distribution I is the expected
squared deviation from its mean (which is 2), so the variance is 2/3 . The variance
of distribution II is 10^2 (because both elements are 10 away from the mean
of zero). The variance of distribution III is 0, since it has a single entry.
So distribution III has the smallest standard deviation and distribution II
has the largest. The answer is (D).
-
question
:
'
Which
of
the
following
is
a
correct
statement
about
correlation?
(A)
If
the
slope
of
the
regression
line
is
exactly
1,
then
the
correlation
is
exactly
1.
(B)
If
the
correlation
is
0,
then
the
slope
of
the
regression
line
is
undefined.
(C)
Switching
which
variable
is
called
x
and
which
is
called
y
changes
the
sign
of
the
correlation.
(D)
The
correlation
r
is
equal
to
the
slope
of
the
regression
line
when
z-scores
for
the
y-variable
are
plotted
against
z-scores
for
the
x-variable.'
target
:
'
Let'
'
s
think
step
by
step.
Statement
A
is
false
because
the
slope
of
the
regression
line
being
exactly
1
can
occur
even
when
the
two
variables
are
not
perfectly
correlated.
Statement
B
is
false
because
uncorrelated
variables
regression
lines
can
have
slope
zero.
Statement
C
is
false
because
correlation
is
symmetric
in
the
two
random
variables.
The
answer
is
(D).'
group
:
mmlu_flan_cot_fewshot_stem
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_statistics
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_us_history.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_us_history"
dataset_name
:
high_school_us_history
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
us
history.
\n\n
Q:
This
question
refers
to
the
following
information.
\n\
school us history.
I
come
not
to
urge
personal
claims,
nor
to
seek
individual
benefits;
I
appear
as
\
fewshot_config
:
\
the
advocate
of
those
who
cannot
plead
their
own
cause;
I
come
as
the
friend
of
\
sampler
:
first_n
\
those
who
are
deserted,
oppressed,
and
desolate.
In
the
Providence
of
God,
I
am
\
samples
:
\
the
voice
of
the
maniac
whose
piercing
cries
from
the
dreary
dungeons
of
your
\
-
question
:
"
This
question
refers
to
the
following
information.
\n
I
come
not
to
urge
\
\
jails
penetrate
not
your
Halls
of
Legislation.
I
am
the
Hope
of
the
poor
crazed
\
\
personal
claims,
nor
to
seek
individual
benefits;
I
appear
as
the
advocate
\
\
beings
who
pine
in
the
cells,
and
stalls,
and
cages,
and
waste
rooms
of
your
poor-houses.
\
\
of
those
who
cannot
plead
their
own
cause;
I
come
as
the
friend
of
those
who
\
\
I
am
the
Revelation
of
hundreds
of
wailing,
suffering
creatures,
hidden
in
your
\
\
are
deserted,
oppressed,
and
desolate.
In
the
Providence
of
God,
I
am
the
\
\
private
dwellings,
and
in
pens
and
cabins—shut
out,
cut
off
from
all
healing
influences,
\
\
voice
of
the
maniac
whose
piercing
cries
from
the
dreary
dungeons
of
your
\
\
from
all
mind-restoring
cares.…
Could
their
melancholy
histories
be
spread
before
\
\
jails
penetrate
not
your
Halls
of
Legislation.
I
am
the
Hope
of
the
poor
crazed
\
\
you
as
revealed
to
my
grieved
spirit
during
the
last
three
months,
how
promptly,
\
\
beings
who
pine
in
the
cells,
and
stalls,
and
cages,
and
waste
rooms
of
your
\
\
how
earnestly
would
you
search
out
the
most
approved
means
of
relief;
how
trifling,
\
\
poor-houses.
I
am
the
Revelation
of
hundreds
of
wailing,
suffering
creatures,
\
\
how
insignificant,
by
comparison,
would
appear
the
sacrifices
you
are
asked
to
\
\
hidden
in
your
private
dwellings,
and
in
pens
and
cabins
\u2014
shut
out,
cut
\
\
make;
how
would
a
few
dimes
and
dollars,
gathered
from
each
citizen,
diminish
\
\
off
from
all
healing
influences,
from
all
mind-restoring
cares.
\u2026
Could
\
\
in
value
as
a
possession,
compared
with
the
certain
benefits
and
vast
good
to
\
\
their
melancholy
histories
be
spread
before
you
as
revealed
to
my
grieved
\
\
be
secured
for
the
suffering
insane...by
the
consecration
and
application
of
a
\
\
spirit
during
the
last
three
months,
how
promptly,
how
earnestly
would
you
\
\
sufficient
fund
to
the
construction
of
a
suitable
hospital.…
\n
—Dorothea
Dix,
Memorial
\
\
search
out
the
most
approved
means
of
relief;
how
trifling,
how
insignificant,
\
\
Soliciting
a
State
Hospital
for
the
Protection
and
Cure
of
the
Insane,
\n
Submitted
\
\
by
comparison,
would
appear
the
sacrifices
you
are
asked
to
make;
how
would
\
\
to
the
General
Assembly
of
North
Carolina,
November
1848
\n
Dorothea
Dix
can
best
\
\
a
few
dimes
and
dollars,
gathered
from
each
citizen,
diminish
in
value
as
\
\
be
compared
to
whom?
\n
(A)
Abigail
Adams
(B)
Clara
Barton
(C)
Shirley
Temple
(D)
\
\
a
possession,
compared
with
the
certain
benefits
and
vast
good
to
be
secured
\
\
Hillary
Clinton
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
\
\
for
the
suffering
insane...by
the
consecration
and
application
of
a
sufficient
\
\
us
history
for
help.
Both
Dorothea
Dix
and
Clara
barton
are
American
nurses.
The
\
\
fund
to
the
construction
of
a
suitable
hospital.
\u2026\n\u2014
Dorothea
Dix,
\
\
answer
is
(B).
\n\n
Q:
This
question
refers
to
the
following
information.
\n\"
As
\
\
Memorial
Soliciting
a
State
Hospital
for
the
Protection
and
Cure
of
the
Insane,
\n\
\
our
late
Conduct
at
the
Conestoga
Manor
and
Lancaster
have
occasioned
much
Speculation
\
Submitted
to
the
General
Assembly
of
North
Carolina,
November
1848
\n
Dorothea
\
\
&
a
great
diversity
of
Sentiments
in
this
and
neighboring
Governments;
some
vindicating
\
\
Dix
can
best
be
compared
to
whom?
\n
(A)
Abigail
Adams
(B)
Clara
Barton
(C)
\
\
&
others
condemning
it;
some
charitably
alleviating
the
Crime,
&
others
maliciously
\
\
Shirley
Temple
(D)
Hillary
Clinton"
\
painting
it
in
the
most
odious
&
detestable
Colours,
we
think
it
our
duty
to
lay
\
target
:
Let's think step by step. We refer to Wikipedia articles on us history
\
before
the
Publick,
the
whole
Matter
as
it
appeared,
&
still
appears,
to
us.
.
\
for help. Both Dorothea Dix and Clara barton are American nurses. The answer
\
.
.
\n\"
If
these
things
are
not
sufficient
to
prove
an
unjustifiable
Attachment
\
is (B).
\
in
the
Quakers
to
the
Indians
Savages,
a
fixed
Resolution
to
befriend
them
&
an
\
-
question
:
"
This
question
refers
to
the
following
information.
\n\"
As
our
late
Conduct
\
\
utter
insensibility
to
human
Distresses,
let
us
consider
a
few
more
recent
Facts.
\
\
at
the
Conestoga
Manor
and
Lancaster
have
occasioned
much
Speculation
&
a
\
\
When
we
found
the
last
Summer
that
we
were
likely
to
get
no
Assistance
from
the
\
\
great
diversity
of
Sentiments
in
this
and
neighboring
Governments;
some
vindicating
\
\
Government,
some
Volunteers
went
out
at
our
own
Expense,
determined
to
drive
our
\
\
&
others
condemning
it;
some
charitably
alleviating
the
Crime,
&
others
maliciously
\
\
Enemies
from
our
Borders;
&
when
we
came
near
to
the
great
Island,
we
understood
\
\
painting
it
in
the
most
odious
&
detestable
Colours,
we
think
it
our
duty
\
\
that
a
Number
of
their
Warriors
had
gone
out
against
our
Frontiers.
Upon
this
\
\
to
lay
before
the
Publick,
the
whole
Matter
as
it
appeared,
&
still
appears,
\
\
we
returned
and
came
up
with
them
and
fought
with
them
at
the
Munfey
Hill
where
\
\
to
us.
.
.
.
\n\"
If
these
things
are
not
sufficient
to
prove
an
unjustifiable
\
\
we
lost
some
of
our
Men
&
killed
some
of
their
Warriors
&
thereby
saved
our
Frontiers
\
\
Attachment
in
the
Quakers
to
the
Indians
Savages,
a
fixed
Resolution
to
befriend
\
\
from
this
Story
in
another
Expedition.
But
no
sooner
had
we
destroyed
their
Provisions
\
\
them
&
an
utter
insensibility
to
human
Distresses,
let
us
consider
a
few
more
\
\
on
the
great
Island,
&
ruined
their
trade
with
the
good
People
at
Bethlehem,
but
\
\
recent
Facts.
When
we
found
the
last
Summer
that
we
were
likely
to
get
no
\
\
these
very
Indians,
who
were
justly
suspected
of
having
murdered
our
Friends
in
\
\
Assistance
from
the
Government,
some
Volunteers
went
out
at
our
own
Expense,
\
\
Northampton
County,
were
by
the
Influence
of
some
Quakers
taken
under
the
Protection
\
\
determined
to
drive
our
Enemies
from
our
Borders;
&
when
we
came
near
to
the
\
\
of
the
Government
to
screen
them
from
the
Resentments
of
the
Friends
and
Relations
\
\
great
Island,
we
understood
that
a
Number
of
their
Warriors
had
gone
out
against
\
\
of
the
Murdered,
&
to
support
them
thro
the
Winter.
\"\n
—
\"
Apology
of
the
Paxton
\
\
our
Frontiers.
Upon
this
we
returned
and
came
up
with
them
and
fought
with
\
\
Boys
\"
(pamphlet),
1764
(Note:
\"
apology
\"
in
this
context
should
be
read
as
an
\
\
them
at
the
Munfey
Hill
where
we
lost
some
of
our
Men
&
killed
some
of
their
\
\
explanation,
not
an
admission
of
guilt
or
regret.
\n
The
sentiments
expressed
in
\
\
Warriors
&
thereby
saved
our
Frontiers
from
this
Story
in
another
Expedition.
\
\
the
explanation
above
reflect
which
of
the
ongoing
tensions
during
the
colonial
\
\
But
no
sooner
had
we
destroyed
their
Provisions
on
the
great
Island,
&
ruined
\
\
period
of
American
history?
\n
(A)
Tensions
between
British
policies
and
the
aspirations
\
\
their
trade
with
the
good
People
at
Bethlehem,
but
these
very
Indians,
who
\
\
of
North
American
colonists.
(B)
Tensions
between
American
Indians
allied
with
\
\
were
justly
suspected
of
having
murdered
our
Friends
in
Northampton
County,
\
\
the
French
and
those
allied
with
the
British.
(C)
Tensions
between
freed
African
\
\
were
by
the
Influence
of
some
Quakers
taken
under
the
Protection
of
the
Government
\
\
Americans
and
white
planters.
(D)
Tensions
between
backcountry
settlers
and
elites
\
\
to
screen
them
from
the
Resentments
of
the
Friends
and
Relations
of
the
Murdered,
\
\
within
colonial
America.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
\
\
&
to
support
them
thro
the
Winter.
\"\n\u2014\"
Apology
of
the
Paxton
Boys
\"\
\
on
us
history
for
help.
After
the
French
and
Indian
War,
the
Scotch-Irish
settlers
\
\
(pamphlet),
1764
(Note:
\"
apology
\"
in
this
context
should
be
read
as
an
explanation,
\
\
attacked
American
Indians.
After
the
attacks
on
the
Conestoga,
about
250
Paxton
\
\
not
an
admission
of
guilt
or
regret.
\n
The
sentiments
expressed
in
the
explanation
\
\
Boys
present
their
grievances
to
the
Pennsylvania
legislature.
As
mentioned
in
\
\
above
reflect
which
of
the
ongoing
tensions
during
the
colonial
period
of
\
\
the
information,
the
Paxton
Boys
cited
resentiment
at
local
elites.
The
answer
\
\
American
history?
\n
(A)
Tensions
between
British
policies
and
the
aspirations
\
\
is
(D).
\n\n
Q:
This
question
refers
to
the
following
information.
\n
Our
leaders
\
\
of
North
American
colonists.
(B)
Tensions
between
American
Indians
allied
\
\
talk
about
stopping
aggression
from
the
north,
but
this
was
a
struggle
among
groups
\
\
with
the
French
and
those
allied
with
the
British.
(C)
Tensions
between
freed
\
\
of
Vietnamese
until
we
intervened.
We
seem
bent
upon
saving
the
Vietnamese
from
\
\
African
Americans
and
white
planters.
(D)
Tensions
between
backcountry
settlers
\
\
Ho
Chi
Minh
even
if
we
have
to
kill
them
and
demolish
their
country
to
do
it.
\
\
and
elites
within
colonial
America."
\
As
the
native
people
survey
bombed-out
villages,
women
and
children
burned
by
\
target
:
Let's think step by step. We refer to Wikipedia articles on us history
\
napalm,
rice
crops
destroyed
and
cities
overrun
with
our
military
personnel,
they
\
for help. After the French and Indian War, the Scotch-Irish settlers attacked
\
are
doubtless
saying
secretly
of
the
Vietcong
guerillas
and
of
the
American
forces,
\
American Indians. After the attacks on the Conestoga, about 250 Paxton Boys
\ \"
A
plague
on
both
your
houses.
\"
…
Stop
the
bombing,
north
and
south,
end
search
\
present their grievances to the Pennsylvania legislature. As mentioned in the
\
and
destroy
offensive
sweeps,
and
confine
our
military
action
to
holding
operations
\
information, the Paxton Boys cited resentiment at local elites. The answer is
\
on
the
ground.
Bombing
the
north
has
failed
to
halt
or
seriously
check
the
flow
\
(D).
\
of
troops
to
the
south
and
may,
in
fact,
have
prompted
a
much
greater
war
effort
\
-
question
:
"
This
question
refers
to
the
following
information.
\n
Our
leaders
talk
about
\
\
by
Hanoi.
\n
—Senator
George
McGovern,
\"
The
Lessons
of
Vietnam,
\"
April
25,
1967
\n\
\
stopping
aggression
from
the
north,
but
this
was
a
struggle
among
groups
of
\
Which
of
the
following
opinions
from
the
1960s
most
directly
reflects
the
perspective
\
\
Vietnamese
until
we
intervened.
We
seem
bent
upon
saving
the
Vietnamese
from
\
\
of
George
McGovern's
speech?
\n
(A)
Americans
must
maximize
their
technological
\
\
Ho
Chi
Minh
even
if
we
have
to
kill
them
and
demolish
their
country
to
do
\
\
edge
in
Vietnam.
(B)
American
bombing
in
Vietnam
is
step
by
step
leading
to
progress
\
\
it.
As
the
native
people
survey
bombed-out
villages,
women
and
children
burned
\
\
in
the
war.
(C)
American
bombing
in
Vietnam
is
a
failure.
(D)
America
must
not
\
\
by
napalm,
rice
crops
destroyed
and
cities
overrun
with
our
military
personnel,
\
\
give
in
to
defeatism
about
the
war
in
Vietnam.
\n
A:
Let's
think
step
by
step.
We
\
\
they
are
doubtless
saying
secretly
of
the
Vietcong
guerillas
and
of
the
American
\
\
refer
to
Wikipedia
articles
on
us
history
for
help.
\"
Stop
the
bombing
\"
and
\"\
\
forces,
\"
A
plague
on
both
your
houses.
\"
\u2026
Stop
the
bombing,
north
and
\
Bombing
the
north
has
failed
to
halt
or
seriously
check
the
flow
of
troops
to
the
\
\
south,
end
search
and
destroy
offensive
sweeps,
and
confine
our
military
action
\
\
south
\"
indicate
that
the
perspective
of
George
McGovern's
speech
is
that
Amerian
\
\
to
holding
operations
on
the
ground.
Bombing
the
north
has
failed
to
halt
\
\
bombing
in
Vietnam
is
a
failure.
The
answer
is
(C).
\n\n
Q:
This
question
refers
\
\
or
seriously
check
the
flow
of
troops
to
the
south
and
may,
in
fact,
have
\
\
to
the
following
information.
\n\"
In
the
new
Code
of
Laws
which
I
suppose
it
will
\
\
prompted
a
much
greater
war
effort
by
Hanoi.
\n\u2014
Senator
George
McGovern,
\
\
be
necessary
for
you
to
make
I
desire
you
would
Remember
the
Ladies,
and
be
more
\
\ \"
The
Lessons
of
Vietnam,
\"
April
25,
1967
\n
Which
of
the
following
opinions
\
\
generous
and
favorable
to
them
than
your
ancestors.
Do
not
put
such
unlimited
\
\
from
the
1960s
most
directly
reflects
the
perspective
of
George
McGovern's
\
\
power
into
the
hands
of
the
Husbands.
Remember
all
Men
would
be
tyrants
if
they
\
\
speech?
\n
(A)
Americans
must
maximize
their
technological
edge
in
Vietnam.
\
\
could.
If
particular
care
and
attention
is
not
paid
to
the
Ladies
we
are
determined
\
\
(B)
American
bombing
in
Vietnam
is
step
by
step
leading
to
progress
in
the
\
\
to
foment
a
Rebellion,
and
will
not
hold
ourselves
bound
by
any
Laws
in
which
\
\
war.
(C)
American
bombing
in
Vietnam
is
a
failure.
(D)
America
must
not
give
\
\
we
have
no
voice,
or
Representation.
\"\n
Abigail
Adams,
in
a
letter
to
John
Adams,
\
\
in
to
defeatism
about
the
war
in
Vietnam."
\
1776
\n\"
Special
legislation
for
woman
has
placed
us
in
a
most
anomalous
position.
\
target
:
Let's think step by step. We refer to Wikipedia articles on us history
\
Women
invested
with
the
rights
of
citizens
in
one
section—voters,
jurors,
office-holders—crossing
\
for help. "Stop the bombing" and "Bombing the north has failed to halt or seriously
\
an
imaginary
line,
are
subjects
in
the
next.
In
some
States,
a
married
woman
may
\
check the flow of troops to the south" indicate that the perspective of George
\
hold
property
and
transact
business
in
her
own
name;
in
others,
her
earnings
belong
\
McGovern's speech is that Amerian bombing in Vietnam is a failure. The answer
\
to
her
husband.
In
some
States,
a
woman
may
testify
against
her
husband,
sue
and
\
is (C).
\
be
sued
in
the
courts;
in
others,
she
has
no
redress
in
case
of
damage
to
person,
\
-
question
:
"
This
question
refers
to
the
following
information.
\n\"
In
the
new
Code
\
\
property,
or
character.
In
case
of
divorce
on
account
of
adultery
in
the
husband,
\
\
of
Laws
which
I
suppose
it
will
be
necessary
for
you
to
make
I
desire
you
\
\
the
innocent
wife
is
held
to
possess
no
right
to
children
or
property,
unless
\
\
would
Remember
the
Ladies,
and
be
more
generous
and
favorable
to
them
than
\
\
by
special
decree
of
the
court.
But
in
no
State
of
the
Union
has
the
wife
the
\
\
your
ancestors.
Do
not
put
such
unlimited
power
into
the
hands
of
the
Husbands.
\
\
right
to
her
own
person,
or
to
any
part
of
the
joint
earnings
of
the
co-partnership
\
\
Remember
all
Men
would
be
tyrants
if
they
could.
If
particular
care
and
attention
\
\
during
the
life
of
her
husband.
In
some
States
women
may
enter
the
law
schools
\
\
is
not
paid
to
the
Ladies
we
are
determined
to
foment
a
Rebellion,
and
will
\
\
and
practice
in
the
courts;
in
others
they
are
forbidden.
In
some
universities
\
\
not
hold
ourselves
bound
by
any
Laws
in
which
we
have
no
voice,
or
Representation.
\"\
\
girls
enjoy
equal
educational
advantages
with
boys,
while
many
of
the
proudest
\
\n
Abigail
Adams,
in
a
letter
to
John
Adams,
1776
\n\"
Special
legislation
for
\
\
institutions
in
the
land
deny
them
admittance,
though
the
sons
of
China,
Japan
\
\
woman
has
placed
us
in
a
most
anomalous
position.
Women
invested
with
the
\
\
and
Africa
are
welcomed
there.
But
the
privileges
already
granted
in
the
several
\
\
rights
of
citizens
in
one
section
\u2014
voters,
jurors,
office-holders
\u2014\
\
States
are
by
no
means
secure.
\"\n
Susan
B.
Anthony,
\"
Declaration
of
Rights
for
\
crossing
an
imaginary
line,
are
subjects
in
the
next.
In
some
States,
a
married
\
\
Women,
\"
July
4,
1876
\n
The
sentiments
expressed
in
the
second
excerpt
by
Susan
\
\
woman
may
hold
property
and
transact
business
in
her
own
name;
in
others,
\
\
B.
Anthony
are
most
likely
in
support
of
\n
(A)
the
Equal
Rights
Amendment
(B)
universal
\
\
her
earnings
belong
to
her
husband.
In
some
States,
a
woman
may
testify
against
\
\
suffrage
(C)
states'
rights
(D)
prohibition
\n
A:
Let's
think
step
by
step.
We
refer
\
\
her
husband,
sue
and
be
sued
in
the
courts;
in
others,
she
has
no
redress
\
\
to
Wikipedia
articles
on
us
history
for
help.
The
above
information
mentioned
\
\
in
case
of
damage
to
person,
property,
or
character.
In
case
of
divorce
on
\
\
that
women
are
in
an
anomalous
position
in
terms
of
legislation.
Women's
earnings
\
\
account
of
adultery
in
the
husband,
the
innocent
wife
is
held
to
possess
no
\
\
do
not
belong
to
themselves,
or
they
cannot
testify
against
her
husbands.
Susan
\
\
right
to
children
or
property,
unless
by
special
decree
of
the
court.
But
\
\
believes
women
should
have
equal
legal
rights
as
men.
The
answer
is
(B).
\n\n
Q:
\
\
in
no
State
of
the
Union
has
the
wife
the
right
to
her
own
person,
or
to
any
\
\
This
question
refers
to
the
following
information.
\n\"
Society
in
every
state
is
\
\
part
of
the
joint
earnings
of
the
co-partnership
during
the
life
of
her
husband.
\
\
a
blessing,
but
government
even
in
its
best
state
is
but
a
necessary
evil;
in
\
\
In
some
States
women
may
enter
the
law
schools
and
practice
in
the
courts;
\
\
its
worst
state
an
intolerable
one;
for
when
we
suffer,
or
are
exposed
to
the
\
\
in
others
they
are
forbidden.
In
some
universities
girls
enjoy
equal
educational
\
\
same
miseries
by
a
government,
which
we
might
expect
in
a
country
without
government,
\
\
advantages
with
boys,
while
many
of
the
proudest
institutions
in
the
land
\
\
our
calamity
is
heightened
by
reflecting
that
we
furnish
the
means
by
which
we
\
\
deny
them
admittance,
though
the
sons
of
China,
Japan
and
Africa
are
welcomed
\
\
suffer.
Government,
like
dress,
is
the
badge
of
lost
innocence;
the
palaces
of
\
\
there.
But
the
privileges
already
granted
in
the
several
States
are
by
no
\
\
kings
are
built
on
the
ruins
of
the
bowers
of
paradise.
For
were
the
impulses
\
\
means
secure.
\"\n
Susan
B.
Anthony,
\"
Declaration
of
Rights
for
Women,
\"
July
\
\
of
conscience
clear,
uniform,
and
irresistibly
obeyed,
man
would
need
no
other
\
\
4,
1876
\n
The
sentiments
expressed
in
the
second
excerpt
by
Susan
B.
Anthony
\
\
lawgiver;
but
that
not
being
the
case,
he
finds
it
necessary
to
surrender
up
a
\
\
are
most
likely
in
support
of
\n
(A)
the
Equal
Rights
Amendment
(B)
universal
\
\
part
of
his
property
to
furnish
means
for
the
protection
of
the
rest;
and
this
\
\
suffrage
(C)
states'
rights
(D)
prohibition"
\
he
is
induced
to
do
by
the
same
prudence
which
in
every
other
case
advises
him
\
target
:
Let's think step by step. We refer to Wikipedia articles on us history
\
out
of
two
evils
to
choose
the
least.
Wherefore,
security
being
the
true
design
\
for help. The above information mentioned that women are in an anomalous position
\
and
end
of
government,
it
unanswerably
follows
that
whatever
form
thereof
appears
\
in terms of legislation. Women's earnings do not belong to themselves, or they
\
most
likely
to
ensure
it
to
us,
with
the
least
expense
and
greatest
benefit,
is
\
cannot testify against her husbands. Susan believes women should have equal
\
preferable
to
all
others.
\"\n
Thomas
Paine,
Common
Sense,
1776
\n
Which
of
the
following
\
legal rights as men. The answer is (B).
\ \"
miseries
\"
alluded
to
above
were
most
condemned
by
Anti-Federalists
of
the
post-Revolutionary
\
-
question
:
'
This
question
refers
to
the
following
information.
\
era?
\n
(A)
Organized
response
to
Bacon's
Rebellion
(B)
Federal
response
to
Shays's
\
\
Rebellion
(C)
Federal
response
to
the
Whiskey
Rebellion
(D)
Federal
response
to
\
"Society
in
every
state
is
a
blessing,
but
government
even
in
its
best
state
\
Pontiac's
Rebellion
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
\
is
but
a
necessary
evil;
in
its
worst
state
an
intolerable
one;
for
when
we
\
on
us
history
for
help.
Anti-Federalists
do
not
believe
centralized
government
\
suffer,
or
are
exposed
to
the
same
miseries
by
a
government,
which
we
might
\
power,
and
suspect
Washington's
military
response
to
Whiskey
Rebellion.
Bacon's
\
expect
in
a
country
without
government,
our
calamity
is
heightened
by
reflecting
\
Rebellion
and
Pontiac's
Rebellion
happen
before
the
Revolution
and
they
can
be
\
that
we
furnish
the
means
by
which
we
suffer.
Government,
like
dress,
is
the
\
ruled
out.
The
answer
is
(C).
\n\n
"
badge
of
lost
innocence;
the
palaces
of
kings
are
built
on
the
ruins
of
the
"
group"
:
"
mmlu_flan_cot_fewshot_humanities"
bowers
of
paradise.
For
were
the
impulses
of
conscience
clear,
uniform,
and
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
irresistibly
obeyed,
man
would
need
no
other
lawgiver;
but
that
not
being
the
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_us_history"
case,
he
finds
it
necessary
to
surrender
up
a
part
of
his
property
to
furnish
means
for
the
protection
of
the
rest;
and
this
he
is
induced
to
do
by
the
same
prudence
which
in
every
other
case
advises
him
out
of
two
evils
to
choose
the
least.
Wherefore,
security
being
the
true
design
and
end
of
government,
it
unanswerably
follows
that
whatever
form
thereof
appears
most
likely
to
ensure
it
to
us,
with
the
least
expense
and
greatest
benefit,
is
preferable
to
all
others."
Thomas
Paine,
Common
Sense,
1776
Which
of
the
following
"miseries"
alluded
to
above
were
most
condemned
by
Anti-Federalists
of
the
post-Revolutionary
era?
(A)
Organized
response
to
Bacon'
'
s
Rebellion
(B)
Federal
response
to
Shays'
'
s
Rebellion
(C)
Federal
response
to
the
Whiskey
Rebellion
(D)
Federal
response
to
Pontiac'
'
s
Rebellion'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
us
history
for
help.
Anti-Federalists
do
not
believe
centralized
government
power,
and
suspect
Washington'
'
s
military
response
to
Whiskey
Rebellion.
Bacon'
'
s
Rebellion
and
Pontiac'
'
s
Rebellion
happen
before
the
Revolution
and
they
can
be
ruled
out.
The
answer
is
(C).'
group
:
mmlu_flan_cot_fewshot_humanities
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_us_history
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_world_history.yaml
View file @
da211969
"
dataset_name"
:
"
high_school_world_history"
dataset_name
:
high_school_world_history
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
high
\
description
:
The following are multiple choice questions (with answers) about high
\
school
world
history.
\n\n
Q:
This
question
refers
to
the
following
information.
\n\
school world history.
\"
At
least
one
of
the
[world's]
societies
would
have
to
somehow
enormously
increase
\
fewshot_config
:
\
its
productivity
[in
order
to
achieve
global
hegemony].
That
quantum
jump
would
\
sampler
:
first_n
\
have
to
be
made
before
the
various
scientific,
technological,
agricultural,
and
\
samples
:
\
industrial
revolutions
on
which
our
post-quantum-leap
world
rests.
It
could
only
\
-
question
:
"
This
question
refers
to
the
following
information.
\n\"
At
least
one
of
\
\
be
accomplished
by
exploiting
the
ecosystems,
mineral
resources,
and
human
assets
\
\
the
[world's]
societies
would
have
to
somehow
enormously
increase
its
productivity
\
\
of
whole
continents
outside
the
lands
of
the
society
making
the
jump.
Western
\
\
[in
order
to
achieve
global
hegemony].
That
quantum
jump
would
have
to
be
\
\
Europe
did
just
that
by
means
of
its
brutality
and
guns
and,
more
important,
by
\
\
made
before
the
various
scientific,
technological,
agricultural,
and
industrial
\
\
geographical
and
ecological
luck.
\"\n
Copyright
©
2015
Cambridge
University
Press.
\n\
\
revolutions
on
which
our
post-quantum-leap
world
rests.
It
could
only
be
accomplished
\
Alfred
Crosby,
historian,
Ecological
Imperialism,
2004
\n
The
\"
quantum
jump
\"
mentioned
\
\
by
exploiting
the
ecosystems,
mineral
resources,
and
human
assets
of
whole
\
\
in
the
passage
most
directly
contributed
to
which
of
the
following
developments
\
\
continents
outside
the
lands
of
the
society
making
the
jump.
Western
Europe
\
\
in
the
period
1450–1750
C.E.?
\n
(A)
A
breakdown
in
trade
routes
through
the
collapse
\
\
did
just
that
by
means
of
its
brutality
and
guns
and,
more
important,
by
geographical
\
\
of
the
established
state
structure
(B)
An
increase
in
the
population
of
the
world
\
\
and
ecological
luck.
\"\n
Copyright
\xA9
2015
Cambridge
University
Press.
\n\
\
through
more
plentiful
supplies
of
food
(C)
The
spread
of
Chinese
and
Indian
belief
\
Alfred
Crosby,
historian,
Ecological
Imperialism,
2004
\n
The
\"
quantum
jump
\"\
\
systems
across
the
world
(D)
An
increase
in
social
unrest
\n
A:
Let's
think
step
\
\
mentioned
in
the
passage
most
directly
contributed
to
which
of
the
following
\
\
by
step.
We
refer
to
Wikipedia
articles
on
world
history
for
help.
The
\"
quantum
\
\
developments
in
the
period
1450
\u2013
1750
C.E.?
\n
(A)
A
breakdown
in
trade
\
\
jump
\"
mentioned
in
the
passage
refers
to
the
conquest
of
the
New
World
and
the
\
\
routes
through
the
collapse
of
the
established
state
structure
(B)
An
increase
\
\
Columbian
Exchange.
Choice
(A)
and
(C)
did
not
happen
in
history.
Choice
(C)
refers
\
\
in
the
population
of
the
world
through
more
plentiful
supplies
of
food
(C)
\
\
to
the
human
assets.
The
answer
is
(B).
\n\n
Q:
This
question
refers
to
the
following
\
\
The
spread
of
Chinese
and
Indian
belief
systems
across
the
world
(D)
An
increase
\
\
information.
\n\"
The
struggle
against
neo-colonialism
is
not
aimed
at
excluding
\
\
in
social
unrest"
\
the
capital
of
the
developed
world
from
operating
in
less
developed
countries.
\
target
:
Let's think step by step. We refer to Wikipedia articles on world history
\
It
is
aimed
at
preventing
the
financial
power
of
the
developed
countries
being
\
for help. The "quantum jump" mentioned in the passage refers to the conquest
\
used
in
such
a
way
as
to
impoverish
the
less
developed.
\n
Non-alignment,
as
practiced
\
of the New World and the Columbian Exchange. Choice (A) and (C) did not happen
\
by
Ghana
and
many
other
countries,
is
based
on
co-operation
with
all
States
whether
\
in history. Choice (C) refers to the human assets. The answer is (B).
\
they
be
capitalist,
socialist
or
have
a
mixed
economy.
Such
a
policy,
therefore,
\
-
question
:
"
This
question
refers
to
the
following
information.
\n\"
The
struggle
against
\
\
involves
foreign
investment
from
capitalist
countries,
but
it
must
be
invested
\
\
neo-colonialism
is
not
aimed
at
excluding
the
capital
of
the
developed
world
\
\
in
accordance
with
a
national
plan
drawn
up
by
the
government
of
the
non-aligned
\
\
from
operating
in
less
developed
countries.
It
is
aimed
at
preventing
the
\
\
State
with
its
own
interests
in
mind.
The
issue
is
not
what
return
the
foreign
\
\
financial
power
of
the
developed
countries
being
used
in
such
a
way
as
to
\
\
investor
receives
on
his
investments…The
question
is
one
of
power.
A
State
in
\
\
impoverish
the
less
developed.
\n
Non-alignment,
as
practiced
by
Ghana
and
many
\
\
the
grip
of
neo-colonialism
is
not
master
of
its
own
destiny.
\"\n
Kwame
Nkrumah,
\
\
other
countries,
is
based
on
co-operation
with
all
States
whether
they
be
\
\
Neo-Colonialism,
1965
\n
Which
of
the
following
provides
the
best
context
for
Nkrumah's
\
\
capitalist,
socialist
or
have
a
mixed
economy.
Such
a
policy,
therefore,
involves
\
\
writings?
\n
(A)
The
Industrial
Revolution
(B)
Decolonization
(C)
Regional
Free
\
\
foreign
investment
from
capitalist
countries,
but
it
must
be
invested
in
accordance
\
\
Trade
Associations
(D)
Autarky
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
\
with
a
national
plan
drawn
up
by
the
government
of
the
non-aligned
State
with
\
\
articles
on
world
history
for
help.
The
passage
expresses
a
point
that
the
successful
\
\
its
own
interests
in
mind.
The
issue
is
not
what
return
the
foreign
investor
\
\
fight
against
neo-colonialism
were
in
danger
and
the
newly
independent
nations
\
\
receives
on
his
investments
\u2026
The
question
is
one
of
power.
A
State
in
\
\
like
Ghana
may
be
re-colonized
via
financial
power
of
the
developed
countries.
\
\
the
grip
of
neo-colonialism
is
not
master
of
its
own
destiny.
\"\n
Kwame
Nkrumah,
\
\
The
answer
is
(B).
\n\n
Q:
This
question
refers
to
the
following
information.
\n\"\
\
Neo-Colonialism,
1965
\n
Which
of
the
following
provides
the
best
context
for
\
Indeed,
as
both
the
fatwas
of
distinguished
[scholars]
who
base
their
opinion
on
\
\
Nkrumah's
writings?
\n
(A)
The
Industrial
Revolution
(B)
Decolonization
(C)
\
\
reason
and
tradition
alike
and
the
consensus
of
the
Sunni
community
agree
that
\
\
Regional
Free
Trade
Associations
(D)
Autarky"
\
the
ancient
obligation
of
extirpation,
extermination,
and
expulsion
of
evil
innovation
\
target
:
Let's think step by step. We refer to Wikipedia articles on world history
\
must
be
the
aim
of
our
exalted
aspiration,
for
\"
Religious
zeal
is
a
victory
for
\
for help. The passage expresses a point that the successful fight against neo-colonialism
\
the
Faith
of
God
the
Beneficent
\"
;
then,
in
accordance
with
the
words
of
the
Prophet
\
were in danger and the newly independent nations like Ghana may be re-colonized
\
(Peace
upon
him!)
\"
Whosoever
introduces
evil
innovation
into
our
order
must
be
\
via financial power of the developed countries. The answer is (B).
\
expelled
\"
and
\"
Whosoever
does
aught
against
our
order
must
be
expelled,
\"
action
\
-
question
:
"
This
question
refers
to
the
following
information.
\n\"
Indeed,
as
both
\
\
has
become
necessary
and
exigent…
\"\n
Letter
from
Ottoman
Sultan
Selim
I
to
Safavid
\
\
the
fatwas
of
distinguished
[scholars]
who
base
their
opinion
on
reason
and
\
\
Shah
Ismail
I,
1514
\n
The
letter
from
Selim
I
is
most
clearly
an
example
of
which
\
\
tradition
alike
and
the
consensus
of
the
Sunni
community
agree
that
the
ancient
\
\
of
the
following?
\n
(A)
The
maintenance
of
military
supremacy
at
all
costs
(B)
\
\
obligation
of
extirpation,
extermination,
and
expulsion
of
evil
innovation
\
\
Expanding
tensions
between
religious
sects
(C)
Factors
that
brought
about
the
\
\
must
be
the
aim
of
our
exalted
aspiration,
for
\"
Religious
zeal
is
a
victory
\
\
collapse
of
the
Ottoman
Empire
(D)
Peacemaking
efforts
among
the
Islamic
empires
\n\
\
for
the
Faith
of
God
the
Beneficent
\"
;
then,
in
accordance
with
the
words
\
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
world
history
for
\
\
of
the
Prophet
(Peace
upon
him!)
\"
Whosoever
introduces
evil
innovation
into
\
\
help.
The
passage
is
an
example
of
expanding
tensions
between
Selim
and
Ismail.
\
\
our
order
must
be
expelled
\"
and
\"
Whosoever
does
aught
against
our
order
\
\
In
the
passage
the
Selim
references
the
fatwa
and
the
consensus
of
the
Sunni
community
\
\
must
be
expelled,
\"
action
has
become
necessary
and
exigent
\u2026\"\n
Letter
\
\
to
against
whosoever
introduces
evil.
The
answer
is
(B).
\n\n
Q:
This
question
refers
\
\
from
Ottoman
Sultan
Selim
I
to
Safavid
Shah
Ismail
I,
1514
\n
The
letter
from
\
\
to
the
following
information.
\n\"
The
real
grievance
of
the
worker
is
the
insecurity
\
\
Selim
I
is
most
clearly
an
example
of
which
of
the
following?
\n
(A)
The
maintenance
\
\
of
his
existence;
he
is
not
sure
that
he
will
always
have
work,
he
is
not
sure
\
\
of
military
supremacy
at
all
costs
(B)
Expanding
tensions
between
religious
\
\
that
he
will
always
be
healthy,
and
he
foresees
that
he
will
one
day
be
old
and
\
\
sects
(C)
Factors
that
brought
about
the
collapse
of
the
Ottoman
Empire
(D)
\
\
unfit
to
work.
If
he
falls
into
poverty,
even
if
only
through
a
prolonged
illness,
\
\
Peacemaking
efforts
among
the
Islamic
empires"
\
he
is
then
completely
helpless,
exam_ins
to
his
own
devices,
and
society
does
\
target
:
Let's think step by step. We refer to Wikipedia articles on world history
\
not
currently
recognize
any
real
obligation
towards
him
beyond
the
usual
help
\
for help. The passage is an example of expanding tensions between Selim and
\
for
the
poor,
even
if
he
has
been
working
all
the
time
ever
so
faithfully
and
\
Ismail. In the passage the Selim references the fatwa and the consensus of the
\
diligently.
The
usual
help
for
the
poor,
however,
leaves
a
lot
to
be
desired,
\
Sunni community to against whosoever introduces evil. The answer is (B).
\
especially
in
large
cities,
where
it
is
very
much
worse
than
in
the
country.
\"\
-
question
:
'
This
question
refers
to
the
following
information.
\n
Otto
von
Bismarck,
1884
\n
Otto
von
Bismarck
likely
made
this
speech
in
reaction
\
\
to
which
of
the
following
issues?
\n
(A)
Social
acceptance
of
child
labor
(B)
Declining
\
"The
real
grievance
of
the
worker
is
the
insecurity
of
his
existence;
he
is
\
life
expectancy
in
Germany
(C)
Criticisms
of
German
trade
tariffs
(D)
Negative
\
not
sure
that
he
will
always
have
work,
he
is
not
sure
that
he
will
always
be
\
effects
attributed
to
industrial
capitalism
\n
A:
Let's
think
step
by
step.
We
refer
\
healthy,
and
he
foresees
that
he
will
one
day
be
old
and
unfit
to
work.
If
he
\
to
Wikipedia
articles
on
world
history
for
help.
The
passage
talks
about
the
grievance
\
falls
into
poverty,
even
if
only
through
a
prolonged
illness,
he
is
then
completely
\
of
the
work
under
the
industrial
capitalism.
The
answer
is
(D).
\n\n
Q:
This
question
\
helpless,
exam_ins
to
his
own
devices,
and
society
does
not
currently
recognize
\
refers
to
the
following
information.
\n
He
contains
all
works
and
desires
and
all
\
any
real
obligation
towards
him
beyond
the
usual
help
for
the
poor,
even
if
\
perfumes
and
all
tastes.
He
enfolds
the
whole
universe
and
in
silence
is
loving
\
he
has
been
working
all
the
time
ever
so
faithfully
and
diligently.
The
usual
\
to
all.
This
is
the
Spirit
that
is
in
my
heart,
this
is
Brahman.
To
him
I
shall
\
help
for
the
poor,
however,
leaves
a
lot
to
be
desired,
especially
in
large
\
come
when
I
go
beyond
this
life,
and
to
him
will
come
he
who
has
faith
and
doubts
\
cities,
where
it
is
very
much
worse
than
in
the
country."
\
not.
\n
—The
Upanishads,
India,
c.
1000
BCE
\n
To
which
religion
does
the
speaker
\
\
most
likely
belong?
\n
(A)
Hinduism
(B)
Buddhism
(C)
Shintoism
(D)
Zoroastrianism
\n\
Otto
von
Bismarck,
1884
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
world
history
for
\
\
help.
Brahman
refers
to
the
ultimate
reality
of
all
things
in
the
Hindu
religion.
\
Otto
von
Bismarck
likely
made
this
speech
in
reaction
to
which
of
the
following
\
In
contrast,
Buddhism
does
not
have
a
concept
of
supreme
God.
The
answer
is
(A).
\n\n
"
issues?
"
group"
:
"
mmlu_flan_cot_fewshot_humanities"
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
(A)
Social
acceptance
of
child
labor
(B)
Declining
life
expectancy
in
Germany
"
task"
:
"
mmlu_flan_cot_fewshot_high_school_world_history"
(C)
Criticisms
of
German
trade
tariffs
(D)
Negative
effects
attributed
to
industrial
capitalism'
target
:
Let's think step by step. We refer to Wikipedia articles on world history
for help. The passage talks about the grievance of the work under the industrial
capitalism. The answer is (D).
-
question
:
"
This
question
refers
to
the
following
information.
\n
He
contains
all
works
\
\
and
desires
and
all
perfumes
and
all
tastes.
He
enfolds
the
whole
universe
\
\
and
in
silence
is
loving
to
all.
This
is
the
Spirit
that
is
in
my
heart,
this
\
\
is
Brahman.
To
him
I
shall
come
when
I
go
beyond
this
life,
and
to
him
will
\
\
come
he
who
has
faith
and
doubts
not.
\n\u2014
The
Upanishads,
India,
c.
1000
\
\
BCE
\n
To
which
religion
does
the
speaker
most
likely
belong?
\n
(A)
Hinduism
\
\
(B)
Buddhism
(C)
Shintoism
(D)
Zoroastrianism"
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
world
history
for
help.
Brahman
refers
to
the
ultimate
reality
of
all
things
in
the
Hindu
religion.
In
contrast,
Buddhism
does
not
have
a
concept
of
supreme
God.
The
answer
is
(A).'
group
:
mmlu_flan_cot_fewshot_humanities
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_high_school_world_history
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_aging.yaml
View file @
da211969
"
dataset_name"
:
"
human_aging"
dataset_name
:
human_aging
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
human
\
description
:
The following are multiple choice questions (with answers) about human
\
aging.
\n\n
Q:
All
other
things
being
equal,
which
of
the
following
persons
is
more
\
aging.
\
likely
to
show
osteoporosis?
\n
(A)
An
older
Hispanic
American
woman
(B)
An
older
\
fewshot_config
:
\
African
American
woman
(C)
An
older
Asian
American
woman
(D)
An
older
Native
American
\
sampler
:
first_n
\
woman
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
human
aging
\
samples
:
\
for
help.
Although
osteoporosis
can
occur
at
any
age,
the
risk
is
higher
for
older
\
-
question
:
'
All
other
things
being
equal,
which
of
the
following
persons
is
more
likely
\
people.
It
is
most
common
in
Asian
and
non-Hispanic
white
women.
The
answer
is
\
to
show
osteoporosis?
\
(C).
\n\n
Q:
The
finding
that
adults
tend
to
remember
events
from
their
adolescence
\
\
better
than
from
other
periods
in
their
lives
is
referred
to
as
the
\n
(A)
Adolescence
\
(A)
An
older
Hispanic
American
woman
(B)
An
older
African
American
woman
(C)
\
advantage
(B)
Reminiscence
bump
(C)
Memorial
memorial
(D)
Quadratic
retrieval
\
An
older
Asian
American
woman
(D)
An
older
Native
American
woman'
\
spike
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
human
aging
\
target
:
Let's think step by step. We refer to Wikipedia articles on human aging
\
for
help.
Reminiscence
bump
is
a
phenomenon
that
older
adults
tend
to
recollect
\
for help. Although osteoporosis can occur at any age, the risk is higher for
\
events
during
their
young
ages.
People
usually
have
a
period
of
childhood
amnesia
\
older people. It is most common in Asian and non-Hispanic white women. The answer
\
from
birth
to
around
age
5,
and
a
reminiscence
bump
between
10
and
30.
The
answer
\
is (C).
\
is
(B).
\n\n
Q:
Which
element
in
tobacco
smoke
is
responsible
for
cancers?
\n
(A)
\
-
question
:
'
The
finding
that
adults
tend
to
remember
events
from
their
adolescence
\
Nicotine
(B)
Tar
(C)
Carbon
monoxide
(D)
Smoke
particles
\n
A:
Let's
think
step
\
better
than
from
other
periods
in
their
lives
is
referred
to
as
the
\
by
step.
We
refer
to
Wikipedia
articles
on
human
aging
for
help.
The
benzene,
\
\
acrylamide
and
acrylonitrile
in
tar
interact
with
the
lungs
and
cause
DNA
mutations
\
(A)
Adolescence
advantage
(B)
Reminiscence
bump
(C)
Memorial
memorial
(D)
Quadratic
\
in
cells
of
the
lungs,
and
lead
to
cancer.
The
answer
is
(B).
\n\n
Q:
When
older
\
retrieval
spike'
\
adults
move
to
a
new
state
after
retirement,
which
of
the
following
is
the
more
\
target
:
Let's think step by step. We refer to Wikipedia articles on human aging
\
likely
destination?
\n
(A)
Texas
(B)
California
(C)
Hawaii
(D)
Vermont
\n
A:
Let's
\
for help. Reminiscence bump is a phenomenon that older adults tend to recollect
\
think
step
by
step.
We
refer
to
Wikipedia
articles
on
human
aging
for
help.
Texas
\
events during their young ages. People usually have a period of childhood amnesia
\
does
not
have
state
tax,
and
has
low
cost
of
living
compared
with
the
other
three
\
from birth to around age 5, and a reminiscence bump between 10 and 30. The answer
\
options.
The
answer
is
(A).
\n\n
"
is (B).
"
group"
:
"
mmlu_flan_cot_fewshot_other"
-
question
:
'
Which
element
in
tobacco
smoke
is
responsible
for
cancers?
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
"
task"
:
"
mmlu_flan_cot_fewshot_human_aging"
(A)
Nicotine
(B)
Tar
(C)
Carbon
monoxide
(D)
Smoke
particles'
target
:
Let's think step by step. We refer to Wikipedia articles on human aging
for help. The benzene, acrylamide and acrylonitrile in tar interact with the
lungs and cause DNA mutations in cells of the lungs, and lead to cancer. The
answer is (B).
-
question
:
'
When
older
adults
move
to
a
new
state
after
retirement,
which
of
the
following
is
the
more
likely
destination?
(A)
Texas
(B)
California
(C)
Hawaii
(D)
Vermont'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
human
aging
for
help.
Texas
does
not
have
state
tax,
and
has
low
cost
of
living
compared
with
the
other
three
options.
The
answer
is
(A).'
group
:
mmlu_flan_cot_fewshot_other
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_human_aging
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_human_sexuality.yaml
View file @
da211969
"
dataset_name"
:
"
human_sexuality"
dataset_name
:
human_sexuality
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
human
\
description
:
The following are multiple choice questions (with answers) about human
\
sexuality.
\n\n
Q:
The
most
common
disorder
among
men
who
seek
sexual
therapy
is:
\n\
sexuality.
(A)
premature
ejaculation
(B)
inhibited
ejaculation
(C)
erectile
disorder
(D)
ejaculatory
\
fewshot_config
:
\
disorder
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
human
\
sampler
:
first_n
\
sexuality
for
help.
The
most
common
type
of
arousal
disorder
for
men
is
erectile
\
samples
:
\
dysfunction,
meaning
that
a
person
is
not
able
to
get
any
physical
satisfaction
\
-
question
:
'
The
most
common
disorder
among
men
who
seek
sexual
therapy
is:
\
from
sexual
activity
although
he
may
be
interested
in
it.
The
answer
is
(C).
\n\
\n
Q:
A
woman
who
knows
she
has
active
herpes
and
untreated
syphilis
but
continues
\
(A)
premature
ejaculation
(B)
inhibited
ejaculation
(C)
erectile
disorder
(D)
\
to
have
sex
without
informing
her
partners
of
her
condition
has,
in
psychoanalytic
\
ejaculatory
disorder'
\
terms:
\n
(A)
a
strong
ego
(B)
a
weak
superego
(C)
a
weak
id
(D)
a
strong
superego
\n\
target
:
Let's think step by step. We refer to Wikipedia articles on human sexuality
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
human
sexuality
for
\
for help. The most common type of arousal disorder for men is erectile dysfunction,
\
help.
A
person
with
weak
superego
tends
to
be
delinquent,
criminal
or
have
antisocial
\
meaning that a person is not able to get any physical satisfaction from sexual
\
personality.
The
action
of
the
woman
who
knows
she
has
active
venereal
disease
\
activity although he may be interested in it. The answer is (C).
\
but
still
have
sex
with
her
partners
indicate
she
may
has
antisocial
personality.
\
-
question
:
'
A
woman
who
knows
she
has
active
herpes
and
untreated
syphilis
but
continues
\
The
answer
is
(B).
\n\n
Q:
The
nature
of
homosexual
activities
that
occur
during
\
to
have
sex
without
informing
her
partners
of
her
condition
has,
in
psychoanalytic
\
preadolescence
include
all
but
which
of
the
following?
\n
(A)
sexual
intercourse
\
terms:
\
(B)
circle
jerks
(C)
exhibitionism
(D)
touching
each
other's
genitals
\n
A:
Let's
\
\
think
step
by
step.
We
refer
to
Wikipedia
articles
on
human
sexuality
for
help.
\
(A)
a
strong
ego
(B)
a
weak
superego
(C)
a
weak
id
(D)
a
strong
superego'
\
Sexual
intercourse
is
prohibited
during
preadolescence.
The
answer
is
(A).
\n\n\
target
:
Let's think step by step. We refer to Wikipedia articles on human sexuality
Q:
Women's
ability
to
have
multiple
orgasms
is
primarily
due
to:
\n
(A)
the
fact
that
\
for help. A person with weak superego tends to be delinquent, criminal or have
\
they
do
not
have
a
refractory
period.
(B)
the
response
of
the
inner
layers
of
\
antisocial personality. The action of the woman who knows she has active venereal
\
the
vagina.
(C)
having
alternating
orgasms
in
different
locations.
(D)
the
G-Spot.
\n\
disease but still have sex with her partners indicate she may has antisocial
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
human
sexuality
for
\
personality. The answer is (B).
\
help.
The
refractory
period
is
the
time
when
a
person
is
not
able
to
be
erect
\
-
question
:
'
The
nature
of
homosexual
activities
that
occur
during
preadolescence
include
\
or
is
not
interested
in
sex.
The
answer
is
(A).
\n\n
Q:
Morning
sickness
is
typically
\
all
but
which
of
the
following?
\
a
problem:
\n
(A)
during
the
first
trimester
(B)
during
the
second
trimester
(C)
\
\
during
the
third
trimester
(D)
all
through
the
pregnancy
\n
A:
Let's
think
step
\
(A)
sexual
intercourse
(B)
circle
jerks
(C)
exhibitionism
(D)
touching
each
\
by
step.
We
refer
to
Wikipedia
articles
on
human
sexuality
for
help.
Morning
sickness
\
other'
'
s
genitals'
\
usually
begins
by
nine
weeks
after
conception,
corresponding
to
the
first
trimester.
\
target
:
Let's think step by step. We refer to Wikipedia articles on human sexuality
\
The
answer
is
(A).
\n\n
"
for help. Sexual intercourse is prohibited during preadolescence. The answer
"
group"
:
"
mmlu_flan_cot_fewshot_social_sciences"
is (A).
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
-
question
:
'
Women'
'
s
ability
to
have
multiple
orgasms
is
primarily
due
to:
"
task"
:
"
mmlu_flan_cot_fewshot_human_sexuality"
(A)
the
fact
that
they
do
not
have
a
refractory
period.
(B)
the
response
of
the
inner
layers
of
the
vagina.
(C)
having
alternating
orgasms
in
different
locations.
(D)
the
G-Spot.'
target
:
Let's think step by step. We refer to Wikipedia articles on human sexuality
for help. The refractory period is the time when a person is not able to be
erect or is not interested in sex. The answer is (A).
-
question
:
'
Morning
sickness
is
typically
a
problem:
(A)
during
the
first
trimester
(B)
during
the
second
trimester
(C)
during
the
third
trimester
(D)
all
through
the
pregnancy'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
human
sexuality
for
help.
Morning
sickness
usually
begins
by
nine
weeks
after
conception,
corresponding
to
the
first
trimester.
The
answer
is
(A).'
group
:
mmlu_flan_cot_fewshot_social_sciences
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_human_sexuality
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_international_law.yaml
View file @
da211969
"
dataset_name"
:
"
international_law"
dataset_name
:
international_law
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
international
\
description
:
The following are multiple choice questions (with answers) about international
\
law.
\n\n
Q:
How
the
consent
to
be
bound
of
a
State
may
be
expressed?
\n
(A)
The
consent
\
law.
\
of
a
State
to
be
bound
is
expressed
only
by
ratification
(B)
The
consent
of
a
\
fewshot_config
:
\
state
to
be
bound
by
a
treaty
may
be
expressed
by
signature,
ratification,
acceptance,
\
sampler
:
first_n
\
approval
or
accession
(C)
The
consent
of
a
State
to
be
bound
is
expressed
by
signature
\
samples
:
\
(D)
The
consent
of
a
State
to
be
bound
is
expressed
by
whatever
means
they
choose
\n\
-
question
:
'
How
the
consent
to
be
bound
of
a
State
may
be
expressed?
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
international
law
\
\
for
help.
Article
11
of
Vienna
Convention
on
the
Law
of
Treaties
signed
in
1969
\
(A)
The
consent
of
a
State
to
be
bound
is
expressed
only
by
ratification
(B)
\
states
that
\"
the
consent
of
a
State
to
be
bound
by
a
treaty
may
be
expressed
\
The
consent
of
a
state
to
be
bound
by
a
treaty
may
be
expressed
by
signature,
\
by
signature,
exchange
of
instruments
constituting
a
treaty,
ratification,
acceptance,
\
ratification,
acceptance,
approval
or
accession
(C)
The
consent
of
a
State
to
\
approval
or
accession,
or
by
any
other
means
if
so
agreed.
\"
(B)
is
the
most
precise
\
be
bound
is
expressed
by
signature
(D)
The
consent
of
a
State
to
be
bound
is
\
and
accurate
answer.
The
answer
is
(B).
\n\n
Q:
What
is
the
judge
ad
hoc?
\n
(A)
If
\
expressed
by
whatever
means
they
choose'
\
a
party
to
a
contentious
case
before
the
ICJ
does
not
have
a
national
sitting
\
target
:
Let's think step by step. We refer to Wikipedia articles on international
\
as
judge,
it
is
entitled
to
nominate
someone
as
a
judge
solely
for
that
case,
\
law for help. Article 11 of Vienna Convention on the Law of Treaties signed
\
with
the
title
of
judge
ad
hoc
(B)
Judge
ad
hoc
is
the
member
of
the
bench
of
\
in 1969 states that "the consent of a State to be bound by a treaty may be expressed
\
the
ICJ
with
a
casting
vote
(C)
Judge
ad
hoc
is
a
surrogate
judge,
in
case
a
judge
\
by signature, exchange of instruments constituting a treaty, ratification, acceptance,
\
is
disqualified
or
passes
away
(D)
Judge
ad
hoc
is
the
judge
that
each
party
will
\
approval or accession, or by any other means if so agreed." (B) is the most
\
always
nominate
in
every
contentious
case
\n
A:
Let's
think
step
by
step.
We
refer
\
precise and accurate answer. The answer is (B).
\
to
Wikipedia
articles
on
international
law
for
help.
As
\"
ad
hoc
\"
implies,
a
\
-
question
:
'
What
is
the
judge
ad
hoc?
\
judge
ad
hoc
is
appointed
only
for
a
specific
case
or
period,
when
a
party
to
\
\
a
contentious
case
before
the
International
Court
of
Justice
does
not
have
a
regular
\
(A)
If
a
party
to
a
contentious
case
before
the
ICJ
does
not
have
a
national
\
national
sitting
as
judge.
The
answer
is
(A).
\n\n
Q:
When
'consent'
can
serve
as
\
sitting
as
judge,
it
is
entitled
to
nominate
someone
as
a
judge
solely
for
that
\
a
circumstance
precluding
the
wrongfulness
of
a
State
conduct?
\n
(A)
Consent
can
\
case,
with
the
title
of
judge
ad
hoc
(B)
Judge
ad
hoc
is
the
member
of
the
bench
\
serve
as
a
circumstance
precluding
the
wrongfulness
whenever
it
is
given
(B)
Consent
\
of
the
ICJ
with
a
casting
vote
(C)
Judge
ad
hoc
is
a
surrogate
judge,
in
case
\
can
never
serve
as
a
circumstance
precluding
wrongfulness
(C)
Consent
can
serve
\
a
judge
is
disqualified
or
passes
away
(D)
Judge
ad
hoc
is
the
judge
that
each
\
as
a
circumstance
precluding
wrongfulness,
provided
the
consent
is
valid
and
to
\
party
will
always
nominate
in
every
contentious
case'
\
the
extent
that
the
conduct
remains
within
the
limits
of
the
consent
given
(D)
\
target
:
Let's think step by step. We refer to Wikipedia articles on international
\
Consent
can
always
serve
as
a
circumstance
precluding
wrongfulness,
no
matter
\
law for help. As "ad hoc" implies, a judge ad hoc is appointed only for a specific
\
which
organ
of
the
State
gives
it
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
case or period, when a party to a contentious case before the International
\
articles
on
international
law
for
help.
Valid
consent
can
serve
as
a
circumstance
\
Court of Justice does not have a regular national sitting as judge. The answer
\
precluding
the
wrongfulness
of
a
State
conduct
if
the
conduct
remains
within
the
\
is (A).
\
limits
of
that
consent,
according
to
Chapter
V
of
the
Responsibility
of
States
\
-
question
:
'
When
'
'
consent'
'
can
serve
as
a
circumstance
precluding
the
wrongfulness
\
for
Internationally
Wrongful
Acts,
2001,
United
Nations.
The
answer
is
(C).
\n\n\
of
a
State
conduct?
Q:
Would
a
reservation
to
the
definition
of
torture
in
the
ICCPR
be
acceptable
in
\
\
contemporary
practice?
\n
(A)
This
is
an
acceptable
reservation
if
the
reserving
\
(A)
Consent
can
serve
as
a
circumstance
precluding
the
wrongfulness
whenever
\
country's
legislation
employs
a
different
definition
(B)
This
is
an
unacceptable
\
it
is
given
(B)
Consent
can
never
serve
as
a
circumstance
precluding
wrongfulness
\
reservation
because
it
contravenes
the
object
and
purpose
of
the
ICCPR
(C)
This
\
(C)
Consent
can
serve
as
a
circumstance
precluding
wrongfulness,
provided
the
\
is
an
unacceptable
reservation
because
the
definition
of
torture
in
the
ICCPR
\
consent
is
valid
and
to
the
extent
that
the
conduct
remains
within
the
limits
\
is
consistent
with
customary
international
law
(D)
This
is
an
acceptable
reservation
\
of
the
consent
given
(D)
Consent
can
always
serve
as
a
circumstance
precluding
\
because
under
general
international
law
States
have
the
right
to
enter
reservations
\
wrongfulness,
no
matter
which
organ
of
the
State
gives
it'
\
to
treaties
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
international
\
target
:
Let's think step by step. We refer to Wikipedia articles on international
\
law
for
help.
For
it
contravenes
the
object
and
purpose
of
the
ICCPR,
this
is
\
law for help. Valid consent can serve as a circumstance precluding the wrongfulness
\
an
unacceptable
reservation
in
contemporary
practice.
The
answer
is
(B).
\n\n
Q:
\
of a State conduct if the conduct remains within the limits of that consent,
\
What
types
of
force
does
Article
2(4)
of
the
UN
Charter
prohibit?
\n
(A)
Article
\
according to Chapter V of the Responsibility of States for Internationally Wrongful
\
2(4)
encompasses
only
armed
force
(B)
Article
2(4)
encompasses
all
types
of
force,
\
Acts, 2001, United Nations. The answer is (C).
\
including
sanctions
(C)
Article
2(4)
encompasses
all
interference
in
the
domestic
\
-
question
:
'
Would
a
reservation
to
the
definition
of
torture
in
the
ICCPR
be
acceptable
\
affairs
of
States
(D)
Article
2(4)
encompasses
force
directed
only
against
a
State's
\
in
contemporary
practice?
\
territorial
integrity
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
\
\
on
international
law
for
help.
Article
2(4)
of
the
UN
Charter
prohibits
states
\
(A)
This
is
an
acceptable
reservation
if
the
reserving
country'
'
s
legislation
\
from
using
armed
forces
in
their
international
relations.
The
answer
is
(A).
\n\n
"
employs
a
different
definition
(B)
This
is
an
unacceptable
reservation
because
"
group"
:
"
mmlu_flan_cot_fewshot_humanities"
it
contravenes
the
object
and
purpose
of
the
ICCPR
(C)
This
is
an
unacceptable
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
reservation
because
the
definition
of
torture
in
the
ICCPR
is
consistent
with
"
task"
:
"
mmlu_flan_cot_fewshot_international_law"
customary
international
law
(D)
This
is
an
acceptable
reservation
because
under
general
international
law
States
have
the
right
to
enter
reservations
to
treaties'
target
:
Let's think step by step. We refer to Wikipedia articles on international
law for help. For it contravenes the object and purpose of the ICCPR, this is
an unacceptable reservation in contemporary practice. The answer is (B).
-
question
:
'
What
types
of
force
does
Article
2(4)
of
the
UN
Charter
prohibit?
(A)
Article
2(4)
encompasses
only
armed
force
(B)
Article
2(4)
encompasses
all
types
of
force,
including
sanctions
(C)
Article
2(4)
encompasses
all
interference
in
the
domestic
affairs
of
States
(D)
Article
2(4)
encompasses
force
directed
only
against
a
State'
'
s
territorial
integrity'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
international
law
for
help.
Article
2(4)
of
the
UN
Charter
prohibits
states
from
using
armed
forces
in
their
international
relations.
The
answer
is
(A).'
group
:
mmlu_flan_cot_fewshot_humanities
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_international_law
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_jurisprudence.yaml
View file @
da211969
"
dataset_name"
:
"
jurisprudence"
dataset_name
:
jurisprudence
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
jurisprudence.
\n\
description
:
The following are multiple choice questions (with answers) about jurisprudence.
\n
Q:
Iverson
Jewelers
wrote
a
letter
to
Miller,
'We
have
received
an
exceptionally
\
fewshot_config
:
\
fine
self
winding
Rolox
watch
which
we
will
sell
to
you
at
a
very
favorable
price.'
\n\
sampler
:
first_n
(A)
The
letter
is
an
offer
to
sell
(B)
A
valid
offer
cannot
be
made
by
letter.
(C)
\
samples
:
\
The
letter
contains
a
valid
offer
which
will
terminate
within
a
reasonable
time.
\
-
question
:
'
Iverson
Jewelers
wrote
a
letter
to
Miller,
'
'
We
have
received
an
exceptionally
\
(D)
The
letter
lacks
one
of
the
essential
elements
of
an
offer.
\n
A:
Let's
think
\
fine
self
winding
Rolox
watch
which
we
will
sell
to
you
at
a
very
favorable
\
step
by
step.
We
refer
to
Wikipedia
articles
on
jurisprudence
for
help.
An
offer
\
price.'
'
\
shows
the
intent
to
enter
into
a
mutually-beneficial
contract
with
specific
terms.
\
\
An
offer
can
be
made
by
a
letter.
While
this
letter
indicates
the
willingness
\
(A)
The
letter
is
an
offer
to
sell
(B)
A
valid
offer
cannot
be
made
by
letter.
\
to
sell,
the
lack
of
specific
terms,
such
as
transaction
price
and
offer
expiration
\
(C)
The
letter
contains
a
valid
offer
which
will
terminate
within
a
reasonable
\
date,
makes
it
an
incomplete
offer.
The
answer
is
(D).
\n\n
Q:
Functions
of
the
\
time.
(D)
The
letter
lacks
one
of
the
essential
elements
of
an
offer.'
\
law
include
all
but
which
of
the
following?
\n
(A)
maximizing
individual
freedom
\
target
:
Let's think step by step. We refer to Wikipedia articles on jurisprudence
\
(B)
providing
a
basis
for
compromise
(C)
keeping
the
peace
(D)
promoting
the
principles
\
for help. An offer shows the intent to enter into a mutually-beneficial contract
\
of
the
free
enterprise
system
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
with specific terms. An offer can be made by a letter. While this letter indicates
\
articles
on
jurisprudence
for
help.
Laws
are
fundamentally
about
helping
resolve
\
the willingness to sell, the lack of specific terms, such as transaction price
\
disputes
between
individuals,
and
therefore
essential
for
maximizing
individual
\
and offer expiration date, makes it an incomplete offer. The answer is (D).
\
freedom,
providing
a
basis
for
compromise,
and
keeping
the
peace.
The
answer
is
\
-
question
:
'
Functions
of
the
law
include
all
but
which
of
the
following?
\
(D).
\n\n
Q:
The
________
School
of
jurisprudence
postulates
that
the
law
is
based
\
\
on
what
is
\"
correct.
\"\n
(A)
Natural
Law
(B)
Analytical
(C)
Historical
(D)
Sociological
\n\
(A)
maximizing
individual
freedom
(B)
providing
a
basis
for
compromise
(C)
keeping
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
jurisprudence
for
\
the
peace
(D)
promoting
the
principles
of
the
free
enterprise
system'
\
help.
Natural
Law
School
of
jurisprudence
focuses
on
the
laws
of
nature,
and
states
\
target
:
Let's think step by step. We refer to Wikipedia articles on jurisprudence
\
that
the
law
should
be
based
on
ethics,
morals,
and
what
is
\"
correct
\"
.
Analytical
\
for help. Laws are fundamentally about helping resolve disputes between individuals,
\
deals
with
the
law
as
it
already
exists,
Historical
postulates
that
the
law
was
\
and therefore essential for maximizing individual freedom, providing a basis
\
found
and
not
made,
and
Sociological
studies
how
the
law
and
society
impact
each
\
for compromise, and keeping the peace. The answer is (D).
\
other.
The
answer
is
(A).
\n\n
Q:
Which
word
best
summarizes
Weber's
explanation
\
-
question
:
'
The
________
School
of
jurisprudence
postulates
that
the
law
is
based
\
of
the
development
of
formally
rational
law?
\n
(A)
Authority.
(B)
Charisma.
(C)
\
on
what
is
"correct."
\
Co-operation.
(D)
Capitalism.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
\
articles
on
jurisprudence
for
help.
Weber
explained
the
development
of
formal
\
(A)
Natural
Law
(B)
Analytical
(C)
Historical
(D)
Sociological'
\
rationality
in
laws
as
how
the
modern
society
moved
from
tradition
to
rationality,
\
target
:
Let's think step by step. We refer to Wikipedia articles on jurisprudence
\
where
people
decide
actions
based
less
on
how
they
were
culturally
done
and
more
\
for help. Natural Law School of jurisprudence focuses on the laws of nature,
\
on
expected
utilities.
How
rational
individuals
optimize
efficiency
of
accomplishing
\
and states that the law should be based on ethics, morals, and what is "correct".
\
tasks
for
higher
rewards
is
a
core
principle
of
Capitalism.
The
answer
is
(D).
\n\
Analytical deals with the law as it already exists, Historical postulates that
\n
Q:
Which
position
does
Rawls
claim
is
the
least
likely
to
be
adopted
by
the
POP
\
the law was found and not made, and Sociological studies how the law and society
\
(people
in
the
original
position)?
\n
(A)
The
POP
would
choose
equality
above
liberty.
\
impact each other. The answer is (A).
\
(B)
The
POP
would
opt
for
the
'maximin'
strategy.
(C)
The
POP
would
opt
for
the
\
-
question
:
'
Which
word
best
summarizes
Weber'
'
s
explanation
of
the
development
of
\
'difference
principle'.
(D)
The
POP
would
reject
the
'system
of
natural
liberty.'
\n\
formally
rational
law?
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
jurisprudence
for
\
\
help.
The
POP
would
opt
for
the
'maximin'
strategy,
opt
for
the
'difference
principle',
\
(A)
Authority.
(B)
Charisma.
(C)
Co-operation.
(D)
Capitalism.'
\
and
reject
the
'system
of
natural
liberty',
but
the
POP
would
not
choose
equality
\
target
:
Let's think step by step. We refer to Wikipedia articles on jurisprudence
\
above
liberty,
since
the
POP
assume
both
equal
and
free
citizens.
The
answer
is
\
for help. Weber explained the development of formal rationality in laws as how
\
(A).
\n\n
"
the modern society moved from tradition to rationality, where people decide
"
group"
:
"
mmlu_flan_cot_fewshot_humanities"
actions based less on how they were culturally done and more on expected utilities.
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
How rational individuals optimize efficiency of accomplishing tasks for higher
"
task"
:
"
mmlu_flan_cot_fewshot_jurisprudence"
rewards is a core principle of Capitalism. The answer is (D).
-
question
:
'
Which
position
does
Rawls
claim
is
the
least
likely
to
be
adopted
by
the
POP
(people
in
the
original
position)?
(A)
The
POP
would
choose
equality
above
liberty.
(B)
The
POP
would
opt
for
the
'
'
maximin'
'
strategy.
(C)
The
POP
would
opt
for
the
'
'
difference
principle'
'
.
(D)
The
POP
would
reject
the
'
'
system
of
natural
liberty.'
'
'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
jurisprudence
for
help.
The
POP
would
opt
for
the
'
'
maximin'
'
strategy,
opt
for
the
'
'
difference
principle'
'
,
and
reject
the
'
'
system
of
natural
liberty'
'
,
but
the
POP
would
not
choose
equality
above
liberty,
since
the
POP
assume
both
equal
and
free
citizens.
The
answer
is
(A).'
group
:
mmlu_flan_cot_fewshot_humanities
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_jurisprudence
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_logical_fallacies.yaml
View file @
da211969
"
dataset_name"
:
"
logical_fallacies"
dataset_name
:
logical_fallacies
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
logical
\
description
:
The following are multiple choice questions (with answers) about logical
\
fallacies.
\n\n
Q:
When
an
arguer
causes
confusion
during
refutation
because
of
\
fallacies.
\
real
or
feigned
lack
of
an
ability
to
engage
in
refutation,
that
arguer
may
have
\
fewshot_config
:
\
committed
the
fallacy
of
\n
(A)
poor
sportsmanship
(B)
appeal
to
compassion
(C)
\
sampler
:
first_n
\
argument
against
the
person
(D)
ignorance
of
refutation
\n
A:
Let's
think
step
by
\
samples
:
\
step.
We
refer
to
Wikipedia
articles
on
logical
fallacies
for
help.
Ignorance
\
-
question
:
'
When
an
arguer
causes
confusion
during
refutation
because
of
real
or
feigned
\
of
refutation,
one
of
Aristotle's
original
list
of
logical
fallacies
in
his
Organon,
\
lack
of
an
ability
to
engage
in
refutation,
that
arguer
may
have
committed
the
\
is
when
someone
causes
confusion
in
an
argument
through
real
or
feigned
inability
\
fallacy
of
\
to
engage
in
refutation,
in
order
to
win
the
argument.
The
answer
is
(D).
\n\n\
Q:
The
complex
question
fallacy
consists
of
\n
(A)
arguing
something
is
inferior
just
\
(A)
poor
sportsmanship
(B)
appeal
to
compassion
(C)
argument
against
the
person
\
because
it
doesn't
do
something
it
was
never
intended
to
do.
(B)
including
more
\
(D)
ignorance
of
refutation'
\
than
one
claim
in
the
proposition
and
treating
proof
for
one
claim
as
proof
for
\
target
:
Let's think step by step. We refer to Wikipedia articles on logical fallacies
\
all
the
claims.
(C)
drawing
a
conclusion
before
examining
the
evidence,
and
only
\
for help. Ignorance of refutation, one of Aristotle's original list of logical
\
considering
evidence
that
supports
that
conclusion.
(D)
asking
a
question
that
\
fallacies in his Organon, is when someone causes confusion in an argument through
\
includes
either
an
unproven
assumption
or
more
than
one
question,
thus
making
\
real or feigned inability to engage in refutation, in order to win the argument.
\
a
straightforward
yes
or
no
answer
meaningless.
\n
A:
Let's
think
step
by
step.
\
The answer is (D).
\
We
refer
to
Wikipedia
articles
on
logical
fallacies
for
help.
The
complex
question
\
-
question
:
'
The
complex
question
fallacy
consists
of
\
fallacy
is
when
someone
makes
a
single
yes
or
no
answer
to
a
question
meaningless,
\
\
by
including
either
an
unproven
assumption
or
many
questions.
The
latter
is
also
\
(A)
arguing
something
is
inferior
just
because
it
doesn'
'
t
do
something
it
was
\
known
as
the
many
questions
fallacy.
The
answer
is
(D).
\n\n
Q:
Arguing
that
what
\
never
intended
to
do.
(B)
including
more
than
one
claim
in
the
proposition
and
\
is
true
of
the
parts
must
be
true
of
the
whole
is
the
fallacy
of...
\n
(A)
Division
\
treating
proof
for
one
claim
as
proof
for
all
the
claims.
(C)
drawing
a
conclusion
\
(B)
Composition
(C)
Appeal
to
the
person
(D)
Appeal
to
ignorance
\n
A:
Let's
think
\
before
examining
the
evidence,
and
only
considering
evidence
that
supports
that
\
step
by
step.
We
refer
to
Wikipedia
articles
on
logical
fallacies
for
help.
Fallacy
\
conclusion.
(D)
asking
a
question
that
includes
either
an
unproven
assumption
\
of
composition
occurs
when
someone
argues
what
is
true
of
the
parts
must
be
true
\
or
more
than
one
question,
thus
making
a
straightforward
yes
or
no
answer
meaningless.'
\
of
the
whole.
The
answer
is
(B).
\n\n
Q:
Which
of
the
following
is
true
of
a
valid
\
target
:
Let's think step by step. We refer to Wikipedia articles on logical fallacies
\
categorical
syllogism?
\n
(A)
The
minor
premise
must
deny
the
antecedent
(B)
The
\
for help. The complex question fallacy is when someone makes a single yes or
\
major
premise
must
affirm
the
consequent
(C)
The
middle
term
must
be
used
in
at
\
no answer to a question meaningless, by including either an unproven assumption
\
least
one
premise
in
a
universal
or
unqualified
sense
(D)
All
of
the
above
\n
A:
\
or many questions. The latter is also known as the many questions fallacy. The
\
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
logical
fallacies
\
answer is (D).
\
for
help.
A
valid
categorical
syllogism
must
satisfy
several
conditions:
(1)
the
\
-
question
:
'
Arguing
that
what
is
true
of
the
parts
must
be
true
of
the
whole
is
the
\
syllogism
must
have
exactly
three
terms
(2)
every
term
of
the
syllogism
must
be
\
fallacy
of...
\
used
twice
exactly,
(3)
a
term
may
be
used
only
once
in
any
premise,
and
(4)
the
\
\
middle
term
must
be
used
in
at
least
one
premise
in
a
universal
or
unqualified
\
(A)
Division
(B)
Composition
(C)
Appeal
to
the
person
(D)
Appeal
to
ignorance'
\
sense,
etc.
Only
(C)
is
true.
The
answer
is
(C).
\n\n
Q:
If
someone
attacks
the
\
target
:
Let's think step by step. We refer to Wikipedia articles on logical fallacies
\
character
of
an
opposing
arguer,
instead
of
responding
to
that
opponent's
arguments,
\
for help. Fallacy of composition occurs when someone argues what is
true
of
\
the
first
person
has
probably
committed
which
of
the
following
fallacies?
\n
(A)
\
the parts must be
true
of the whole. The answer is (B).
\
tu
quoque
(B)
horse
laugh
(C)
argument
against
the
person
(D)
ignoratio
elenchi
\n\
-
question
:
'
Which
of
the
following
is
true
of
a
valid
categorical
syllogism?
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
logical
fallacies
\
\
for
help.
The
argument
against
the
person
fallacy
occurs
when
someone
irrelevantly
\
(A)
The
minor
premise
must
deny
the
antecedent
(B)
The
major
premise
must
affirm
\
attacks
the
character
of
an
opposing
arguer,
instead
of
addressing
that
opponent's
\
the
consequent
(C)
The
middle
term
must
be
used
in
at
least
one
premise
in
a
\
arguments.
The
answer
is
(C).
\n\n
"
universal
or
unqualified
sense
(D)
All
of
the
above'
"
group"
:
"
mmlu_flan_cot_fewshot_humanities"
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
logical
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
fallacies
for
help.
A
valid
categorical
syllogism
must
satisfy
several
conditions:
"
task"
:
"
mmlu_flan_cot_fewshot_logical_fallacies"
(1)
the
syllogism
must
have
exactly
three
terms
(2)
every
term
of
the
syllogism
must
be
used
twice
exactly,
(3)
a
term
may
be
used
only
once
in
any
premise,
and
(4)
the
middle
term
must
be
used
in
at
least
one
premise
in
a
universal
or
unqualified
sense,
etc.
Only
(C)
is
true.
The
answer
is
(C).'
-
question
:
'
If
someone
attacks
the
character
of
an
opposing
arguer,
instead
of
responding
to
that
opponent'
'
s
arguments,
the
first
person
has
probably
committed
which
of
the
following
fallacies?
(A)
tu
quoque
(B)
horse
laugh
(C)
argument
against
the
person
(D)
ignoratio
elenchi'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
logical
fallacies
for
help.
The
argument
against
the
person
fallacy
occurs
when
someone
irrelevantly
attacks
the
character
of
an
opposing
arguer,
instead
of
addressing
that
opponent'
'
s
arguments.
The
answer
is
(C).'
group
:
mmlu_flan_cot_fewshot_humanities
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_logical_fallacies
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_machine_learning.yaml
View file @
da211969
"
dataset_name"
:
"
machine_learning"
dataset_name
:
machine_learning
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
machine
\
description
:
The following are multiple choice questions (with answers) about machine
\
learning.
\n\n
Q:
Which
image
data
augmentation
is
most
common
for
natural
images?
\n\
learning.
(A)
random
crop
and
horizontal
flip
(B)
random
crop
and
vertical
flip
(C)
posterization
\
fewshot_config
:
\
(D)
dithering
\n
A:
Let's
think
step
by
step.
Data
augmentation
is
used
to
increase
\
sampler
:
first_n
\
the
diversity
of
images
in
the
training
dataset.
It
is
important
that
natural
\
samples
:
\
images
are
kept
natural
after
being
augmented.
Vertical
flips
of
images
are
not
\
-
question
:
'
Which
image
data
augmentation
is
most
common
for
natural
images?
\
natural,
so
(B)
is
false.
Posterization
makes
the
image
look
like
a
poster
and
\
\
and
dithering
increases
color
depth.
None
of
these
two
preserve
the
natural
property.
\
(A)
random
crop
and
horizontal
flip
(B)
random
crop
and
vertical
flip
(C)
posterization
\
The
only
natural
data
augmentation
technique
is
(A).
The
answer
is
(A).
\n\n
Q:
\
(D)
dithering'
\
Traditionally,
when
we
have
a
real-valued
input
attribute
during
decision-tree
\
target
:
Let's think step by step. Data augmentation is used to increase the diversity
\
learning
we
consider
a
binary
split
according
to
whether
the
attribute
is
above
\
of images in the training dataset. It is important that natural images are kept
\
or
below
some
threshold.
Pat
suggests
that
instead
we
should
just
have
a
multiway
\
natural after being augmented. Vertical flips of images are not natural, so
\
split
with
one
branch
for
each
of
the
distinct
values
of
the
attribute.
From
the
\
(B) is
false
. Posterization makes the image look like a poster and and dithering
\
list
below
choose
the
single
biggest
problem
with
Pat’s
suggestion:
\n
(A)
It
is
\
increases color depth. None of these two preserve the natural property. The
\
too
computationally
expensive.
(B)
It
would
probably
result
in
a
decision
tree
\
only natural data augmentation technique is (A). The answer is (A).
\
that
scores
badly
on
the
training
set
and
a
testset.
(C)
It
would
probably
result
\
-
question
:
"
Traditionally,
when
we
have
a
real-valued
question
attribute
during
decision-tree
\
\
in
a
decision
tree
that
scores
well
on
the
training
set
but
badly
on
a
testset.
\
\
learning
we
consider
a
binary
split
according
to
whether
the
attribute
is
\
\
(D)
It
would
probably
result
in
a
decision
tree
that
scores
well
on
a
testset
\
\
above
or
below
some
threshold.
Pat
suggests
that
instead
we
should
just
have
\
\
but
badly
on
a
training
set.
\n
A:
Let's
think
step
by
step.
Because
the
input
is
\
\
a
multiway
split
with
one
branch
for
each
of
the
distinct
values
of
the
attribute.
\
\
real
valued,
it
is
unlikely
that
the
same
values
appear
both
at
training
and
test
\
\
From
the
list
below
choose
the
single
biggest
problem
with
Pat
\u2019
s
suggestion:
\n\
\
time.
This
means
that
while
such
a
decision
tree
could
yield
good
performance
\
(A)
It
is
too
computationally
expensive.
(B)
It
would
probably
result
in
a
decision
\
\
on
the
training
data,
when
evaluated
on
the
test
data
it
will
perform
badly
because
\
\
tree
that
scores
badly
on
the
training
set
and
a
testset.
(C)
It
would
probably
\
\
the
decision
tree
won’t
know
what
to
do
with
numbers
that
did
not
appear
in
the
\
\
result
in
a
decision
tree
that
scores
well
on
the
training
set
but
badly
on
\
\
training
data.
The
answer
is
(C).
\n\n
Q:
You
are
reviewing
papers
for
the
World’s
\
\
a
testset.
(D)
It
would
probably
result
in
a
decision
tree
that
scores
well
\
\
Fanciest
Machine
Learning
Conference,
and
you
see
submissions
with
the
following
\
\
on
a
testset
but
badly
on
a
training
set."
\
claims.
Which
ones
would
you
consider
accepting?
\n
(A)
My
method
achieves
a
training
\
target
:
"
Let's
think
step
by
step.
Because
the
question
is
real
valued,
it
is
unlikely
\
\
error
lower
than
all
previous
methods!
(B)
My
method
achieves
a
test
error
lower
\
\
that
the
same
values
appear
both
at
training
and
test
time.
This
means
that
\
\
than
all
previous
methods!
(Footnote:
When
regularisation
parameter
λ
is
chosen
\
\
while
such
a
decision
tree
could
yield
good
performance
on
the
training
data,
\
\
so
as
to
minimise
test
error.)
(C)
My
method
achieves
a
test
error
lower
than
\
\
when
evaluated
on
the
test
data
it
will
perform
badly
because
the
decision
\
\
all
previous
methods!
(Footnote:
When
regularisation
parameter
λ
is
chosen
so
\
\
tree
won
\u2019
t
know
what
to
do
with
numbers
that
did
not
appear
in
the
training
\
\
as
to
minimise
cross-validaton
error.)
(D)
My
method
achieves
a
cross-validation
\
\
data.
The
answer
is
(C)."
\
error
lower
than
all
previous
methods!
(Footnote:
When
regularisation
parameter
\
-
question
:
"
You
are
reviewing
papers
for
the
World
\u2019
s
Fanciest
Machine
Learning
\
\
λ
is
chosen
so
as
to
minimise
cross-validaton
error.)
\n
A:
Let's
think
step
by
\
\
Conference,
and
you
see
submissions
with
the
following
claims.
Which
ones
\
\
step.
In
machine
learning,
we
train
with
some
data
and
fixed
hyperparameters
and
\
\
would
you
consider
accepting?
\n
(A)
My
method
achieves
a
training
error
lower
\
\
the
training
error
can
be
arbitrarily
low,
so
(A)
can’t
be
right.
Then,
one
compares
\
\
than
all
previous
methods!
(B)
My
method
achieves
a
test
error
lower
than
\
\
different
hyperparameters
by
selecting
the
model
with
the
lowest
cross-validation
\
\
all
previous
methods!
(Footnote:
When
regularisation
parameter
\u03BB
is
chosen
\
\
error,
this
means
that
(B)
and
(D)
are
not
the
right
procedure.
The
only
relevant
\
\
so
as
to
minimise
test
error.)
(C)
My
method
achieves
a
test
error
lower
than
\
\
number
after
these
is
the
test
error
and
thus
(C)
is
the
right
answer.
The
answer
\
\
all
previous
methods!
(Footnote:
When
regularisation
parameter
\u03BB
is
chosen
\
\
is
(C).
\n\n
Q:
A
6-sided
die
is
rolled
15
times
and
the
results
are:
side
1
comes
\
\
so
as
to
minimise
cross-validaton
error.)
(D)
My
method
achieves
a
cross-validation
\
\
up
0
times;
side
2:
1
time;
side
3:
2
times;
side
4:
3
times;
side
5:
4
times;
\
\
error
lower
than
all
previous
methods!
(Footnote:
When
regularisation
parameter
\
\
side
6:
5
times.
Based
on
these
results,
what
is
the
probability
of
side
3
coming
\
\ \u03BB
is
chosen
so
as
to
minimise
cross-validaton
error.)"
\
up
when
using
Add-1
Smoothing?
\n
(A)
2.0/15
(B)
1.0/7
(C)
3.0/16
(D)
1.0/5
\n
A:
\
target
:
"
Let's
think
step
by
step.
In
machine
learning,
we
train
with
some
data
\
\
Let's
think
step
by
step.
Add-1
smoothing
adds
the
value
of
one
to
the
different
\
\
and
fixed
hyperparameters
and
the
training
error
can
be
arbitrarily
low,
so
\
\
counts
and
then
normalizes
the
probabilities
accordingly.
The
counts
after
adding
\
\
(A)
can
\u2019
t
be
right.
Then,
one
compares
different
hyperparameters
by
selecting
\
\
one
will
be:
side
1
comes
up
1
time;
side
2:
2
times;
side
3:
3
times;
side
4:
\
\
the
model
with
the
lowest
cross-validation
error,
this
means
that
(B)
and
\
\
4
times;
side
5:
5
times;
side
6:
6
times.
The
number
of
sum
one
die
rolls
will
\
\
(D)
are
not
the
right
procedure.
The
only
relevant
number
after
these
is
the
\
\
be
21,
so
the
probability
of
drawing
a
three
is
3/21
=
1/7.
The
answer
is
(B).
\n\
\
test
error
and
thus
(C)
is
the
right
answer.
The
answer
is
(C)."
\n
Q:
To
achieve
an
0/1
loss
estimate
that
is
less
than
1
percent
of
the
true
0/1
\
-
question
:
'
A
6-sided
die
is
rolled
15
times
and
the
results
are:
side
1
comes
up
\
loss
(with
probability
95%),
according
to
Hoeffding's
inequality
the
IID
test
\
0
times;
side
2:
1
time;
side
3:
2
times;
side
4:
3
times;
side
5:
4
times;
\
set
must
have
how
many
examples?
\n
(A)
around
10
examples
(B)
around
100
examples
\
side
6:
5
times.
Based
on
these
results,
what
is
the
probability
of
side
3
coming
\
(C)
between
100
and
500
examples
(D)
more
than
1000
examples
\n
A:
Let's
think
step
\
up
when
using
Add-1
Smoothing?
\
by
step.
By
the
Hoeffding’s
inequality,
we
expect
that
with
95%
probability
the
\
\
in-sample
and
out-of-sample
errors
differ
by
epsilon
when
we
have
N
samples
if
\
(A)
2.0/15
(B)
1.0/7
(C)
3.0/16
(D)
1.0/5'
\
2
exp(-2
epsilon^2
N)<0.05,
this
implies
that
N
>
-1/(2*epsilon**2)
log
(
0.05/2
\
target
:
'
Let'
'
s
think
step
by
step.
Add-1
smoothing
adds
the
value
of
one
to
the
\
)=
log
(40)*5000.
Since
log(40)>1,
we
have
that
one
needs
more
than
1000
examples.
\
different
counts
and
then
normalizes
the
probabilities
accordingly.
The
counts
\
The
answer
is
(D).
\n\n
"
after
adding
one
will
be:
side
1
comes
up
1
time;
side
2:
2
times;
side
3:
3
"
group"
:
"
mmlu_flan_cot_fewshot_stem"
times;
side
4:
4
times;
side
5:
5
times;
side
6:
6
times.
The
number
of
sum
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
one
die
rolls
will
be
21,
so
the
probability
of
drawing
a
three
is
3/21
=
1/7.
"
task"
:
"
mmlu_flan_cot_fewshot_machine_learning"
The
answer
is
(B).'
-
question
:
'
To
achieve
an
0/1
loss
estimate
that
is
less
than
1
percent
of
the
true
0/1
loss
(with
probability
95%),
according
to
Hoeffding'
'
s
inequality
the
IID
test
set
must
have
how
many
examples?
(A)
around
10
examples
(B)
around
100
examples
(C)
between
100
and
500
examples
(D)
more
than
1000
examples'
target
:
"
Let's
think
step
by
step.
By
the
Hoeffding
\u2019
s
inequality,
we
expect
\
\
that
with
95%
probability
the
in-sample
and
out-of-sample
errors
differ
by
\
\
epsilon
when
we
have
N
samples
if
2
exp(-2
epsilon^2
N)<0.05,
this
implies
\
\
that
N
>
-1/(2*epsilon**2)
log
(
0.05/2
)=
log
(40)*5000.
Since
log(40)>1,
\
\
we
have
that
one
needs
more
than
1000
examples.
The
answer
is
(D).
\n\n
"
group
:
mmlu_flan_cot_fewshot_stem
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_machine_learning
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_management.yaml
View file @
da211969
"
dataset_name"
:
"
management"
dataset_name
:
management
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
management.
\n\
description
:
The following are multiple choice questions (with answers) about management.
\n
Q:
How
can
organisational
structures
that
are
characterised
by
democratic
and
\
fewshot_config
:
\
inclusive
styles
of
management
be
described?
\n
(A)
Hierarchical
(B)
Bureaucratic
\
sampler
:
first_n
\
(C)
Flat
(D)
Functional
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
\
samples
:
\
on
management
for
help.
Flat
organizational
structures
are
characterized
by
democratic
\
-
question
:
'
How
can
organisational
structures
that
are
characterised
by
democratic
\
and
inclusive
styles
of
management,
and
have
few
(if
any)
levels
of
management
\
and
inclusive
styles
of
management
be
described?
\
between
the
workers
and
managers.
The
answer
is
(C).
\n\n
Q:
Hygiene
factors
are
\
\
associated
with
which
writer?
\n
(A)
Frederick
Hertzberg
(B)
D.C.
McClelland
(C)
\
(A)
Hierarchical
(B)
Bureaucratic
(C)
Flat
(D)
Functional'
\
Abraham
Maslow
(D)
Douglas
McGregor
\n
A:
Let's
think
step
by
step.
We
refer
to
\
target
:
Let's think step by step. We refer to Wikipedia articles on management
\
Wikipedia
articles
on
management
for
help.
Hygiene
factors
include
compensation,
\
for help. Flat organizational structures are characterized by democratic and
\
company
policies,
supervision,
interpersonal
relations,
and
work
environments.
\
inclusive styles of management, and have few (if any) levels of management between
\
Hertzberg
lists
them
as
factors
that
cannot
motivate
employees
but
can
minimize
\
the workers and managers. The answer is (C).
\
job
dissatisfaction.
The
answer
is
(A).
\n\n
Q:
What
characteristic
is
not
a
key
\
-
question
:
'
Hygiene
factors
are
associated
with
which
writer?
\
feature
of
the
'open
systems'
model
of
management?
\n
(A)
Morale
(B)
Innovation
\
\
(C)
Growth
resource
(D)
Adaptation
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
(A)
Frederick
Hertzberg
(B)
D.C.
McClelland
(C)
Abraham
Maslow
(D)
Douglas
McGregor'
\
articles
on
management
for
help.
The
key
characteristics
of
an
open
system
in
\
target
:
Let's think step by step. We refer to Wikipedia articles on management
\
management
include
innovation,
growth
resource,
and
adaption,
but
do
not
include
\
for help. Hygiene factors include compensation, company policies, supervision,
\
morale.
The
answer
is
(A).
\n\n
Q:
Which
element
of
the
cultural
web
forms
regalia?
\n\
interpersonal relations, and work environments. Hertzberg lists them as factors
(A)
Symbols
(B)
Rituals
and
routines
(C)
Power
structures
(D)
Control
systems
\n\
that cannot motivate employees but can minimize job dissatisfaction. The answer
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
management
for
help.
\
is (A).
\
The
cultural
web
is
a
tool
for
mapping
an
organization's
culture,
where
symbols
\
-
question
:
'
What
characteristic
is
not
a
key
feature
of
the
'
'
open
systems'
'
model
\
form
the
regalia
that
visually
expresses
the
values
that
the
organization
holds
\
of
management?
\
as
important.
The
answer
is
(A).
\n\n
Q:
What
are
the
two
main
dimensions
of
the
\
\
Ohio
Studies
into
leadership?
\n
(A)
Starting
position
and
end
position
(B)
Initial
\
(A)
Morale
(B)
Innovation
(C)
Growth
resource
(D)
Adaptation'
\
environment
and
changed
environment
(C)
Organisational
structure
and
conditioning
\
target
:
Let's think step by step. We refer to Wikipedia articles on management
\
(D)
Initiating
structure
and
considerations
\n
A:
Let's
think
step
by
step.
We
refer
\
for help. The key characteristics of an open system in management include innovation,
\
to
Wikipedia
articles
on
management
for
help.
The
Ohio
State
Leadership
Studies
\
growth resource, and adaption, but do not include morale. The answer is (A).
\
conducted
in
the
1940s
identified
initiating
structure
and
consideration
as
the
\
-
question
:
'
Which
element
of
the
cultural
web
forms
regalia?
\
two
main
dimensions
of
leader
behavior.
The
answer
is
(D).
\n\n
"
"
group"
:
"
mmlu_flan_cot_fewshot_other"
(A)
Symbols
(B)
Rituals
and
routines
(C)
Power
structures
(D)
Control
systems'
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
target
:
Let's think step by step. We refer to Wikipedia articles on management
"
task"
:
"
mmlu_flan_cot_fewshot_management"
for help. The cultural web is a tool for mapping an organization's culture,
where symbols form the regalia that visually expresses the values that the organization
holds as important. The answer is (A).
-
question
:
'
What
are
the
two
main
dimensions
of
the
Ohio
Studies
into
leadership?
(A)
Starting
position
and
end
position
(B)
Initial
environment
and
changed
environment
(C)
Organisational
structure
and
conditioning
(D)
Initiating
structure
and
considerations'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
management
for
help.
The
Ohio
State
Leadership
Studies
conducted
in
the
1940s
identified
initiating
structure
and
consideration
as
the
two
main
dimensions
of
leader
behavior.
The
answer
is
(D).'
group
:
mmlu_flan_cot_fewshot_other
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_management
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_marketing.yaml
View file @
da211969
"
dataset_name"
:
"
marketing"
dataset_name
:
marketing
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
marketing.
\n\
description
:
The following are multiple choice questions (with answers) about marketing.
\n
Q:
Although
the
content
and
quality
can
be
as
controlled
as
direct
mail,
response
\
fewshot_config
:
\
rates
of
this
medium
are
lower
because
of
the
lack
of
a
personal
address
mechanism.
\
sampler
:
first_n
\
This
media
format
is
known
as:
\n
(A)
Care
lines.
(B)
Direct
mail.
(C)
Inserts.
\
samples
:
\
(D)
Door
to
door.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
\
-
question
:
'
Although
the
content
and
quality
can
be
as
controlled
as
direct
mail,
\
on
marketing
for
help.
Door
to
door
marketing
delivers
non-addressed
items
within
\
response
rates
of
this
medium
are
lower
because
of
the
lack
of
a
personal
address
\
all
buildings
within
a
geographic
area.
While
it
can
control
the
content
and
quality
\
mechanism.
This
media
format
is
known
as:
\
as
well
as
direct
mail
marketing,
its
response
rate
is
lower
because
of
the
lack
\
\
of
a
personal
address
mechanism.
The
answer
is
(D).
\n\n
Q:
In
an
organization,
\
(A)
Care
lines.
(B)
Direct
mail.
(C)
Inserts.
(D)
Door
to
door.'
\
the
group
of
people
tasked
with
buying
decisions
is
referred
to
as
the
_______________.
\n\
target
:
Let's think step by step. We refer to Wikipedia articles on marketing
(A)
Outsourcing
unit.
(B)
Procurement
centre.
(C)
Chief
executive
unit.
(D)
Decision-making
\
for help. Door to door marketing delivers non-addressed items within all buildings
\
unit.
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
articles
on
marketing
\
within a geographic area. While it can control the content and quality as well
\
for
help.
In
an
organization,
the
group
of
the
people
tasked
with
buying
decision
\
as direct mail marketing, its response rate is lower because of the lack of
\
is
referred
to
as
the
decision-making
unit.
The
answer
is
(D).
\n\n
Q:
The
single
\
a personal address mechanism. The answer is (D).
\
group
within
society
that
is
most
vulnerable
to
reference
group
influence
is:
\n\
-
question
:
'
In
an
organization,
the
group
of
people
tasked
with
buying
decisions
is
(A)
The
older
consumer
who
feels
somewhat
left
out
of
things.
(B)
The
married
women,
\
referred
to
as
the
_______________.
\
many
of
whom
feel
a
need
for
stability
in
their
lives.
(C)
New
immigrants
who
\
\
really
want
to
assimilate
into
their
new
culture.
(D)
Children,
who
base
most
\
(A)
Outsourcing
unit.
(B)
Procurement
centre.
(C)
Chief
executive
unit.
(D)
\
of
their
buying
decisions
on
outside
influences.
\n
A:
Let's
think
step
by
step.
\
Decision-making
unit.'
\
We
refer
to
Wikipedia
articles
on
marketing
for
help.
Children,
who
mostly
based
\
target
:
Let's think step by step. We refer to Wikipedia articles on marketing
\
their
buying
decisions
on
outside
influences,
are
the
single
group
within
society
\
for help. In an organization, the group of the people tasked with buying decision
\
that
is
more
vulnerable
to
reference
group
influence.
The
answer
is
(D).
\n\n
Q:
\
is referred to as the decision-making unit. The answer is (D).
\
Which
of
the
following
is
an
assumption
in
Maslow's
hierarchy
of
needs?
\n
(A)
Needs
\
-
question
:
'
The
single
group
within
society
that
is
most
vulnerable
to
reference
group
\
are
dependent
on
culture
and
also
on
social
class.
(B)
Lower-level
needs
must
\
influence
is:
\
be
at
least
partially
satisfied
before
higher
needs
can
affect
behaviour.
(C)
\
\
Needs
are
not
prioritized
or
arranged
in
any
particular
order.
(D)
Satisfied
needs
\
(A)
The
older
consumer
who
feels
somewhat
left
out
of
things.
(B)
The
married
\
are
motivators,
and
new
needs
emerge
when
current
needs
remain
unmet.
\n
A:
Let's
\
women,
many
of
whom
feel
a
need
for
stability
in
their
lives.
(C)
New
immigrants
\
think
step
by
step.
We
refer
to
Wikipedia
articles
on
marketing
for
help.
Maslow's
\
who
really
want
to
assimilate
into
their
new
culture.
(D)
Children,
who
base
\
hierarchy
of
needs,
from
the
bottom
upwards,
are
physiological
(food
and
clothing),
\
most
of
their
buying
decisions
on
outside
influences.'
\
safety,
love
and
belonging
needs,
esteem,
and
self-actualization.
Lower-level
\
target
:
Let's think step by step. We refer to Wikipedia articles on marketing
\
needs
must
be
at
least
partially
satisfied
before
higher
ones
can
affect
behavior.
\
for help. Children, who mostly based their buying decisions on outside influences,
\
The
answer
is
(B).
\n\n
Q:
_____________
is
a
natural
outcome
when
combining
demographic
\
are the single group within society that is more vulnerable to reference group
\
and
geographic
variables.
\n
(A)
Geodemographics
(B)
Product
differentiation.
(C)
\
influence. The answer is (D).
\
ANSOFF
matrix.
(D)
Brand
management.
\n
A:
Let's
think
step
by
step.
We
refer
to
\
-
question
:
'
Which
of
the
following
is
an
assumption
in
Maslow'
'
s
hierarchy
of
needs?
\
Wikipedia
articles
on
marketing
for
help.
Geodemographics
is
a
natural
outcome
\
\
when
combining
demographic
and
geographic
variables.
The
answer
is
(A).
\n\n
"
(A)
Needs
are
dependent
on
culture
and
also
on
social
class.
(B)
Lower-level
"
group"
:
"
mmlu_flan_cot_fewshot_other"
needs
must
be
at
least
partially
satisfied
before
higher
needs
can
affect
behaviour.
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
(C)
Needs
are
not
prioritized
or
arranged
in
any
particular
order.
(D)
Satisfied
"
task"
:
"
mmlu_flan_cot_fewshot_marketing"
needs
are
motivators,
and
new
needs
emerge
when
current
needs
remain
unmet.'
target
:
Let's think step by step. We refer to Wikipedia articles on marketing
for help. Maslow's hierarchy of needs, from the bottom upwards, are physiological
(food and clothing), safety, love and belonging needs, esteem, and self-actualization.
Lower-level needs must be at least partially satisfied before higher ones can
affect behavior. The answer is (B).
-
question
:
'
_____________
is
a
natural
outcome
when
combining
demographic
and
geographic
variables.
(A)
Geodemographics
(B)
Product
differentiation.
(C)
ANSOFF
matrix.
(D)
Brand
management.'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
marketing
for
help.
Geodemographics
is
a
natural
outcome
when
combining
demographic
and
geographic
variables.
The
answer
is
(A).'
group
:
mmlu_flan_cot_fewshot_other
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_marketing
lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_medical_genetics.yaml
View file @
da211969
"
dataset_name"
:
"
medical_genetics"
dataset_name
:
medical_genetics
"
description"
:
"
The
following
are
multiple
choice
questions
(with
answers)
about
medical
\
description
:
The following are multiple choice questions (with answers) about medical
\
genetics.
\n\n
Q:
The
stage
of
meiosis
in
which
chromosomes
pair
and
cross
over
\
genetics.
\
is:
\n
(A)
prophase
I
(B)
metaphase
I
(C)
prophase
II
(D)
metaphase
II
\n
A:
Let's
\
fewshot_config
:
\
think
step
by
step.
We
refer
to
Wikipedia
articles
on
medical
genetics
for
help.
\
sampler
:
first_n
\
Prophase
I
is
the
stage
of
meiosis
where
homologous
chromosomes
pair
with
each
\
samples
:
\
other
and
exchange
genetic
material.
The
answer
is
(A).
\n\n
Q:
DNA
ligase
is
\n\
-
question
:
'
The
stage
of
meiosis
in
which
chromosomes
pair
and
cross
over
is:
(A)
an
enzyme
that
joins
fragments
in
normal
DNA
replication
(B)
an
enzyme
of
bacterial
\
\
origin
which
cuts
DNA
at
defined
base
sequences
(C)
an
enzyme
that
facilitates
\
(A)
prophase
I
(B)
metaphase
I
(C)
prophase
II
(D)
metaphase
II'
\
transcription
of
specific
genes
(D)
an
enzyme
which
limits
the
level
to
which
\
target
:
Let's think step by step. We refer to Wikipedia articles on medical genetics
\
a
particular
nutrient
reaches
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
for help. Prophase I is the stage of meiosis where homologous chromosomes pair
\
articles
on
medical
genetics
for
help.
DNA
ligase
is
a
type
of
enzyme
(EC
6.5.1.1)
\
with each other and exchange genetic material. The answer is (A).
\
responsible
for
joining
DNA
strands
together
by
catalyzing
a
phosphodiester
bond.
\
-
question
:
'
DNA
ligase
is
\
The
answer
is
(A).
\n\n
Q:
Which
of
the
following
conditions
does
not
show
multifactorial
\
\
inheritance?
\n
(A)
Pyloric
stenosis
(B)
Schizophrenia
(C)
Spina
bifida
(neural
\
(A)
an
enzyme
that
joins
fragments
in
normal
DNA
replication
(B)
an
enzyme
of
\
tube
defects)
(D)
Marfan
syndrome
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
bacterial
origin
which
cuts
DNA
at
defined
base
sequences
(C)
an
enzyme
that
\
articles
on
medical
genetics
for
help.
Multifactorial
inheritance
is
when
more
\
facilitates
transcription
of
specific
genes
(D)
an
enzyme
which
limits
the
level
\
than
a
single
factor
is
responsible
for
causing
a
given
trait
or
health
problem.
\
to
which
a
particular
nutrient
reaches'
\
Genes
cannot
be
the
only
factor.
Marfan
syndrome,
on
the
other
hand,
requires
\
target
:
Let's think step by step. We refer to Wikipedia articles on medical genetics
\
only
one
abnormal
copy
of
the
of
the
Marfan
gene,
from
one
parent,
to
inherit
\
for help. DNA ligase is a type of enzyme (EC 6.5.1.1) responsible for joining
\
the
trait.
The
answer
is
(D).
\n\n
Q:
A
gene
showing
codominance
\n
(A)
has
both
alleles
\
DNA strands together by catalyzing a phosphodiester bond. The answer is (A).
\
independently
expressed
in
the
heterozygote
(B)
has
one
allele
dominant
to
the
\
-
question
:
'
Which
of
the
following
conditions
does
not
show
multifactorial
inheritance?
\
other
(C)
has
alleles
tightly
linked
on
the
same
chromosome
(D)
has
alleles
expressed
\
\
at
the
same
time
in
development
\n
A:
Let's
think
step
by
step.
We
refer
to
Wikipedia
\
(A)
Pyloric
stenosis
(B)
Schizophrenia
(C)
Spina
bifida
(neural
tube
defects)
\
articles
on
medical
genetics
for
help.
Codominance,
as
it
relates
to
genetics,
\
(D)
Marfan
syndrome'
\
refers
to
a
type
of
genetic
inheritance
where
the
phenotype
of
both
the
parents
\
target
:
Let's think step by step. We refer to Wikipedia articles on medical genetics
\
is
easily
observed
in
the
offspring.
A
heterozygote
is
an
individual
having
two
\
for help. Multifactorial inheritance is when more than a single factor is responsible
\
different
alleles
of
a
gene.
The
answer
is
(A).
\n\n
Q:
Large
triplet
repeat
expansions
\
for causing a given trait or health problem. Genes cannot be the only factor.
\
can
be
detected
by:
\n
(A)
polymerase
chain
reaction.
(B)
single
strand
conformational
\
Marfan syndrome, on the other hand, requires only one abnormal copy of the of
\
polymorphism
analysis.
(C)
Southern
blotting.
(D)
Western
blotting.
\n
A:
Let's
\
the Marfan gene, from one parent, to inherit the trait. The answer is (D).
\
think
step
by
step.
We
refer
to
Wikipedia
articles
on
medical
genetics
for
help.
\
-
question
:
'
A
gene
showing
codominance
\
A
Southern
blot
is
a
method
in
molecular
biology
for
detecting
specific
DNA
sequences
\
\
in
a
sample.
Large
triplet
repeat
expansions
are
usually
detected
with
this
method.
\
(A)
has
both
alleles
independently
expressed
in
the
heterozygote
(B)
has
one
\
The
answer
is
(C).
\n\n
"
allele
dominant
to
the
other
(C)
has
alleles
tightly
linked
on
the
same
chromosome
"
group"
:
"
mmlu_flan_cot_fewshot_other"
(D)
has
alleles
expressed
at
the
same
time
in
development'
"
include"
:
"
_mmlu_flan_cot_fewshot_template_yaml"
target
:
Let's think step by step. We refer to Wikipedia articles on medical genetics
"
task"
:
"
mmlu_flan_cot_fewshot_medical_genetics"
for help. Codominance, as it relates to genetics, refers to a type of genetic
inheritance where the phenotype of both the parents is easily observed in the
offspring. A heterozygote is an individual having two different alleles of a
gene. The answer is (A).
-
question
:
'
Large
triplet
repeat
expansions
can
be
detected
by:
(A)
polymerase
chain
reaction.
(B)
single
strand
conformational
polymorphism
analysis.
(C)
Southern
blotting.
(D)
Western
blotting.'
target
:
'
Let'
'
s
think
step
by
step.
We
refer
to
Wikipedia
articles
on
medical
genetics
for
help.
A
Southern
blot
is
a
method
in
molecular
biology
for
detecting
specific
DNA
sequences
in
a
sample.
Large
triplet
repeat
expansions
are
usually
detected
with
this
method.
The
answer
is
(C).'
group
:
mmlu_flan_cot_fewshot_other
include
:
_mmlu_flan_cot_fewshot_template_yaml
task
:
mmlu_flan_cot_fewshot_medical_genetics
Prev
1
…
22
23
24
25
26
27
28
29
30
…
33
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment
