update mmlu

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_business_ethics.yaml

@@ -70,6 +70,6 @@ fewshot_config:
       \ moral arguments relating to: negative *externalities*, the *power* that corporations\
       \ possess and the *mutual independence* of business and society. The answer\
       \ is (D).\n\n"
-group: mmlu_flan_cot_fewshot_other
+tag: mmlu_flan_cot_fewshot_other
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_business_ethics

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_clinical_knowledge.yaml

@@ -43,6 +43,6 @@ fewshot_config:
     target: 'Let''s think step by step. We refer to Wikipedia articles on clinical
       knowledge for help. The energy for muscular contraction is provided by ATP (adenosine
       triphosphate), which is the powerhouse of the cell. The answer is (A).'
-group: mmlu_flan_cot_fewshot_other
+tag: mmlu_flan_cot_fewshot_other
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_clinical_knowledge

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_biology.yaml

@@ -70,6 +70,6 @@ fewshot_config:
       that have different origins, which is not the case for the human and bird forearms,
       which rules out (D). Humans and birds do belong to the same clade - a group
       of organisms composed of a common ancestor. The answer is (C).'
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_college_biology

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_chemistry.yaml

@@ -44,6 +44,6 @@ fewshot_config:
       \ into 2 lines. This will be further split into 4 lines by the interaction with\
       \ three equivalent 1H nuclei. The total number of lines is therefore $2 \\cdot\
       \ 4 = 8$. The answer is (E).\n\n"
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_college_chemistry

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_computer_science.yaml

@@ -175,6 +175,6 @@ fewshot_config:
       (1000 nanoseconds / cache miss) * (1 cache miss / 50 instructions) * (50 instructions
       / 27000 nanoseconds) = 1000 * (1/50) * (50/27000) = 1000/27000 = 1/27. The answer
       is (B).'
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_college_computer_science

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_mathematics.yaml

@@ -68,6 +68,6 @@ fewshot_config:
       \ Then, for all $t \\in \\mathbb{R}$, we have $(s(t))-2=K e^{-t / 25}$, and\
       \ so $s(t)=2+K e^{-t / 25}$. Then $3=s(0)=2+K e^{0}=2+K$, so $K=1$. Then $s(100)=2+K\
       \ e^{-100 / 25}=2+1 \\cdot e^{-4}=2+e^{-4}$. The answer is (D).\n\n"
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_college_mathematics

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_medicine.yaml

@@ -63,6 +63,6 @@ fewshot_config:
       for help. Glucose (also known as the blood sugar) is the main sugar found in
       the human body. It is transported into the muscle cell via diffusion through
       protein transporters called GLUT4. The answer is (A).'
-group: mmlu_flan_cot_fewshot_other
+tag: mmlu_flan_cot_fewshot_other
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_college_medicine

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_college_physics.yaml

@@ -56,6 +56,6 @@ fewshot_config:
       of the gas container is constant, no work will be done (since work is pressure
       times change in volume). So, at constant volume, all of the heat goes into the
       internal energy. The answer is (B).'
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_college_physics

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_computer_security.yaml

@@ -45,6 +45,6 @@ fewshot_config:
       of the TLS heartbeat extension. The vulnerability was classified as a buffer
       over-read, a situation where more data can be read than should be allowed. The
       answer is (C).'
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_computer_security

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_conceptual_physics.yaml

@@ -44,6 +44,6 @@ fewshot_config:
       \ orthogonal to the wind is the same as it would be in the absence of the wind.\
       \ The total speed, which is these two components added in quadrature, is thus\
       \ greater than the speed in still air. The answer is (B).\n\n"
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_conceptual_physics

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_econometrics.yaml

@@ -82,6 +82,6 @@ fewshot_config:
     target: 'Let''s think step by step. We refer to Wikipedia articles on econometrics
       for help. This is a formal logic problem about stationally process. For a stationary
       autoregressive process, shocks will eventually die away. The answer is (A).'
-group: mmlu_flan_cot_fewshot_social_sciences
+tag: mmlu_flan_cot_fewshot_social_sciences
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_econometrics

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_electrical_engineering.yaml

@@ -42,6 +42,6 @@ fewshot_config:
     target: 'Let''s think step by step. In lap winding, effectively two resistors
       are connected in parallel, so the actual resistance of each pair is 1 Ohm. Since
       we have 50 pairs, we get a total resistance of 50 Ohms. The answer is (C).'
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_electrical_engineering

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_elementary_mathematics.yaml

@@ -72,6 +72,6 @@ fewshot_config:
       (D) (5 x 9) x (6 x 9)'
     target: 'Let''s think step by step. We know that 9 = (5 + 4), so 5 x 9 = 5 x (5
       + 4) = (5 x 5) + (5 x 4). The answer is (B).'
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_elementary_mathematics

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_formal_logic.yaml

@@ -65,6 +65,6 @@ fewshot_config:
       \ p do not drive on Mars.\nOf all these options, Option (C) appears to be the\
       \ best and most meaningful interpretation of the argument \u201CNo people drive\
       \ on Mars.\u201D The answer is (C).\n\n"
-group: mmlu_flan_cot_fewshot_humanities
+tag: mmlu_flan_cot_fewshot_humanities
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_formal_logic

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_global_facts.yaml

@@ -44,6 +44,6 @@ fewshot_config:
       for help. As of 2019, most people tend to be optimistic about their own future
       but pessimistic about the future of their nation or the world. The answer is
       (B).'
-group: mmlu_flan_cot_fewshot_other
+tag: mmlu_flan_cot_fewshot_other
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_global_facts

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_biology.yaml

@@ -64,6 +64,6 @@ fewshot_config:
       core cell cycle regulators inside the cell. The most common regulators are cyclins
       and cyclin-dependent kinases. Fibroblast cells do not play any role in cell
       division. The answer is (D).'
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_high_school_biology

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_chemistry.yaml

@@ -61,6 +61,6 @@ fewshot_config:
       \ strong acid, Nitric acid, will react with the conjugate base. Therefore the\
       \ maximum amount of acid that can be added will be equal to the amount of acetate\
       \ ion, or 2 moles. The answer is (C).\n\n"
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_high_school_chemistry

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_computer_science.yaml

@@ -79,6 +79,6 @@ fewshot_config:
       its value is greater than 100, regardless of the elements in the list. Choice
       D is incorrect because its step 3 does not increment the value of position,
       so it will repeat forever. The answer is (B).'
-group: mmlu_flan_cot_fewshot_stem
+tag: mmlu_flan_cot_fewshot_stem
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_high_school_computer_science

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_european_history.yaml

@@ -194,6 +194,6 @@ fewshot_config:
       wrote extensively against the monoplization of power and advocated for a system
       of checks and balances in government to prevent the rise of despotism. The answer
       is (B).'
-group: mmlu_flan_cot_fewshot_humanities
+tag: mmlu_flan_cot_fewshot_humanities
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_high_school_european_history

lm_eval/tasks/mmlu/flan_cot_fewshot/mmlu_high_school_geography.yaml

@@ -48,6 +48,6 @@ fewshot_config:
     target: 'Let''s think step by step. We refer to Wikipedia articles on geography
       for help. The difference between number of births and deaths gives the population
       increase at any given time. The answer is (A).'
-group: mmlu_flan_cot_fewshot_social_sciences
+tag: mmlu_flan_cot_fewshot_social_sciences
 include: _mmlu_flan_cot_fewshot_template_yaml
 task: mmlu_flan_cot_fewshot_high_school_geography