Skip to content

GitLab

Unverified Commit 56a4e794 authored by Lintang Sutawika's avatar Lintang Sutawika Committed by GitHub
Browse files

formatting (#2104)

parent 9884ad6e
Show whitespace changes
Inline Side-by-side
Showing with 35 additions and 40 deletions
lm_eval/tasks/afrimmlu/direct/utils.py
View file @ 56a4e794
import re
import sys
import unicodedata
from sklearn.metrics import f1_score from sklearn.metrics import f1_score
from lm_eval.filters.extraction import RegexFilter
def doc_to_choice(doc): def doc_to_choice(doc):
...@@ -26,12 +21,14 @@ def doc_to_text(doc): ...@@ -26,12 +21,14 @@ def doc_to_text(doc):
Answer: """ Answer: """
choices = eval(doc["choices"]) choices = eval(doc["choices"])
text = output.format(subject=doc['subject'], text = output.format(
question=doc['question'], subject=doc["subject"],
question=doc["question"],
choice1=choices[0], choice1=choices[0],
choice2=choices[1], choice2=choices[1],
choice3=choices[2], choice3=choices[2],
choice4=choices[3]) choice4=choices[3],
)
return text return text
......
lm_eval/tasks/afrimmlu/translate/afrimmlu_translate_eng.yaml
View file @ 56a4e794
dataset_name: eng dataset_name: eng
include: afrimmlu_common_translate_yaml include: afrimmlu_common_translate_yaml
task: afrimmlu_translate_eng task: afrimmlu_translate_eng
lm_eval/tasks/afrimmlu/translate/afrimmlu_translate_ewe.yaml
View file @ 56a4e794
dataset_name: ewe dataset_name: ewe
include: afrimmlu_common_translate_yaml include: afrimmlu_common_translate_yaml
task: afrimmlu_translate_ewe task: afrimmlu_translate_ewe
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment