pubmed_qa implemented and tested

lm_eval/tasks/pubmedqa/preprocess_pubmedqa.py

+def doc_to_text(doc):
+    ctxs = "\n".join(doc["context"]["contexts"])
+    return "Abstract: {}\nQuestion: {}\nAnswer:".format(
+        ctxs, doc["question"], doc["final_decision"]
+    )
+
+def doc_to_target(doc):
+    return " {}".format(doc["final_decision"])
+
+def gold_alias(doc):
+    dict_to_label = {
+        'yes': 0,
+        'no': 1,
+        'maybe': 2
+    }
+    return dict_to_label[doc["final_decision"]]
\ No newline at end of file

lm_eval/tasks/pubmedqa/pubmedqa.yaml

+group:
+  - multiple_choice
+task: pubmed_qa
+dataset_path: pubmed_qa
+dataset_name: pqa_labeled
+output_type: multiple_choice
+training_split: null
+validation_split: null
+test_split: train
+template_aliases: "{% set answer_choices = ['yes', 'no', 'maybe'] %}{% set gold = final_decision %}"
+doc_to_text: !function preprocess_pubmedqa.doc_to_text
+doc_to_target: !function preprocess_pubmedqa.doc_to_target
+gold_alias: !function preprocess_pubmedqa.gold_alias
+metric_list:
+  - metric: acc
+    aggregation: mean
+    higher_is_better: true
+  - metric: acc_norm
+    aggregation: mean
+    higher_is_better: true
\ No newline at end of file