Additional process for doc_to_choice (#1093)

* Additional process for doc_to_choice

* doc_to_choice can also parse a string

docs/new_task_guide.md

@@ -123,6 +123,13 @@ doc_to_target: 3
 doc_to_choice: ['No', 'Yes']
 ```
 
+if a dataset feature is already a list, you can set the name of the feature as `doc_to_choice` (See [Hellaswag](https://github.com/EleutherAI/lm-evaluation-harness/blob/e0eda4d3ffa10e5f65e0976161cd134bec61983a/lm_eval/tasks/hellaswag/hellaswag.yaml#L13))
+```
+doc_to_choice: choices
+```
+
+
+
 ### Writing a prompt with Jinja 2
 
 We support the [Jinja 2](https://jinja.palletsprojects.com/en/3.1.x/) templating language for writing prompts. In practice, this means you can take your dataset's columns and do many basic string manipulations to place each document into prompted format.

lm_eval/api/task.py

@@ -936,7 +936,10 @@ class ConfigurableTask(Task):
             doc_to_choice = self.config.doc_to_choice
 
         if type(doc_to_choice) == str:
-            return ast.literal_eval(utils.apply_template(doc_to_choice, doc))
+            if doc_to_choice in self.features:
+                return doc[doc_to_choice]
+            else:
+                return ast.literal_eval(utils.apply_template(doc_to_choice, doc))
         elif type(doc_to_choice) == list:
             return doc_to_choice
         elif type(doc_to_choice) == dict:

lm_eval/tasks/hellaswag/hellaswag.yaml

@@ -10,7 +10,7 @@ test_split: null
 process_docs: !function utils.process_docs
 doc_to_text: "{{query}}"
 doc_to_target: "{{label}}"
-doc_to_choice: "{{choices}}"
+doc_to_choice: "choices"
 metric_list:
   - metric: acc
     aggregation: mean