pre-process in utils function

lm_eval/tasks/paloma/README.md

@@ -9,6 +9,15 @@ Paloma is a comprehensive benchmark designed to evaluate open language models ac
 
 Homepage: https://allenai.org/olmo
 
+
+### Note
+
+If you are running the entire `paloma` benchmark (or just `paloma_dolma_100_programing_languages`) with a HuggingFace model, make sure to pass `logits_cache=False` to `--model_args`, for example:
+```
+lm_eval --model hf --model_args pretrained=EleutherAI/pythia-160m,logits_cache=False --tasks paloma
+```
+
+
 ### Citation
 ```
 @article{paloma,

lm_eval/tasks/paloma/paloma.yaml

@@ -5,9 +5,9 @@ output_type: loglikelihood_rolling
 validation_split: val
 test_split: test
 doc_to_text: ""
-doc_to_target: "{{text}}"
+doc_to_target: !function paloma_utils.doc_to_target
 should_decontaminate: true
-doc_to_decontamination_query: "{{text}}"
+doc_to_decontamination_query: !function paloma_utils.doc_to_target
 metric_list:
   - metric: word_perplexity
     aggregation: weighted_perplexity

lm_eval/tasks/paloma/paloma_utils.py

+def doc_to_target(doc):
+    return str(doc["text"])