Commit 08c54c63 authored by Baber's avatar Baber
Browse files

nq_open: move multi_target to `exact_match`

parent 8f924e1c
...@@ -24,3 +24,6 @@ journal = {Transactions of the Association of Computational Linguistics}} ...@@ -24,3 +24,6 @@ journal = {Transactions of the Association of Computational Linguistics}}
### Tasks ### Tasks
* `nq_open` * `nq_open`
### Changelog
* 2025-07-21: Added `multi_target` to `exact_match`. Scores should not change.
task: nq_open task: nq_open
dataset_path: nq_open dataset_path: google-research-datasets/nq_open
output_type: generate_until output_type: generate_until
training_split: train training_split: train
validation_split: validation validation_split: validation
description: "Answer these questions:\n\n" description: "Answer these questions:\n\n"
doc_to_text: "Q: {{question}}?\nA:" doc_to_text: "Q: {{question}}?\nA:"
doc_to_target: "{{answer}}" # TODO: should be multi-target doc_to_target: "{{answer}}"
fewshot_delimiter: "\n" fewshot_delimiter: "\n"
generation_kwargs: generation_kwargs:
until: until:
...@@ -28,5 +28,6 @@ metric_list: ...@@ -28,5 +28,6 @@ metric_list:
ignore_punctuation: true ignore_punctuation: true
regexes_to_ignore: regexes_to_ignore:
- "\\b(?:The |the |An |A |The |a |an )" - "\\b(?:The |the |An |A |The |a |an )"
multi_target: true
metadata: metadata:
version: 4.0 version: 4.0
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment