Unverified Commit 466115b2 authored by Quentin Lhoest's avatar Quentin Lhoest Committed by GitHub
Browse files

Fix missing reference titles in retrieval evaluation of RAG (#7817)

parent 464b53f5
......@@ -72,7 +72,7 @@ def get_precision_at_k(args, preds_path, gold_data_path):
em = total = 0
for hypo, reference in zip(hypos, references):
hypo_provenance = set(hypo.split("\t")[:k])
ref_provenance = set(reference.split("\t")[1 : (k + 1)])
ref_provenance = set(reference.split("\t"))
total += 1
em += len(hypo_provenance & ref_provenance) / k
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment