Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
mimic_repsum_perplexity.yaml 347 Bytes
Newer Older
PabloAgustin's avatar
New healthcare benchmark: careqa (#2714)  
PabloAgustin committed
1
2
3
4
5
6
7
8
9
10
11
12 
include: mimic_repsum.yaml
task: mimic_repsum_perplexity
output_type: loglikelihood_rolling
doc_to_text: ""
process_results: !function utils_perplexity.process_results
metric_list:
  - metric: word_perplexity
    higher_is_better: false
  - metric: byte_perplexity
    higher_is_better: false
  - metric: bits_per_byte
    higher_is_better: false