Skip to content

GitLab

"tests/testdata/triviaqa-v0-res.json" did not exist on "10d4b64a4c5c083064c2af19dc0e51fd33b38ffd"
Switch branch/tag
Find file Normal viewHistoryPermalink
default.yaml 281 Bytes
Newer Older
lintangsutawika's avatar
change all mentions of `greedy_until` to `generate_until`  
lintangsutawika committed
1 
include: _template_yaml
lintangsutawika's avatar
update squadv2  
lintangsutawika committed
2 
task: squadv2
lintangsutawika's avatar
change all mentions of `greedy_until` to `generate_until`  
lintangsutawika committed
3 
output_type: generate_until
lintangsutawika's avatar
update squadv2  
lintangsutawika committed
4
5
6
7 
generation_kwargs:
  until:
    - "\n"
metric_list:
lintangsutawika's avatar
moved benchmarks back to tasks/  
lintangsutawika committed
8
9
10
11
12 
  - metric: !function utils.exact
    aggregation: mean
    higher_is_better: true
  - metric: !function utils.f1
    aggregation: mean
lintangsutawika's avatar
update  
lintangsutawika committed
13 
    higher_is_better: true