Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
date_understanding.yaml 1.63 KB
Newer Older
Clémentine Fourrier's avatar
Making hardcoded few shots compatible with the chat template mechanism (#1895)  
Clémentine Fourrier committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73 
dataset_name: date_understanding
description: 'Infer the date from context.


  '
doc_to_text: 'Q: {{input}}

  A: Let''s think step by step.

  '
fewshot_config:
  sampler: first_n
  samples:
  - input: 'Today is Christmas Eve of 1937. What is the date 10 days ago in MM/DD/YYYY?

      Options:

      (A) 12/14/2026

      (B) 12/14/1950

      (C) 12/14/2007

      (D) 12/14/1937

      (E) 07/14/1938

      (F) 12/14/1988'
    target: 'Let''s think step by step.

      If today is Christmas Eve of 1937, then today''s date is December 24, 1937.
      10 days before today is December 14, 1937, that is 12/14/1937. So the answer
      is (D).'
  - input: 'Tomorrow is 11/12/2019. What is the date one year ago from today in MM/DD/YYYY?

      Options:

      (A) 09/04/2018

      (B) 11/11/2018

      (C) 08/25/2018

      (D) 11/02/2018

      (E) 11/04/2018'
    target: 'Let''s think step by step.

      If tomorrow is 11/12/2019, then today is 11/11/2019. The date one year ago from
      today is 11/11/2018. So the answer is (B).'
  - input: 'Jane and John married on Jan 2, 1958. It is their 5-year anniversary today.
      What is the date tomorrow in MM/DD/YYYY?

      Options:

      (A) 01/11/1961

      (B) 01/03/1963

      (C) 01/18/1961

      (D) 10/14/1960

      (E) 01/03/1982

      (F) 12/03/1960'
    target: 'Let''s think step by step.

      If Jane and John married on Jan 2, 1958, then and if it is their 5-year anniversary
      today, then today''s date is Jan 2, 1963. The date tomorrow is Jan 3, 1963,
      that is 01/03/1963. So the answer is (B).'
include: _cot_fewshot_template_yaml
task: bbh_cot_fewshot_date_understanding