Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
835cc40e
"vscode:/vscode.git/clone" did not exist on "eec9de3eb3bc7298302fa042d5d5b82a5b169d52"
Commit
835cc40e
authored
Dec 06, 2023
by
lintangsutawika
Browse files
merged latest and added altworld files
parents
8da401e0
c9bbec6e
Changes
430
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
143 additions
and
0 deletions
+143
-0
lm_eval/tasks/arc/alternative_worlds/arc_easy_alt.yaml
lm_eval/tasks/arc/alternative_worlds/arc_easy_alt.yaml
+10
-0
lm_eval/tasks/arc/alternative_worlds/style_01/a.yaml
lm_eval/tasks/arc/alternative_worlds/style_01/a.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_01/b.yaml
lm_eval/tasks/arc/alternative_worlds/style_01/b.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_01/c.yaml
lm_eval/tasks/arc/alternative_worlds/style_01/c.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_02/a.yaml
lm_eval/tasks/arc/alternative_worlds/style_02/a.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_02/b.yaml
lm_eval/tasks/arc/alternative_worlds/style_02/b.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_02/c.yaml
lm_eval/tasks/arc/alternative_worlds/style_02/c.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_03/a.yaml
lm_eval/tasks/arc/alternative_worlds/style_03/a.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_03/b.yaml
lm_eval/tasks/arc/alternative_worlds/style_03/b.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_03/c.yaml
lm_eval/tasks/arc/alternative_worlds/style_03/c.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_04/a.yaml
lm_eval/tasks/arc/alternative_worlds/style_04/a.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_04/b.yaml
lm_eval/tasks/arc/alternative_worlds/style_04/b.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_04/c.yaml
lm_eval/tasks/arc/alternative_worlds/style_04/c.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_05/a.yaml
lm_eval/tasks/arc/alternative_worlds/style_05/a.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_05/b.yaml
lm_eval/tasks/arc/alternative_worlds/style_05/b.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_05/c.yaml
lm_eval/tasks/arc/alternative_worlds/style_05/c.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_06/a.yaml
lm_eval/tasks/arc/alternative_worlds/style_06/a.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_06/b.yaml
lm_eval/tasks/arc/alternative_worlds/style_06/b.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_06/c.yaml
lm_eval/tasks/arc/alternative_worlds/style_06/c.yaml
+7
-0
lm_eval/tasks/arc/alternative_worlds/style_07/a.yaml
lm_eval/tasks/arc/alternative_worlds/style_07/a.yaml
+7
-0
No files found.
lm_eval/tasks/arc/alternative_worlds/arc_easy_alt.yaml
0 → 100644
View file @
835cc40e
group
:
arc_easy_alt
task
:
-
arc_easy_01
-
arc_easy_02
-
arc_easy_03
-
arc_easy_04
-
arc_easy_05
-
arc_easy_06
-
arc_easy_07
-
arc_easy_08
lm_eval/tasks/arc/alternative_worlds/style_01/a.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_01
group_alias
:
style_01
task
:
arc_easy_01a
task_alias
:
a
doc_to_text
:
!function
../styles.template_01
doc_to_choice
:
!function
../styles.choice_01a
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_01/b.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_01
group_alias
:
style_01
task
:
arc_easy_01b
task_alias
:
b
doc_to_text
:
!function
../styles.template_01
doc_to_choice
:
!function
../styles.choice_01b
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_01/c.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_01
group_alias
:
style_01
task
:
arc_easy_01c
task_alias
:
c
doc_to_text
:
!function
../styles.template_01
doc_to_choice
:
!function
../styles.choice_01c
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_02/a.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_02
group_alias
:
style_02
task
:
arc_easy_02a
task_alias
:
a
doc_to_text
:
!function
../styles.template_02
doc_to_choice
:
!function
../styles.choice_02a
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_02/b.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_02
group_alias
:
style_02
task
:
arc_easy_02b
task_alias
:
b
doc_to_text
:
!function
../styles.template_02
doc_to_choice
:
!function
../styles.choice_02b
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_02/c.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_02
group_alias
:
style_02
task
:
arc_easy_02c
task_alias
:
c
doc_to_text
:
!function
../styles.template_02
doc_to_choice
:
!function
../styles.choice_02c
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_03/a.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_03
group_alias
:
style_03
task
:
arc_easy_03a
task_alias
:
a
doc_to_text
:
!function
../styles.template_03
doc_to_choice
:
!function
../styles.choice_03a
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_03/b.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_03
group_alias
:
style_03
task
:
arc_easy_03b
task_alias
:
b
doc_to_text
:
!function
../styles.template_03
doc_to_choice
:
!function
../styles.choice_03b
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_03/c.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_03
group_alias
:
style_03
task
:
arc_easy_03c
task_alias
:
c
doc_to_text
:
!function
../styles.template_03
doc_to_choice
:
!function
../styles.choice_03c
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_04/a.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_04
group_alias
:
style_04
task
:
arc_easy_04a
task_alias
:
a
doc_to_text
:
!function
../styles.template_04
doc_to_choice
:
!function
../styles.choice_04a
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_04/b.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_04
group_alias
:
style_04
task
:
arc_easy_04b
task_alias
:
b
doc_to_text
:
!function
../styles.template_04
doc_to_choice
:
!function
../styles.choice_04b
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_04/c.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_04
group_alias
:
style_04
task
:
arc_easy_04c
task_alias
:
c
doc_to_text
:
!function
../styles.template_04
doc_to_choice
:
!function
../styles.choice_04c
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_05/a.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_05
group_alias
:
style_05
task
:
arc_easy_05a
task_alias
:
a
doc_to_text
:
!function
../styles.template_05
doc_to_choice
:
!function
../styles.choice_05a
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_05/b.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_05
group_alias
:
style_05
task
:
arc_easy_05b
task_alias
:
b
doc_to_text
:
!function
../styles.template_05
doc_to_choice
:
!function
../styles.choice_05b
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_05/c.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_05
group_alias
:
style_05
task
:
arc_easy_05c
task_alias
:
c
doc_to_text
:
!function
../styles.template_05
doc_to_choice
:
!function
../styles.choice_05c
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_06/a.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_06
group_alias
:
style_06
task
:
arc_easy_06a
task_alias
:
a
doc_to_text
:
!function
../styles.template_06
doc_to_choice
:
!function
../styles.choice_06a
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_06/b.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_06
group_alias
:
style_06
task
:
arc_easy_06b
task_alias
:
b
doc_to_text
:
!function
../styles.template_06
doc_to_choice
:
!function
../styles.choice_06b
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_06/c.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_06
group_alias
:
style_06
task
:
arc_easy_06c
task_alias
:
c
doc_to_text
:
!function
../styles.template_06
doc_to_choice
:
!function
../styles.choice_06c
\ No newline at end of file
lm_eval/tasks/arc/alternative_worlds/style_07/a.yaml
0 → 100644
View file @
835cc40e
include
:
../_arc_easy_alt_yaml
group
:
arc_easy_07
group_alias
:
style_07
task
:
arc_easy_07a
task_alias
:
a
doc_to_text
:
!function
../styles.template_07
doc_to_choice
:
!function
../styles.choice_07a
\ No newline at end of file
Prev
1
2
3
4
5
6
…
22
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment