Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
37a6dbe2
Commit
37a6dbe2
authored
Jul 12, 2024
by
lintangsutawika
Browse files
adjust naming
parent
75e3f993
Changes
60
Show whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
24 additions
and
22 deletions
+24
-22
lm_eval/tasks/afrimgsm/direct/_afrimgsm_direct.yaml
lm_eval/tasks/afrimgsm/direct/_afrimgsm_direct.yaml
+4
-0
lm_eval/tasks/afrimgsm/direct/_direct_yaml
lm_eval/tasks/afrimgsm/direct/_direct_yaml
+2
-4
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_amh.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_amh.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_eng.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_eng.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_ewe.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_ewe.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_fra.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_fra.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_hau.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_hau.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_ibo.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_ibo.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_kin.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_kin.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_lin.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_lin.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_lug.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_lug.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_orm.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_orm.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_sna.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_sna.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_sot.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_sot.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_swa.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_swa.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_twi.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_twi.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_wol.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_wol.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_xho.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_xho.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_yor.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_yor.yaml
+1
-1
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_zul.yaml
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_zul.yaml
+1
-1
No files found.
lm_eval/tasks/afrimgsm/direct/_afrimgsm_direct.yaml
0 → 100644
View file @
37a6dbe2
group
:
afrimgsm_direct
group_alias
:
AfriMGSM
task
:
-
afrimgsm_direct_tasks
\ No newline at end of file
lm_eval/tasks/afrimgsm/direct/direct_yaml
→
lm_eval/tasks/afrimgsm/direct/
_
direct_yaml
View file @
37a6dbe2
# This file will be included in the generated language-specific task configs.
# This file will be included in the generated language-specific task configs.
# It doesn't have a yaml file extension as it is not meant to be imported directly
# It doesn't have a yaml file extension as it is not meant to be imported directly
# by the harness.
# by the harness.
group:
tag:
- mgsm
- afrimgsm_direct_tasks
- afrimgsm
- afrimgsm_direct
dataset_path: masakhane/afrimgsm
dataset_path: masakhane/afrimgsm
dataset_name: null # Overridden by language-specific config.
dataset_name: null # Overridden by language-specific config.
output_type: generate_until
output_type: generate_until
...
...
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_amh.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_amh
task
:
afrimgsm_direct_amh
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_eng.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_eng
task
:
afrimgsm_direct_eng
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_ewe.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_ewe
task
:
afrimgsm_direct_ewe
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_fra.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_fra
task
:
afrimgsm_direct_fra
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_hau.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_hau
task
:
afrimgsm_direct_hau
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_ibo.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_ibo
task
:
afrimgsm_direct_ibo
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_kin.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_kin
task
:
afrimgsm_direct_kin
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_lin.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_lin
task
:
afrimgsm_direct_lin
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_lug.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_lug
task
:
afrimgsm_direct_lug
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_orm.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_orm
task
:
afrimgsm_direct_orm
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_sna.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_sna
task
:
afrimgsm_direct_sna
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_sot.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_sot
task
:
afrimgsm_direct_sot
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_swa.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_swa
task
:
afrimgsm_direct_swa
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_twi.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_twi
task
:
afrimgsm_direct_twi
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_wol.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_wol
task
:
afrimgsm_direct_wol
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_xho.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_xho
task
:
afrimgsm_direct_xho
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_yor.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_yor
task
:
afrimgsm_direct_yor
lm_eval/tasks/afrimgsm/direct/afrimgsm_direct_zul.yaml
View file @
37a6dbe2
...
@@ -8,5 +8,5 @@ generation_kwargs:
...
@@ -8,5 +8,5 @@ generation_kwargs:
-
'
Question:'
-
'
Question:'
-
</s>
-
</s>
-
<|im_end|>
-
<|im_end|>
include
:
direct_yaml
include
:
_
direct_yaml
task
:
afrimgsm_direct_zul
task
:
afrimgsm_direct_zul
Prev
1
2
3
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment