Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
ffc9e6a0
Commit
ffc9e6a0
authored
Jul 15, 2024
by
lintangsutawika
Browse files
udpate tasks
parent
86f3bb3d
Changes
100
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
20 additions
and
23 deletions
+20
-23
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_twi.yaml
...frixnli/anli_prompt/en-direct/afrixnli_en_direct_twi.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_wol.yaml
...frixnli/anli_prompt/en-direct/afrixnli_en_direct_wol.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_xho.yaml
...frixnli/anli_prompt/en-direct/afrixnli_en_direct_xho.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_yor.yaml
...frixnli/anli_prompt/en-direct/afrixnli_en_direct_yor.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_zul.yaml
...frixnli/anli_prompt/en-direct/afrixnli_en_direct_zul.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/_afrixnli_native_direct_yaml
...li/anli_prompt/native-direct/_afrixnli_native_direct_yaml
+1
-4
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_amh.yaml
...anli_prompt/native-direct/afrixnli_native_direct_amh.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_eng.yaml
...anli_prompt/native-direct/afrixnli_native_direct_eng.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_ewe.yaml
...anli_prompt/native-direct/afrixnli_native_direct_ewe.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_fra.yaml
...anli_prompt/native-direct/afrixnli_native_direct_fra.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_hau.yaml
...anli_prompt/native-direct/afrixnli_native_direct_hau.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_ibo.yaml
...anli_prompt/native-direct/afrixnli_native_direct_ibo.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_kin.yaml
...anli_prompt/native-direct/afrixnli_native_direct_kin.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_lin.yaml
...anli_prompt/native-direct/afrixnli_native_direct_lin.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_lug.yaml
...anli_prompt/native-direct/afrixnli_native_direct_lug.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_orm.yaml
...anli_prompt/native-direct/afrixnli_native_direct_orm.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_sna.yaml
...anli_prompt/native-direct/afrixnli_native_direct_sna.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_sot.yaml
...anli_prompt/native-direct/afrixnli_native_direct_sot.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_swa.yaml
...anli_prompt/native-direct/afrixnli_native_direct_swa.yaml
+1
-1
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_twi.yaml
...anli_prompt/native-direct/afrixnli_native_direct_twi.yaml
+1
-1
No files found.
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_twi.yaml
View file @
ffc9e6a0
# Generated by utils.py
dataset_name
:
twi
include
:
afrixnli_en_direct_yaml
include
:
_
afrixnli_en_direct_yaml
task
:
afrixnli_en_direct_twi
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_wol.yaml
View file @
ffc9e6a0
# Generated by utils.py
dataset_name
:
wol
include
:
afrixnli_en_direct_yaml
include
:
_
afrixnli_en_direct_yaml
task
:
afrixnli_en_direct_wol
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_xho.yaml
View file @
ffc9e6a0
# Generated by utils.py
dataset_name
:
xho
include
:
afrixnli_en_direct_yaml
include
:
_
afrixnli_en_direct_yaml
task
:
afrixnli_en_direct_xho
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_yor.yaml
View file @
ffc9e6a0
# Generated by utils.py
dataset_name
:
yor
include
:
afrixnli_en_direct_yaml
include
:
_
afrixnli_en_direct_yaml
task
:
afrixnli_en_direct_yor
lm_eval/tasks/afrixnli/anli_prompt/en-direct/afrixnli_en_direct_zul.yaml
View file @
ffc9e6a0
# Generated by utils.py
dataset_name
:
zul
include
:
afrixnli_en_direct_yaml
include
:
_
afrixnli_en_direct_yaml
task
:
afrixnli_en_direct_zul
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_yaml
→
lm_eval/tasks/afrixnli/anli_prompt/native-direct/
_
afrixnli_native_direct_yaml
View file @
ffc9e6a0
group:
- xnli
- afrixnli
- afrixnli_native_direct
tag: afrixnli_native_direct
dataset_path: masakhane/afrixnli
dataset_name: null
output_type: multiple_choice
...
...
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_amh.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
amh
doc_to_choice
:
'
{{[premise+",
ትክክል?
አዎ,
"+hypothesis,premise+",
ትክክል?
እንዲሁም,
"+hypothesis,premise+",
ትክክል?
አይ,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_amh
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_eng.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
eng
doc_to_choice
:
'
{{[premise+",
Right?
Yes,
"+hypothesis,premise+",
Right?
Also,
"+hypothesis,premise+",
Right?
No,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_eng
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_ewe.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
ewe
doc_to_choice
:
'
{{[premise+",
Esɔ
gbe?
Ɛ̃,
"+hypothesis,premise+",
Esɔ
gbe?
Hã,
"+hypothesis,premise+",
Esɔ
gbe?
Ao,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_ewe
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_fra.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
fra
doc_to_choice
:
'
{{[premise+",
correct?
Oui,
"+hypothesis,premise+",
correct?
Aussi,
"+hypothesis,premise+",
correct?
Non,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_fra
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_hau.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
hau
doc_to_choice
:
'
{{[premise+",
Daidai?
Ee,
"+hypothesis,premise+",
Daidai?
Haka
kuma,
"+hypothesis,premise+",
Daidai?
A'
'
a,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_hau
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_ibo.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
ibo
doc_to_choice
:
'
{{[premise+",
Ziri
ezi?
Éè,
"+hypothesis,premise+",
Ziri
ezi?
Ọzọkwa,
"+hypothesis,premise+",
Ziri
ezi?
Mba,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_ibo
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_kin.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
kin
doc_to_choice
:
'
{{[premise+",
Nibyo?
Yego,
"+hypothesis,premise+",
Nibyo?
Na
none,
"+hypothesis,premise+",
Nibyo?
Oya,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_kin
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_lin.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
lin
doc_to_choice
:
'
{{[premise+",
Malamu?
Iyo,
"+hypothesis,premise+",
Malamu?
Lisusu,
"+hypothesis,premise+",
Malamu?
Te,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_lin
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_lug.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
lug
doc_to_choice
:
'
{{[premise+",
Kituufu?
Yee,
"+hypothesis,premise+",
Kituufu?
N’ekirala,
"+hypothesis,premise+",
Kituufu?
Nedda,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_lug
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_orm.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
orm
doc_to_choice
:
'
{{[premise+",
Sirrii?
Eeyyee,
"+hypothesis,premise+",
Sirrii?
Akkasumas,
"+hypothesis,premise+",
Sirrii?
Lakki,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_orm
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_sna.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
sna
doc_to_choice
:
'
{{[premise+",
Chokwadi?
Hongu,
"+hypothesis,premise+",
Chokwadi?
Uye,
"+hypothesis,premise+",
Chokwadi?
Kwete,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_sna
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_sot.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
sot
doc_to_choice
:
'
{{[premise+",
Nepile?
E,
"+hypothesis,premise+",
Nepile?
Hape,
"+hypothesis,premise+",
Nepile?
Tjhe,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_sot
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_swa.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
swa
doc_to_choice
:
'
{{[premise+",
Sahihi?
Ndiyo,
"+hypothesis,premise+",
Sahihi?
Pia,
"+hypothesis,premise+",
Sahihi?
Hapana,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_swa
lm_eval/tasks/afrixnli/anli_prompt/native-direct/afrixnli_native_direct_twi.yaml
View file @
ffc9e6a0
...
...
@@ -2,5 +2,5 @@
dataset_name
:
twi
doc_to_choice
:
'
{{[premise+",
Nifa?
Aane,
"+hypothesis,premise+",
Nifa?
Anaasɛ,
"+hypothesis,premise+",
Nifa?
Daabi,
"+hypothesis]}}'
include
:
afrixnli_native_direct_yaml
include
:
_
afrixnli_native_direct_yaml
task
:
afrixnli_native_direct_twi
Prev
1
2
3
4
5
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment