Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
7d09b24c
"examples/asr/emformer_rnnt/common.py" did not exist on "87d7694d08d9873b1c9e8cf7e8fe92ea398a1488"
Commit
7d09b24c
authored
Jul 03, 2024
by
haileyschoelkopf
Browse files
fix alllll the merge conflicts
parents
96dfe976
6348b947
Changes
395
Hide whitespace changes
Inline
Side-by-side
Showing
20 changed files
with
19 additions
and
22 deletions
+19
-22
lm_eval/tasks/csatqa/_default_csatqa_yaml
lm_eval/tasks/csatqa/_default_csatqa_yaml
+0
-1
lm_eval/tasks/fda/task.py
lm_eval/tasks/fda/task.py
+1
-1
lm_eval/tasks/fld/fld_default.yaml
lm_eval/tasks/fld/fld_default.yaml
+0
-2
lm_eval/tasks/french_bench/README.md
lm_eval/tasks/french_bench/README.md
+2
-2
lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml
lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_boolqa.yaml
lm_eval/tasks/french_bench/french_bench_boolqa.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_fquadv2.yaml
lm_eval/tasks/french_bench/french_bench_fquadv2.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml
lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml
lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml
lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_grammar.yaml
lm_eval/tasks/french_bench/french_bench_grammar.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_hellaswag.yaml
lm_eval/tasks/french_bench/french_bench_hellaswag.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_multifquad.yaml
lm_eval/tasks/french_bench/french_bench_multifquad.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml
lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml
...l/tasks/french_bench/french_bench_orangesum_abstract.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml
lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_reading_comp.yaml
lm_eval/tasks/french_bench/french_bench_reading_comp.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml
lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_trivia.yaml
lm_eval/tasks/french_bench/french_bench_trivia.yaml
+1
-1
lm_eval/tasks/french_bench/french_bench_vocab.yaml
lm_eval/tasks/french_bench/french_bench_vocab.yaml
+1
-1
No files found.
lm_eval/tasks/csatqa/_default_csatqa_yaml
View file @
7d09b24c
group: csatqa
dataset_path: EleutherAI/csatqa
test_split: test
output_type: multiple_choice
...
...
lm_eval/tasks/fda/task.py
View file @
7d09b24c
...
...
@@ -12,7 +12,7 @@ class FDA(ConfigurableTask):
DATASET_PATH
=
"hazyresearch/based-fda"
DATASET_NAME
=
"default"
def
__init__
(
self
):
def
__init__
(
self
,
**
kwargs
):
super
().
__init__
(
config
=
{
"metadata"
:
{
"version"
:
self
.
VERSION
}})
def
has_training_docs
(
self
):
...
...
lm_eval/tasks/fld/fld_default.yaml
View file @
7d09b24c
group
:
-
fld
task
:
fld_default
dataset_path
:
hitachi-nlp/FLD.v2
dataset_name
:
default
...
...
lm_eval/tasks/french_bench/README.md
View file @
7d09b24c
...
...
@@ -20,9 +20,9 @@ This benchmark is constructed both from openly available datasets, as well as ne
}
```
### Groups and Tasks
### Groups
, Tags,
and Tasks
####
Group
s
####
Tag
s
-
`french_bench`
: All tasks (non-perplexity based)
-
`french_bench_gen`
: All official generative tasks
...
...
lm_eval/tasks/french_bench/french_bench_arc_challenge.yaml
View file @
7d09b24c
group
:
tag
:
-
french_bench
-
french_bench_mc
task
:
french_bench_arc_challenge
...
...
lm_eval/tasks/french_bench/french_bench_boolqa.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_extra
description
:
"
D'après
l'information
dans
le
contexte
donné,
quelle
est
la
réponse
à
la
question
?"
...
...
lm_eval/tasks/french_bench/french_bench_fquadv2.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_extra
description
:
"
D'après
l'information
dans
le
contexte
donné,
donne
la
réponse
à
la
question
en
citant
quelques
mots
du
contexte.
Si
il
est
impossible
de
répondre
avec
les
informations
du
contexte,
répond
'Impossible'."
...
...
lm_eval/tasks/french_bench/french_bench_fquadv2_bool.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_extra
description
:
"
D'après
l'information
présente
dans
le
contexte,
est
il
possible
de
répondre
à
la
question
?"
...
...
lm_eval/tasks/french_bench/french_bench_fquadv2_genq.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_gen
description
:
"
D'après
l'information
dans
le
contexte
donné,
quelle
question
a
été
posée
pour
obtenir
la
réponse
donnée
?"
...
...
lm_eval/tasks/french_bench/french_bench_fquadv2_hasAns.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_gen
description
:
"
D'après
l'information
dans
le
contexte
donné,
donne
la
réponse
à
la
question
en
citant
quelques
mots
du
contexte.
Si
il
est
impossible
de
répondre
avec
les
informations
du
contexte,
répond
'Impossible'."
...
...
lm_eval/tasks/french_bench/french_bench_grammar.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_mc
description
:
"
Répond
au
mieux
en
complétant
la
question
avec
une
des
réponses
proposées."
...
...
lm_eval/tasks/french_bench/french_bench_hellaswag.yaml
View file @
7d09b24c
group
:
tag
:
-
french_bench
-
french_bench_mc
task
:
french_bench_hellaswag
...
...
lm_eval/tasks/french_bench/french_bench_multifquad.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_gen
description
:
"
D'après
l'information
dans
le
contexte
donné,
donne
la
réponse
à
la
question
en
citant
quelques
extraits
du
contexte."
...
...
lm_eval/tasks/french_bench/french_bench_opus_perplexity.yaml
View file @
7d09b24c
group
:
tag
:
-
french_bench_perplexity
task
:
french_bench_opus_perplexity
dataset_path
:
manu/opus100-en-fr
...
...
lm_eval/tasks/french_bench/french_bench_orangesum_abstract.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_gen
description
:
"
Résume
l'article
en
une
phrase."
...
...
lm_eval/tasks/french_bench/french_bench_orangesum_title.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_extra
description
:
"
Trouve
le
titre
de
l'article."
...
...
lm_eval/tasks/french_bench/french_bench_reading_comp.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_extra
# description: "Répond au mieux en complétant la question avec une des réponses proposées."
...
...
lm_eval/tasks/french_bench/french_bench_topic_based_nli.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_extra
description
:
"
A
propos
du
thème
spécifié,
l'avis
client
est
il
positif,
négatif,
ou
neutre
?"
...
...
lm_eval/tasks/french_bench/french_bench_trivia.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_gen
task
:
french_bench_trivia
...
...
lm_eval/tasks/french_bench/french_bench_vocab.yaml
View file @
7d09b24c
include
:
"
_default_template_yaml"
group
:
tag
:
-
french_bench
-
french_bench_mc
# description: "Répond au mieux en complétant la question avec une des réponses proposées."
...
...
Prev
1
…
4
5
6
7
8
9
10
11
12
…
20
Next
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment