Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
6588bc74
Commit
6588bc74
authored
Apr 28, 2022
by
Zdenek Kasner
Browse files
e2e_nlg_cleaned: Update stopping criteria to "\n\n", exclude an accuracy-based prompt
parent
b7f070d8
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
lm_eval/tasks/e2e_nlg_cleaned.py
lm_eval/tasks/e2e_nlg_cleaned.py
+2
-2
No files found.
lm_eval/tasks/e2e_nlg_cleaned.py
View file @
6588bc74
...
...
@@ -59,11 +59,11 @@ class E2E_NLG_Cleaned(PromptSourceTask):
return
self
.
dataset
[
"test"
]
def
stopping_criteria
(
self
):
return
'
\n
'
return
'
\n
\n
'
def
invalid_doc_for_prompt
(
self
,
doc
)
->
bool
:
"""The QA prompts are not applicable to all the examples, we want to filter these out."""
return
self
.
prompt
.
name
.
endswith
(
"_qa"
)
return
self
.
prompt
.
name
.
endswith
(
"_qa"
)
or
self
.
prompt
.
name
==
"family_friendly_yes_no"
def
doc_to_text
(
self
,
doc
)
->
str
:
# if the response is not defined in PS, the text will be a single-element list containing an empty string
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment