Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
6e2c28a1
"examples/movement-pruning/masked_run_squad.py" did not exist on "d1ab1fab1be7199e082129dfbe46eb52bca92799"
Commit
6e2c28a1
authored
Jan 16, 2020
by
Lysandre
Browse files
Run SQuAD warning when the doc stride may be too high
parent
b8f43cb2
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
7 additions
and
0 deletions
+7
-0
examples/run_squad.py
examples/run_squad.py
+7
-0
No files found.
examples/run_squad.py
View file @
6e2c28a1
...
...
@@ -671,6 +671,13 @@ def main():
parser
.
add_argument
(
"--threads"
,
type
=
int
,
default
=
1
,
help
=
"multiple threads for converting example to features"
)
args
=
parser
.
parse_args
()
if
args
.
doc_stride
>=
args
.
max_seq_length
-
args
.
max_query_length
:
logger
.
warning
(
"WARNING - You've set a doc stride which may be superior to the document length in some "
"examples. This could result in errors when building features from the examples. Please reduce the doc "
"stride or increase the maximum length to ensure the features are correctly built."
)
if
(
os
.
path
.
exists
(
args
.
output_dir
)
and
os
.
listdir
(
args
.
output_dir
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment