Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
bebbdd0f
Unverified
Commit
bebbdd0f
authored
Jun 10, 2021
by
Matt
Committed by
GitHub
Jun 10, 2021
Browse files
Appending label2id and id2label to models to ensure inference works properly (#12102)
parent
4cda08de
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
8 additions
and
0 deletions
+8
-0
examples/pytorch/text-classification/run_glue.py
examples/pytorch/text-classification/run_glue.py
+4
-0
examples/pytorch/text-classification/run_glue_no_trainer.py
examples/pytorch/text-classification/run_glue_no_trainer.py
+4
-0
No files found.
examples/pytorch/text-classification/run_glue.py
View file @
bebbdd0f
...
@@ -370,6 +370,10 @@ def main():
...
@@ -370,6 +370,10 @@ def main():
elif
data_args
.
task_name
is
None
and
not
is_regression
:
elif
data_args
.
task_name
is
None
and
not
is_regression
:
label_to_id
=
{
v
:
i
for
i
,
v
in
enumerate
(
label_list
)}
label_to_id
=
{
v
:
i
for
i
,
v
in
enumerate
(
label_list
)}
if
label_to_id
is
not
None
:
model
.
config
.
label2id
=
label_to_id
model
.
config
.
id2label
=
{
id
:
label
for
label
,
id
in
config
.
label2id
.
items
()}
if
data_args
.
max_seq_length
>
tokenizer
.
model_max_length
:
if
data_args
.
max_seq_length
>
tokenizer
.
model_max_length
:
logger
.
warning
(
logger
.
warning
(
f
"The max_seq_length passed (
{
data_args
.
max_seq_length
}
) is larger than the maximum length for the"
f
"The max_seq_length passed (
{
data_args
.
max_seq_length
}
) is larger than the maximum length for the"
...
...
examples/pytorch/text-classification/run_glue_no_trainer.py
View file @
bebbdd0f
...
@@ -282,6 +282,10 @@ def main():
...
@@ -282,6 +282,10 @@ def main():
elif
args
.
task_name
is
None
:
elif
args
.
task_name
is
None
:
label_to_id
=
{
v
:
i
for
i
,
v
in
enumerate
(
label_list
)}
label_to_id
=
{
v
:
i
for
i
,
v
in
enumerate
(
label_list
)}
if
label_to_id
is
not
None
:
model
.
config
.
label2id
=
label_to_id
model
.
config
.
id2label
=
{
id
:
label
for
label
,
id
in
config
.
label2id
.
items
()}
padding
=
"max_length"
if
args
.
pad_to_max_length
else
False
padding
=
"max_length"
if
args
.
pad_to_max_length
else
False
def
preprocess_function
(
examples
):
def
preprocess_function
(
examples
):
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment