Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
cd665396
Unverified
Commit
cd665396
authored
Sep 08, 2021
by
Sylvain Gugger
Committed by
GitHub
Sep 08, 2021
Browse files
Don't modify labels inplace in `LabelSmoother` (#13464)
parent
c164c651
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
src/transformers/trainer_pt_utils.py
src/transformers/trainer_pt_utils.py
+1
-1
No files found.
src/transformers/trainer_pt_utils.py
View file @
cd665396
...
@@ -458,7 +458,7 @@ class LabelSmoother:
...
@@ -458,7 +458,7 @@ class LabelSmoother:
padding_mask
=
labels
.
eq
(
self
.
ignore_index
)
padding_mask
=
labels
.
eq
(
self
.
ignore_index
)
# In case the ignore_index is -100, the gather will fail, so we replace labels by 0. The padding_mask
# In case the ignore_index is -100, the gather will fail, so we replace labels by 0. The padding_mask
# will ignore them in any case.
# will ignore them in any case.
labels
.
clamp_
min
_
(
0
)
labels
=
torch
.
clamp
(
labels
,
min
=
0
)
nll_loss
=
log_probs
.
gather
(
dim
=-
1
,
index
=
labels
)
nll_loss
=
log_probs
.
gather
(
dim
=-
1
,
index
=
labels
)
# works for fp16 input tensor too, by internally upcasting it to fp32
# works for fp16 input tensor too, by internally upcasting it to fp32
smoothed_loss
=
log_probs
.
sum
(
dim
=-
1
,
keepdim
=
True
,
dtype
=
torch
.
float32
)
smoothed_loss
=
log_probs
.
sum
(
dim
=-
1
,
keepdim
=
True
,
dtype
=
torch
.
float32
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment