Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
cd665396
"docs/vscode:/vscode.git/clone" did not exist on "ea1507fb458d41a0ee43045bcee2bcd77bf79497"
Unverified
Commit
cd665396
authored
Sep 08, 2021
by
Sylvain Gugger
Committed by
GitHub
Sep 08, 2021
Browse files
Don't modify labels inplace in `LabelSmoother` (#13464)
parent
c164c651
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
src/transformers/trainer_pt_utils.py
src/transformers/trainer_pt_utils.py
+1
-1
No files found.
src/transformers/trainer_pt_utils.py
View file @
cd665396
...
@@ -458,7 +458,7 @@ class LabelSmoother:
...
@@ -458,7 +458,7 @@ class LabelSmoother:
padding_mask
=
labels
.
eq
(
self
.
ignore_index
)
padding_mask
=
labels
.
eq
(
self
.
ignore_index
)
# In case the ignore_index is -100, the gather will fail, so we replace labels by 0. The padding_mask
# In case the ignore_index is -100, the gather will fail, so we replace labels by 0. The padding_mask
# will ignore them in any case.
# will ignore them in any case.
labels
.
clamp_
min
_
(
0
)
labels
=
torch
.
clamp
(
labels
,
min
=
0
)
nll_loss
=
log_probs
.
gather
(
dim
=-
1
,
index
=
labels
)
nll_loss
=
log_probs
.
gather
(
dim
=-
1
,
index
=
labels
)
# works for fp16 input tensor too, by internally upcasting it to fp32
# works for fp16 input tensor too, by internally upcasting it to fp32
smoothed_loss
=
log_probs
.
sum
(
dim
=-
1
,
keepdim
=
True
,
dtype
=
torch
.
float32
)
smoothed_loss
=
log_probs
.
sum
(
dim
=-
1
,
keepdim
=
True
,
dtype
=
torch
.
float32
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment