Skip to content

GitLab

Commit 557ae9c4 authored by sunxx1's avatar sunxx1
Browse files

Merge branch 'hepj-test' into 'main' 

修改VIT模型loss就地操作

See merge request dcutoolkit/deeplearing/dlexamples_new!59
parents 0ae03c71 0a159036
Show whitespace changes
Inline Side-by-side
Showing with 4 additions and 2 deletions
PyTorch/NLP/Vision_Transformer/engine_pretrain.py
View file @ 557ae9c4
...@@ -53,7 +53,9 @@ def train_one_epoch(model: torch.nn.Module, ...@@ -53,7 +53,9 @@ def train_one_epoch(model: torch.nn.Module,
print("Loss is {}, stopping training".format(loss_value)) print("Loss is {}, stopping training".format(loss_value))
sys.exit(1) sys.exit(1)
loss /= accum_iter loss_new= loss/accum_iter
loss=loss_new
loss_scaler(loss, optimizer, parameters=model.parameters(), loss_scaler(loss, optimizer, parameters=model.parameters(),
update_grad=(data_iter_step + 1) % accum_iter == 0) update_grad=(data_iter_step + 1) % accum_iter == 0)
if (data_iter_step + 1) % accum_iter == 0: if (data_iter_step + 1) % accum_iter == 0:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment