Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
renzhc
diffusers_dcu
Commits
fc8afa3a
Unverified
Commit
fc8afa3a
authored
Jan 24, 2023
by
Suraj Patil
Committed by
GitHub
Jan 24, 2023
Browse files
[dreambooth] fix multi on gpu. (#2088)
unwrap model on multi gpu
parent
31336dae
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
9 additions
and
4 deletions
+9
-4
examples/dreambooth/train_dreambooth.py
examples/dreambooth/train_dreambooth.py
+9
-4
No files found.
examples/dreambooth/train_dreambooth.py
View file @
fc8afa3a
...
@@ -716,11 +716,16 @@ def main(args):
...
@@ -716,11 +716,16 @@ def main(args):
" doing mixed precision training. copy of the weights should still be float32."
" doing mixed precision training. copy of the weights should still be float32."
)
)
if
unet
.
dtype
!=
torch
.
float32
:
if
accelerator
.
unwrap_model
(
unet
).
dtype
!=
torch
.
float32
:
raise
ValueError
(
f
"Unet loaded as datatype
{
unet
.
dtype
}
.
{
low_precision_error_string
}
"
)
raise
ValueError
(
f
"Unet loaded as datatype
{
accelerator
.
unwrap_model
(
unet
).
dtype
}
.
{
low_precision_error_string
}
"
)
if
args
.
train_text_encoder
and
text_encoder
.
dtype
!=
torch
.
float32
:
if
args
.
train_text_encoder
and
accelerator
.
unwrap_model
(
text_encoder
).
dtype
!=
torch
.
float32
:
raise
ValueError
(
f
"Text encoder loaded as datatype
{
text_encoder
.
dtype
}
.
{
low_precision_error_string
}
"
)
raise
ValueError
(
f
"Text encoder loaded as datatype
{
accelerator
.
unwrap_model
(
text_encoder
).
dtype
}
."
f
"
{
low_precision_error_string
}
"
)
# We need to recalculate our total training steps as the size of the training dataloader may have changed.
# We need to recalculate our total training steps as the size of the training dataloader may have changed.
num_update_steps_per_epoch
=
math
.
ceil
(
len
(
train_dataloader
)
/
args
.
gradient_accumulation_steps
)
num_update_steps_per_epoch
=
math
.
ceil
(
len
(
train_dataloader
)
/
args
.
gradient_accumulation_steps
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment