Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
deepspeed
Commits
63da7f56
"encoding/kernel/generic/encoding_utils.c" did not exist on "80a12ef6e028f242775b851b40ebd44b35e4374b"
Unverified
Commit
63da7f56
authored
Apr 08, 2020
by
marload
Committed by
GitHub
Apr 07, 2020
Browse files
refactoring: Deduplication (#185)
parent
5fb22a05
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
3 deletions
+3
-3
deepspeed/pt/deepspeed_config.py
deepspeed/pt/deepspeed_config.py
+3
-3
No files found.
deepspeed/pt/deepspeed_config.py
View file @
63da7f56
...
...
@@ -324,21 +324,21 @@ class DeepSpeedConfig(object):
elif
train_batch
is
not
None
and
\
micro_batch
is
not
None
:
grad_acc
=
train_batch
//
micro_batch
grad_acc
=
grad_acc
//
self
.
world_size
grad_acc
//
=
self
.
world_size
self
.
gradient_accumulation_steps
=
grad_acc
#micro_batch_per_gpu needs to be set
elif
train_batch
is
not
None
and
\
grad_acc
is
not
None
:
micro_batch
=
train_batch
//
self
.
world_size
micro_batch
=
micro_batch
//
grad_acc
micro_batch
//
=
grad_acc
self
.
train_micro_batch_size_per_gpu
=
micro_batch
#train_batch_size needs to be set
elif
micro_batch
is
not
None
and
\
grad_acc
is
not
None
:
train_batch_size
=
micro_batch
*
grad_acc
train_batch_size
=
train_batch_size
*
self
.
world_size
train_batch_size
*
=
self
.
world_size
self
.
train_batch_size
=
train_batch_size
#gradient_accumulation_steps and micro_batch_per_gpus is set
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment