Skip to content

GitLab

Unverified Commit 90017d3a authored by Samyam Rajbhandari's avatar Samyam Rajbhandari Committed by GitHub
Browse files

AllReduce bucket fix. (#186)

parent 4cbfcc75
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
deepspeed/pt/deepspeed_light.py
View file @ 90017d3a
...@@ -840,6 +840,7 @@ class DeepSpeedLight(Module): ...@@ -840,6 +840,7 @@ class DeepSpeedLight(Module):
if numel > numel_per_bucket: if numel > numel_per_bucket:
self.allreduce_and_copy(small_bucket) self.allreduce_and_copy(small_bucket)
small_bucket = [] small_bucket = []
numel = 0
if len(small_bucket) > 0: if len(small_bucket) > 0:
self.allreduce_and_copy(small_bucket) self.allreduce_and_copy(small_bucket)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment