Unverified Commit 7cab55c7 authored by Cheng Li's avatar Cheng Li Committed by GitHub
Browse files

Checks for None tensors and skip them when splitting the buckets in zero stage 2. (#728)

* check none tensors when splitting buckets
parent ec8b1cb0
...@@ -37,7 +37,7 @@ def split_half_float_double(tensors): ...@@ -37,7 +37,7 @@ def split_half_float_double(tensors):
] ]
buckets = [] buckets = []
for i, dtype in enumerate(dtypes): for i, dtype in enumerate(dtypes):
bucket = [t for t in tensors if t.type() == dtype] bucket = [t for t in tensors if t is not None and t.type() == dtype]
if bucket: if bucket:
buckets.append(bucket) buckets.append(bucket)
return buckets return buckets
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment