Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ColossalAI
Commits
11f1e426
Unverified
Commit
11f1e426
authored
Sep 27, 2023
by
littsk
Committed by
GitHub
Sep 27, 2023
Browse files
[hotfix] Correct several erroneous code comments (#4794)
parent
54b3ad89
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
2 additions
and
2 deletions
+2
-2
colossalai/shardformer/policies/base_policy.py
colossalai/shardformer/policies/base_policy.py
+1
-1
colossalai/zero/low_level/bookkeeping/bucket_store.py
colossalai/zero/low_level/bookkeeping/bucket_store.py
+1
-1
No files found.
colossalai/shardformer/policies/base_policy.py
View file @
11f1e426
...
@@ -50,7 +50,7 @@ class ModulePolicyDescription:
...
@@ -50,7 +50,7 @@ class ModulePolicyDescription:
new_weight = shard_rowwise(weight, process_group)
new_weight = shard_rowwise(weight, process_group)
module.weight = torch.nn.Parameter(new_weight)
module.weight = torch.nn.Parameter(new_weight)
```
```
sub_module_replacement (List[SubModuleReplacementDescription]): each element in the list is a
Param
ReplacementDescription
sub_module_replacement (List[SubModuleReplacementDescription]): each element in the list is a
SubModule
ReplacementDescription
object which specifies the module to be replaced and the target module used to replacement.
object which specifies the module to be replaced and the target module used to replacement.
method_replace (Dict[str, Callable]): key is the method name, value is the method for replacement
method_replace (Dict[str, Callable]): key is the method name, value is the method for replacement
"""
"""
...
...
colossalai/zero/low_level/bookkeeping/bucket_store.py
View file @
11f1e426
...
@@ -92,7 +92,7 @@ class BucketStore(BaseStore):
...
@@ -92,7 +92,7 @@ class BucketStore(BaseStore):
def
get_flatten_grad
(
self
)
->
Tensor
:
def
get_flatten_grad
(
self
)
->
Tensor
:
"""Return the flattened gradients slices in the bucket, the data orginization of the flattened tensor:
"""Return the flattened gradients slices in the bucket, the data orginization of the flattened tensor:
[grad0_rank0, grad1_rank0, ..., grad_
1
_rank
0
, grad1_rank1, ....]
[grad0_rank0, grad1_rank0, ..., grad_
0
_rank
1
, grad1_rank1, ....]
Returns:
Returns:
Tensor: the flattened gradients slices in the bucket
Tensor: the flattened gradients slices in the bucket
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment