Skip to content

GitLab

"applications/Chat/coati/ray/callbacks/base.py" did not exist on "41fb7236aa32c307e83b0b9cc50ce2a6da279343"
Switch branch/tag
Find file Normal viewHistoryPermalink
colossalai_ddp.py 128 Bytes
Newer Older
mandoxzhang's avatar
add RoBERTa (#1980)  
mandoxzhang committed
1
2
3
4 
from colossalai.zero.shard_utils import TensorShardStrategy
from colossalai.nn.optimizer import FusedAdam

clip_grad_norm = 1.0