Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
__init__.py 335 Bytes
Newer Older
xingjinliang's avatar
升级megatron_v0.10  
xingjinliang committed
1 
# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved.
Raul Puri's avatar
initial commit  
Raul Puri committed
2
Mohammad Shoeybi's avatar
Bfloat fused softmax + fused layer norm  
Mohammad Shoeybi committed
3 
from .fused_layer_norm import MixedFusedLayerNorm as LayerNorm
xingjinliang's avatar
升级megatron_v0.10  
xingjinliang committed
4 
from .rms_norm import RMSNorm
mohammad's avatar
Add the option for fp32 residual connection (fp32 residual connection...  
mohammad committed
5
Vijay Korthikanti's avatar
pipeline code simplification  
Vijay Korthikanti committed
6
7 
from .bert_model import BertModel
from .gpt_model import GPTModel
Jared Casper's avatar
Integrate code from t5_main into existing code.  
Jared Casper committed
8 
from .t5_model import T5Model
Boris Fomitchev's avatar
Implementing lazy parallel initialization  
Boris Fomitchev committed
9 
from .language_model import get_language_model
Mohammad Shoeybi's avatar
Bfloat with fp32 grad acc  
Mohammad Shoeybi committed
10 
from .module import Float16Module