Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
__init__.py 354 Bytes
Newer Older
Jared Casper's avatar
Clean up licensing.  
Jared Casper committed
1 
# Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
Raul Puri's avatar
initial commit  
Raul Puri committed
2
Mohammad Shoeybi's avatar
Bfloat fused softmax + fused layer norm  
Mohammad Shoeybi committed
3 
from .fused_layer_norm import MixedFusedLayerNorm as LayerNorm
mohammad's avatar
Add the option for fp32 residual connection (fp32 residual connection...  
mohammad committed
4
Jared Casper's avatar
Fixing text generation and zeroshot eval and addressing comments.  
Jared Casper committed
5 
from .distributed import DistributedDataParallel
Vijay Korthikanti's avatar
pipeline code simplification  
Vijay Korthikanti committed
6
7 
from .bert_model import BertModel
from .gpt_model import GPTModel
Jared Casper's avatar
Integrate code from t5_main into existing code.  
Jared Casper committed
8 
from .t5_model import T5Model
Boris Fomitchev's avatar
Implementing lazy parallel initialization  
Boris Fomitchev committed
9 
from .language_model import get_language_model
Mohammad Shoeybi's avatar
Bfloat with fp32 grad acc  
Mohammad Shoeybi committed
10 
from .module import Float16Module