# apex.transformer

`apex.transformer` is a module which enables efficient large Transformer models at scale.

`apex.transformer.tensor_parallel` is based on [NVIDIA/Megatron-LM](https://github.com/NVIDIA/Megatron-LM)'s `megatron.mpu` module.