Megatron Core is a library for efficient and scalable training of transformer based models.