API Guide ========= .. toctree:: :maxdepth: 4 models tensor_parallel context_parallel pipeline_parallel fusions transformer moe dist_checkpointing dist_optimizer distributed datasets num_microbatches_calculator optimizer_param_scheduler
