“13.0”

1106877d · jerrrrry · 1106877d · 1106877d · 1106877d · 1106877d
Commit 1106877d authored Sep 23, 2025 by jerrrrry
20 changed files
--- a/Megatron-LM/docs/source/api-guide/custom_fsdp.md
+++ b/Megatron-LM/docs/source/api-guide/custom_fsdp.md
--- a/Megatron-LM/docs/source/api-guide/datasets.rst
+++ b/Megatron-LM/docs/source/api-guide/datasets.rst
--- a/Megatron-LM/docs/source/api-guide/dist_checkpointing.rst
+++ b/Megatron-LM/docs/source/api-guide/dist_checkpointing.rst
--- a/Megatron-LM/docs/source/api-guide/dist_checkpointing.strategies.rst
+++ b/Megatron-LM/docs/source/api-guide/dist_checkpointing.strategies.rst
+dist\_checkpointing.strategies package
+======================================
+
+Package defining different checkpoint formats (backends) and saving/loading algorithms (strategies).
+
+Strategies can be used for implementing new checkpoint formats or implementing new (more optimal for a given use case) ways of saving/loading of existing formats.
+Strategies are passed to `dist_checkpointing.load` and `dist_checkpointing.save` functions and control the actual saving/loading procedure.
+
+Submodules
+----------
+
+dist\_checkpointing.strategies.base module
+------------------------------------------
+
+.. automodule:: core.dist_checkpointing.strategies.base
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+dist\_checkpointing.strategies.tensorstore module
+-------------------------------------------------
+
+.. automodule:: core.dist_checkpointing.strategies.tensorstore
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+dist\_checkpointing.strategies.two\_stage module
+------------------------------------------------
+
+.. automodule:: core.dist_checkpointing.strategies.two_stage
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+dist\_checkpointing.strategies.zarr module
+------------------------------------------
+
+.. automodule:: core.dist_checkpointing.strategies.zarr
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Module contents
+---------------
+
+.. automodule:: core.dist_checkpointing.strategies
+   :members:
+   :undoc-members:
+   :show-inheritance:
--- a/Megatron-LM/docs/source/api-guide/dist_optimizer.md
+++ b/Megatron-LM/docs/source/api-guide/dist_optimizer.md
--- a/Megatron-LM/docs/source/api-guide/distributed.rst
+++ b/Megatron-LM/docs/source/api-guide/distributed.rst
--- a/Megatron-LM/docs/source/api-guide/encoder_decoder_parallelism.rst
+++ b/Megatron-LM/docs/source/api-guide/encoder_decoder_parallelism.rst
--- a/Megatron-LM/docs/source/api-guide/fusions.rst
+++ b/Megatron-LM/docs/source/api-guide/fusions.rst
--- a/Megatron-LM/docs/source/api-guide/index.rst
+++ b/Megatron-LM/docs/source/api-guide/index.rst
--- a/Megatron-LM/docs/source/api-guide/models.bert.rst
+++ b/Megatron-LM/docs/source/api-guide/models.bert.rst
--- a/Megatron-LM/docs/source/api-guide/models.gpt.rst
+++ b/Megatron-LM/docs/source/api-guide/models.gpt.rst
--- a/Megatron-LM/docs/source/api-guide/models.rst
+++ b/Megatron-LM/docs/source/api-guide/models.rst
--- a/Megatron-LM/docs/source/api-guide/models.t5.rst
+++ b/Megatron-LM/docs/source/api-guide/models.t5.rst
--- a/Megatron-LM/docs/source/api-guide/moe.rst
+++ b/Megatron-LM/docs/source/api-guide/moe.rst
+Mixture of Experts package
+==========================
+
+.. mdinclude :: ../../../megatron/core/transformer/moe/README.md
--- a/Megatron-LM/docs/source/api-guide/multi_latent_attention.rst
+++ b/Megatron-LM/docs/source/api-guide/multi_latent_attention.rst
--- a/Megatron-LM/docs/source/api-guide/multi_token_prediction.md
+++ b/Megatron-LM/docs/source/api-guide/multi_token_prediction.md
--- a/Megatron-LM/docs/source/api-guide/num_microbatches_calculator.rst
+++ b/Megatron-LM/docs/source/api-guide/num_microbatches_calculator.rst
--- a/Megatron-LM/docs/source/api-guide/optimizer_cpu_offload.rst
+++ b/Megatron-LM/docs/source/api-guide/optimizer_cpu_offload.rst
+Optimizer CPU offload package
+==============================
+
+.. mdinclude :: ../../../megatron/core/optimizer/cpu_offloading/README.md
--- a/Megatron-LM/docs/source/api-guide/optimizer_param_scheduler.rst
+++ b/Megatron-LM/docs/source/api-guide/optimizer_param_scheduler.rst
--- a/Megatron-LM/docs/source/api-guide/pipeline_parallel.rst
+++ b/Megatron-LM/docs/source/api-guide/pipeline_parallel.rst