DeepSpeed
=========

Model Setup
-----------

.. toctree::
   :maxdepth: 2

   initialize

Training API
------------

.. toctree::
   :maxdepth: 2

   training


Checkpointing API
-----------------
.. toctree::
   :maxdepth: 2

   model-checkpointing
   activation-checkpointing


ZeRO API
--------
.. toctree::
   :maxdepth: 2

   zero3
   cpu-adam



Transformer Kernel API
----------------------
.. toctree::
   :maxdepth: 2

   kernel

Pipeline Parallelism
--------------------
.. toctree::
   :maxdepth: 2

   pipeline


Indices and tables
------------------

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
