"vscode:/vscode.git/clone" did not exist on "a3904d7e3485f7468ebb165f4383c5a71befe52d"
index.rst 617 Bytes
Newer Older
Shaden Smith's avatar
Shaden Smith committed
1
2
3
DeepSpeed
=========

Jeff Rasley's avatar
Jeff Rasley committed
4
5
6
Model Setup
-----------

Shaden Smith's avatar
Shaden Smith committed
7
8
9
.. toctree::
   :maxdepth: 2

10
   initialize
Jeff Rasley's avatar
Jeff Rasley committed
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

Training API
------------

.. toctree::
   :maxdepth: 2

   training


Checkpointing API
-----------------
.. toctree::
   :maxdepth: 2

   model-checkpointing
   activation-checkpointing
Shaden Smith's avatar
Shaden Smith committed
28
29


Samyam Rajbhandari's avatar
Samyam Rajbhandari committed
30
31
32
33
34
35
36
37
38
39
ZeRO API
--------
.. toctree::
   :maxdepth: 2

   zero3
   cpu-adam



40
41
42
43
44
45
46
Transformer Kernel API
----------------------
.. toctree::
   :maxdepth: 2

   kernel

47
48
49
50
51
52
53
Pipeline Parallelism
--------------------
.. toctree::
   :maxdepth: 2

   pipeline

54

Shaden Smith's avatar
Shaden Smith committed
55
Indices and tables
Jeff Rasley's avatar
Jeff Rasley committed
56
------------------
Shaden Smith's avatar
Shaden Smith committed
57
58
59
60

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`