Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "a3904d7e3485f7468ebb165f4383c5a71befe52d"
Switch branch/tag
Find file Normal viewHistoryPermalink
index.rst 617 Bytes
Newer Older
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
1
2
3 
DeepSpeed
=========
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
4
5
6 
Model Setup
-----------
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
7
8
9 
.. toctree::
   :maxdepth: 2
Shaden Smith's avatar
README and RTD improvements. (#198)  
Shaden Smith committed
10 
   initialize
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27 

Training API
------------

.. toctree::
   :maxdepth: 2

   training


Checkpointing API
-----------------
.. toctree::
   :maxdepth: 2

   model-checkpointing
   activation-checkpointing
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
28
29
Samyam Rajbhandari's avatar
ZeRO 3 Offload (#834)  
Samyam Rajbhandari committed
30
31
32
33
34
35
36
37
38
39 
ZeRO API
--------
.. toctree::
   :maxdepth: 2

   zero3
   cpu-adam
Jeff Rasley's avatar
Transformer kernel release (#242)  
Jeff Rasley committed
40
41
42
43
44
45
46 
Transformer Kernel API
----------------------
.. toctree::
   :maxdepth: 2

   kernel
Shaden Smith's avatar
Pipeline parallel training engine. (#392)  
Shaden Smith committed
47
48
49
50
51
52
53 
Pipeline Parallelism
--------------------
.. toctree::
   :maxdepth: 2

   pipeline
Jeff Rasley's avatar
Transformer kernel release (#242)  
Jeff Rasley committed
54
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
55 
Indices and tables
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
56 
------------------
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
57
58
59
60 

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`