Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
index.rst 857 Bytes
Newer Older
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
1
2
3 
DeepSpeed
=========
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
4
5
6 
Model Setup
-----------
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
7
8
9 
.. toctree::
   :maxdepth: 2
Shaden Smith's avatar
README and RTD improvements. (#198)  
Shaden Smith committed
10 
   initialize
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27 

Training API
------------

.. toctree::
   :maxdepth: 2

   training


Checkpointing API
-----------------
.. toctree::
   :maxdepth: 2

   model-checkpointing
   activation-checkpointing
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
28
29
Samyam Rajbhandari's avatar
ZeRO 3 Offload (#834)  
Samyam Rajbhandari committed
30
31
32
33
34
35
36
37
38 
ZeRO API
--------
.. toctree::
   :maxdepth: 2

   zero3
Jeff Rasley's avatar
Transformer kernel release (#242)  
Jeff Rasley committed
39
40
41
42
43
44
45 
Transformer Kernel API
----------------------
.. toctree::
   :maxdepth: 2

   kernel
Shaden Smith's avatar
Pipeline parallel training engine. (#392)  
Shaden Smith committed
46
47
48
49
50
51
52 
Pipeline Parallelism
--------------------
.. toctree::
   :maxdepth: 2

   pipeline
Cheng Li's avatar
Add optimizers and schedules to RTD and updated the corresponding part in the website (#799)  
Cheng Li committed
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72 
Optimizers
--------------------
.. toctree::
   :maxdepth: 2

   optimizers

Learning Rate Schedulers
--------------------
.. toctree::
   :maxdepth: 2

   schedulers

Flops Profiler
--------------------
.. toctree::
   :maxdepth: 2

   flops-profiler
Jeff Rasley's avatar
Transformer kernel release (#242)  
Jeff Rasley committed
73
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
74 
Indices and tables
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
75 
------------------
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
76
77
78
79 

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`