Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
index.rst 1.16 KB
Newer Older
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
1
2
3 
DeepSpeed
=========
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
4
5
6 
Model Setup
-----------
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
7
8
9 
.. toctree::
   :maxdepth: 2
Shaden Smith's avatar
README and RTD improvements. (#198)  
Shaden Smith committed
10 
   initialize
aiss's avatar
push Deepspeed 0.6.3 rocm version  
aiss committed
11 
   inference-init
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
12
13
14
15
16
17
18
19
20 

Training API
------------

.. toctree::
   :maxdepth: 2

   training
aiss's avatar
push Deepspeed 0.6.3 rocm version  
aiss committed
21
22
23
24
25
26
27 
Inference API
------------

.. toctree::
   :maxdepth: 2

   inference-engine
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
28
29
30
31
32
33
34
35 

Checkpointing API
-----------------
.. toctree::
   :maxdepth: 2

   model-checkpointing
   activation-checkpointing
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
36
37
Samyam Rajbhandari's avatar
ZeRO 3 Offload (#834)  
Samyam Rajbhandari committed
38
39
40
41
42
43
44 
ZeRO API
--------
.. toctree::
   :maxdepth: 2

   zero3
aiss's avatar
push Deepspeed 0.6.3 rocm version  
aiss committed
45
46
47
48 
Mixture of Experts (MoE)
------------------------
.. toctree::
   :maxdepth: 2
Samyam Rajbhandari's avatar
ZeRO 3 Offload (#834)  
Samyam Rajbhandari committed
49
aiss's avatar
push Deepspeed 0.6.3 rocm version  
aiss committed
50 
   moe
Samyam Rajbhandari's avatar
ZeRO 3 Offload (#834)  
Samyam Rajbhandari committed
51
Jeff Rasley's avatar
Transformer kernel release (#242)  
Jeff Rasley committed
52
53
54
55
56
57
58 
Transformer Kernel API
----------------------
.. toctree::
   :maxdepth: 2

   kernel
Shaden Smith's avatar
Pipeline parallel training engine. (#392)  
Shaden Smith committed
59
60
61
62
63
64
65 
Pipeline Parallelism
--------------------
.. toctree::
   :maxdepth: 2

   pipeline
Cheng Li's avatar
Add optimizers and schedules to RTD and updated the corresponding part in the website (#799)  
Cheng Li committed
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85 
Optimizers
--------------------
.. toctree::
   :maxdepth: 2

   optimizers

Learning Rate Schedulers
--------------------
.. toctree::
   :maxdepth: 2

   schedulers

Flops Profiler
--------------------
.. toctree::
   :maxdepth: 2

   flops-profiler
Jeff Rasley's avatar
Transformer kernel release (#242)  
Jeff Rasley committed
86
aiss's avatar
push Deepspeed 0.6.3 rocm version  
aiss committed
87
88
89
90
91
92
93
94
95
96
97
98
99
100 
Autotuning
--------------------
.. toctree::
   :maxdepth: 2

   autotuning

Memory Usage
------------------
.. toctree::
   :maxdepth: 2

   memory
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
101 
Indices and tables
Jeff Rasley's avatar
ZeRO-2 (#217)  
Jeff Rasley committed
102 
------------------
Shaden Smith's avatar
drafting Jekyll webpage (#143)  
Shaden Smith committed
103
104
105
106 

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`