Unverified Commit 1496247a authored by Shaden Smith's avatar Shaden Smith Committed by GitHub
Browse files

JSON configuration cleanup. (#151)

* Better config filename

* Clean up configuration ToC
parent 29855c27
...@@ -413,7 +413,7 @@ as the hostname. ...@@ -413,7 +413,7 @@ as the hostname.
| Article | Description | | Article | Description |
| ---------------------------------------------------------------------------------------------- | -------------------------------------------- | | ---------------------------------------------------------------------------------------------- | -------------------------------------------- |
| [DeepSpeed Features](https://www.deepspeed.ai/features/) | DeepSpeed features | | [DeepSpeed Features](https://www.deepspeed.ai/features/) | DeepSpeed features |
| [DeepSpeed JSON Configuration](https://www.deepspeed.ai/docs/config_json/) | Configuring DeepSpeed | | [DeepSpeed JSON Configuration](https://www.deepspeed.ai/docs/config-json/) | Configuring DeepSpeed |
| [API Documentation](https://deepspeed.readthedocs.io/en/latest/) | Generated DeepSpeed API documentation | | [API Documentation](https://deepspeed.readthedocs.io/en/latest/) | Generated DeepSpeed API documentation |
| [CIFAR-10 Tutorial](https://www.deepspeed.ai/tutorials/cifar-10) | Getting started with CIFAR-10 and DeepSpeed | | [CIFAR-10 Tutorial](https://www.deepspeed.ai/tutorials/cifar-10) | Getting started with CIFAR-10 and DeepSpeed |
| [Megatron-LM Tutorial](https://www.deepspeed.ai/tutorials/megatron/) | Train GPT2 with DeepSpeed and Megatron-LM | | [Megatron-LM Tutorial](https://www.deepspeed.ai/tutorials/megatron/) | Train GPT2 with DeepSpeed and Megatron-LM |
......
...@@ -48,7 +48,7 @@ defaults: ...@@ -48,7 +48,7 @@ defaults:
- scope: - scope:
path: "_pages" path: "_pages"
values: values:
permalink: /docs/:basename permalink: /docs/:basename/
- scope: - scope:
path: "" path: ""
type: posts type: posts
......
...@@ -25,7 +25,20 @@ lnav: ...@@ -25,7 +25,20 @@ lnav:
- title: "Launching" - title: "Launching"
url: /getting-started/#launching-deepspeed-training url: /getting-started/#launching-deepspeed-training
- title: "Configuration" - title: "Configuration"
url: /docs/config_json/ url: /docs/config-json/
children:
- title: "Batch size"
url: /docs/config-json/#batch-size-related-parameters
- title: "Optimizer"
url: /docs/config-json/#optimizer-parameters
- title: "Scheduler"
url: /docs/config-json/#scheduler-parameters
- title: "Communication"
url: /docs/config-json/#communication-options
- title: "FP16"
url: /docs/config-json/#fp16-training-options
- title: "Logging"
url: /docs/config-json/#logging
- title: "Tutorials" - title: "Tutorials"
url: /tutorials/ url: /tutorials/
children: children:
......
--- ---
title: "DeepSpeed Configuration JSON" title: "DeepSpeed Configuration JSON"
--- ---
## REQUIRED DeepSpeed Config JSON Parameters
### Batch Size Related Parameters
**Note:** configuring ***train\_batch\_size*** is required.
{: .notice--warning}
***train\_batch\_size***: [integer] ***train\_batch\_size***: [integer]
...@@ -9,9 +13,6 @@ title: "DeepSpeed Configuration JSON" ...@@ -9,9 +13,6 @@ title: "DeepSpeed Configuration JSON"
| ------------------------------------------------------------ | ------- | | ------------------------------------------------------------ | ------- |
| The effective training batch size. This is the amount of data samples that leads to one step of model update. ***train\_batch\_size*** is aggregated by the batch size that a single GPU processes in one forward/backward pass (a.k.a., ***train\_step\_batch\_size***), the gradient accumulation steps (a.k.a., ***gradient\_accumulation\_steps***), and the number of GPUs. | `32` | | The effective training batch size. This is the amount of data samples that leads to one step of model update. ***train\_batch\_size*** is aggregated by the batch size that a single GPU processes in one forward/backward pass (a.k.a., ***train\_step\_batch\_size***), the gradient accumulation steps (a.k.a., ***gradient\_accumulation\_steps***), and the number of GPUs. | `32` |
## OPTIONAL DeepSpeed Config JSON Parameters
### Batch Size Related Parameters
***train\_micro\_batch\_size\_per\_gpu***: [integer] ***train\_micro\_batch\_size\_per\_gpu***: [integer]
......
...@@ -116,7 +116,7 @@ the `step` value is stored as part of the `client_sd`. ...@@ -116,7 +116,7 @@ the `step` value is stored as part of the `client_sd`.
DeepSpeed features can be enabled, disabled, or configured using a config JSON DeepSpeed features can be enabled, disabled, or configured using a config JSON
file that should be specified as `args.deepspeed_config`. A sample config file file that should be specified as `args.deepspeed_config`. A sample config file
is shown below. For a full set of features see [ API is shown below. For a full set of features see [ API
doc](/docs/config_json/). doc](/docs/config-json/).
```json ```json
{ {
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment