Unverified Commit 78e776a9 authored by Stas Bekman's avatar Stas Bekman Committed by GitHub
Browse files

[install] fixes/improvements/docs (#752)


Co-authored-by: default avatarJeff Rasley <jerasley@microsoft.com>
parent 6fb16100
......@@ -65,6 +65,14 @@ Available `DS_BUILD` options include:
* `DS_BUILD_STOCHASTIC_TRANSFORMER` builds the stochastic transformer op
* `DS_BUILD_UTILS` builds various optimized utilities
To speed up the build-all process, you can parallelize the compilation process with:
```bash
DS_BUILD_OPS=1 pip install deepspeed --global-option="build_ext" --global-option="-j8"
```
This should complete the full build 2-3 times faster. You can adjust `-j` to specify how many cpu-cores are to be used during the build. In the example it is set to 8 cores.
## Install DeepSpeed from source
......
......@@ -11,7 +11,7 @@ usage() {
echo """
Usage: install.sh [options...]
By default will install deepspeed and all third party dependecies accross all machines listed in
By default will install deepspeed and all third party dependencies across all machines listed in
hostfile (hostfile: /job/hostfile). If no hostfile exists, will only install locally
[optional]
......@@ -45,6 +45,10 @@ while [[ $# -gt 0 ]]
do
key="$1"
case $key in
-l|--local_only)
local_only=1;
shift
;;
-s|--pip_sudo)
pip_sudo=1;
shift
......@@ -69,7 +73,7 @@ case $key in
-H|--hostfile)
hostfile=$2
if [ ! -f $2 ]; then
echo "User provided hostfile does not exist at $hostfile, exiting"
echo "User-provided hostfile does not exist at $hostfile, exiting"
exit 1
fi
shift
......@@ -80,7 +84,7 @@ case $key in
exit 0
;;
*)
echo "Unkown argument(s)"
echo "Unknown argument(s)"
usage
exit 1
shift
......@@ -98,7 +102,7 @@ if [ "$allow_sudo" == "0" ]; then
fi
if [ "$ds_only" == "1" ] && [ "$tp_only" == "1" ]; then
echo "-d and -t are mutually exclusive, only choose one or none"
echo "-d and -t are mutually exclusive, only choose one of the two"
usage
exit 1
fi
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment