Unverified Commit 78e776a9 authored by Stas Bekman's avatar Stas Bekman Committed by GitHub
Browse files

[install] fixes/improvements/docs (#752)


Co-authored-by: default avatarJeff Rasley <jerasley@microsoft.com>
parent 6fb16100
...@@ -65,6 +65,14 @@ Available `DS_BUILD` options include: ...@@ -65,6 +65,14 @@ Available `DS_BUILD` options include:
* `DS_BUILD_STOCHASTIC_TRANSFORMER` builds the stochastic transformer op * `DS_BUILD_STOCHASTIC_TRANSFORMER` builds the stochastic transformer op
* `DS_BUILD_UTILS` builds various optimized utilities * `DS_BUILD_UTILS` builds various optimized utilities
To speed up the build-all process, you can parallelize the compilation process with:
```bash
DS_BUILD_OPS=1 pip install deepspeed --global-option="build_ext" --global-option="-j8"
```
This should complete the full build 2-3 times faster. You can adjust `-j` to specify how many cpu-cores are to be used during the build. In the example it is set to 8 cores.
## Install DeepSpeed from source ## Install DeepSpeed from source
......
...@@ -11,7 +11,7 @@ usage() { ...@@ -11,7 +11,7 @@ usage() {
echo """ echo """
Usage: install.sh [options...] Usage: install.sh [options...]
By default will install deepspeed and all third party dependecies accross all machines listed in By default will install deepspeed and all third party dependencies across all machines listed in
hostfile (hostfile: /job/hostfile). If no hostfile exists, will only install locally hostfile (hostfile: /job/hostfile). If no hostfile exists, will only install locally
[optional] [optional]
...@@ -45,6 +45,10 @@ while [[ $# -gt 0 ]] ...@@ -45,6 +45,10 @@ while [[ $# -gt 0 ]]
do do
key="$1" key="$1"
case $key in case $key in
-l|--local_only)
local_only=1;
shift
;;
-s|--pip_sudo) -s|--pip_sudo)
pip_sudo=1; pip_sudo=1;
shift shift
...@@ -69,7 +73,7 @@ case $key in ...@@ -69,7 +73,7 @@ case $key in
-H|--hostfile) -H|--hostfile)
hostfile=$2 hostfile=$2
if [ ! -f $2 ]; then if [ ! -f $2 ]; then
echo "User provided hostfile does not exist at $hostfile, exiting" echo "User-provided hostfile does not exist at $hostfile, exiting"
exit 1 exit 1
fi fi
shift shift
...@@ -80,7 +84,7 @@ case $key in ...@@ -80,7 +84,7 @@ case $key in
exit 0 exit 0
;; ;;
*) *)
echo "Unkown argument(s)" echo "Unknown argument(s)"
usage usage
exit 1 exit 1
shift shift
...@@ -98,7 +102,7 @@ if [ "$allow_sudo" == "0" ]; then ...@@ -98,7 +102,7 @@ if [ "$allow_sudo" == "0" ]; then
fi fi
if [ "$ds_only" == "1" ] && [ "$tp_only" == "1" ]; then if [ "$ds_only" == "1" ] && [ "$tp_only" == "1" ]; then
echo "-d and -t are mutually exclusive, only choose one or none" echo "-d and -t are mutually exclusive, only choose one of the two"
usage usage
exit 1 exit 1
fi fi
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment