Unverified Commit d720fdb6 authored by Shaden Smith's avatar Shaden Smith Committed by GitHub
Browse files

updating website dependencies (#475)

parent 7ddfda85
GEM GEM
remote: https://rubygems.org/ remote: https://rubygems.org/
specs: specs:
activesupport (6.0.3.2) activesupport (6.0.3.4)
concurrent-ruby (~> 1.0, >= 1.0.2) concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (>= 0.7, < 2) i18n (>= 0.7, < 2)
minitest (~> 5.1) minitest (~> 5.1)
...@@ -19,35 +19,36 @@ GEM ...@@ -19,35 +19,36 @@ GEM
concurrent-ruby (1.1.7) concurrent-ruby (1.1.7)
dnsruby (1.61.4) dnsruby (1.61.4)
simpleidn (~> 0.1) simpleidn (~> 0.1)
em-websocket (0.5.1) em-websocket (0.5.2)
eventmachine (>= 0.12.9) eventmachine (>= 0.12.9)
http_parser.rb (~> 0.6.0) http_parser.rb (~> 0.6.0)
ethon (0.12.0) ethon (0.12.0)
ffi (>= 1.3.0) ffi (>= 1.3.0)
eventmachine (1.2.7) eventmachine (1.2.7)
execjs (2.7.0) execjs (2.7.0)
faraday (1.0.1) faraday (1.1.0)
multipart-post (>= 1.2, < 3) multipart-post (>= 1.2, < 3)
ruby2_keywords
ffi (1.13.1) ffi (1.13.1)
forwardable-extended (2.6.0) forwardable-extended (2.6.0)
gemoji (3.0.1) gemoji (3.0.1)
github-pages (207) github-pages (209)
github-pages-health-check (= 1.16.1) github-pages-health-check (= 1.16.1)
jekyll (= 3.9.0) jekyll (= 3.9.0)
jekyll-avatar (= 0.7.0) jekyll-avatar (= 0.7.0)
jekyll-coffeescript (= 1.1.1) jekyll-coffeescript (= 1.1.1)
jekyll-commonmark-ghpages (= 0.1.6) jekyll-commonmark-ghpages (= 0.1.6)
jekyll-default-layout (= 0.1.4) jekyll-default-layout (= 0.1.4)
jekyll-feed (= 0.13.0) jekyll-feed (= 0.15.1)
jekyll-gist (= 1.5.0) jekyll-gist (= 1.5.0)
jekyll-github-metadata (= 2.13.0) jekyll-github-metadata (= 2.13.0)
jekyll-mentions (= 1.5.1) jekyll-mentions (= 1.6.0)
jekyll-optional-front-matter (= 0.3.2) jekyll-optional-front-matter (= 0.3.2)
jekyll-paginate (= 1.1.0) jekyll-paginate (= 1.1.0)
jekyll-readme-index (= 0.3.0) jekyll-readme-index (= 0.3.0)
jekyll-redirect-from (= 0.15.0) jekyll-redirect-from (= 0.16.0)
jekyll-relative-links (= 0.6.1) jekyll-relative-links (= 0.6.1)
jekyll-remote-theme (= 0.4.1) jekyll-remote-theme (= 0.4.2)
jekyll-sass-converter (= 1.5.2) jekyll-sass-converter (= 1.5.2)
jekyll-seo-tag (= 2.6.1) jekyll-seo-tag (= 2.6.1)
jekyll-sitemap (= 1.4.0) jekyll-sitemap (= 1.4.0)
...@@ -55,7 +56,7 @@ GEM ...@@ -55,7 +56,7 @@ GEM
jekyll-theme-architect (= 0.1.1) jekyll-theme-architect (= 0.1.1)
jekyll-theme-cayman (= 0.1.1) jekyll-theme-cayman (= 0.1.1)
jekyll-theme-dinky (= 0.1.1) jekyll-theme-dinky (= 0.1.1)
jekyll-theme-hacker (= 0.1.1) jekyll-theme-hacker (= 0.1.2)
jekyll-theme-leap-day (= 0.1.1) jekyll-theme-leap-day (= 0.1.1)
jekyll-theme-merlot (= 0.1.1) jekyll-theme-merlot (= 0.1.1)
jekyll-theme-midnight (= 0.1.1) jekyll-theme-midnight (= 0.1.1)
...@@ -66,14 +67,14 @@ GEM ...@@ -66,14 +67,14 @@ GEM
jekyll-theme-tactile (= 0.1.1) jekyll-theme-tactile (= 0.1.1)
jekyll-theme-time-machine (= 0.1.1) jekyll-theme-time-machine (= 0.1.1)
jekyll-titles-from-headings (= 0.5.3) jekyll-titles-from-headings (= 0.5.3)
jemoji (= 0.11.1) jemoji (= 0.12.0)
kramdown (= 2.3.0) kramdown (= 2.3.0)
kramdown-parser-gfm (= 1.1.0) kramdown-parser-gfm (= 1.1.0)
liquid (= 4.0.3) liquid (= 4.0.3)
mercenary (~> 0.3) mercenary (~> 0.3)
minima (= 2.5.1) minima (= 2.5.1)
nokogiri (>= 1.10.4, < 2.0) nokogiri (>= 1.10.4, < 2.0)
rouge (= 3.19.0) rouge (= 3.23.0)
terminal-table (~> 1.4) terminal-table (~> 1.4)
github-pages-health-check (1.16.1) github-pages-health-check (1.16.1)
addressable (~> 2.3) addressable (~> 2.3)
...@@ -114,16 +115,16 @@ GEM ...@@ -114,16 +115,16 @@ GEM
rouge (>= 2.0, < 4.0) rouge (>= 2.0, < 4.0)
jekyll-default-layout (0.1.4) jekyll-default-layout (0.1.4)
jekyll (~> 3.0) jekyll (~> 3.0)
jekyll-feed (0.13.0) jekyll-feed (0.15.1)
jekyll (>= 3.7, < 5.0) jekyll (>= 3.7, < 5.0)
jekyll-gist (1.5.0) jekyll-gist (1.5.0)
octokit (~> 4.2) octokit (~> 4.2)
jekyll-github-metadata (2.13.0) jekyll-github-metadata (2.13.0)
jekyll (>= 3.4, < 5.0) jekyll (>= 3.4, < 5.0)
octokit (~> 4.0, != 4.4.0) octokit (~> 4.0, != 4.4.0)
jekyll-include-cache (0.2.0) jekyll-include-cache (0.2.1)
jekyll (>= 3.7, < 5.0) jekyll (>= 3.7, < 5.0)
jekyll-mentions (1.5.1) jekyll-mentions (1.6.0)
html-pipeline (~> 2.3) html-pipeline (~> 2.3)
jekyll (>= 3.7, < 5.0) jekyll (>= 3.7, < 5.0)
jekyll-optional-front-matter (0.3.2) jekyll-optional-front-matter (0.3.2)
...@@ -131,14 +132,15 @@ GEM ...@@ -131,14 +132,15 @@ GEM
jekyll-paginate (1.1.0) jekyll-paginate (1.1.0)
jekyll-readme-index (0.3.0) jekyll-readme-index (0.3.0)
jekyll (>= 3.0, < 5.0) jekyll (>= 3.0, < 5.0)
jekyll-redirect-from (0.15.0) jekyll-redirect-from (0.16.0)
jekyll (>= 3.3, < 5.0) jekyll (>= 3.3, < 5.0)
jekyll-relative-links (0.6.1) jekyll-relative-links (0.6.1)
jekyll (>= 3.3, < 5.0) jekyll (>= 3.3, < 5.0)
jekyll-remote-theme (0.4.1) jekyll-remote-theme (0.4.2)
addressable (~> 2.0) addressable (~> 2.0)
jekyll (>= 3.5, < 5.0) jekyll (>= 3.5, < 5.0)
rubyzip (>= 1.3.0) jekyll-sass-converter (>= 1.0, <= 3.0.0, != 2.0.0)
rubyzip (>= 1.3.0, < 3.0)
jekyll-sass-converter (1.5.2) jekyll-sass-converter (1.5.2)
sass (~> 3.4) sass (~> 3.4)
jekyll-seo-tag (2.6.1) jekyll-seo-tag (2.6.1)
...@@ -155,8 +157,8 @@ GEM ...@@ -155,8 +157,8 @@ GEM
jekyll-theme-dinky (0.1.1) jekyll-theme-dinky (0.1.1)
jekyll (~> 3.5) jekyll (~> 3.5)
jekyll-seo-tag (~> 2.0) jekyll-seo-tag (~> 2.0)
jekyll-theme-hacker (0.1.1) jekyll-theme-hacker (0.1.2)
jekyll (~> 3.5) jekyll (> 3.5, < 5.0)
jekyll-seo-tag (~> 2.0) jekyll-seo-tag (~> 2.0)
jekyll-theme-leap-day (0.1.1) jekyll-theme-leap-day (0.1.1)
jekyll (~> 3.5) jekyll (~> 3.5)
...@@ -190,7 +192,7 @@ GEM ...@@ -190,7 +192,7 @@ GEM
jekyll (>= 3.3, < 5.0) jekyll (>= 3.3, < 5.0)
jekyll-watch (2.2.1) jekyll-watch (2.2.1)
listen (~> 3.0) listen (~> 3.0)
jemoji (0.11.1) jemoji (0.12.0)
gemoji (~> 3.0) gemoji (~> 3.0)
html-pipeline (~> 2.2) html-pipeline (~> 2.2)
jekyll (>= 3.0, < 5.0) jekyll (>= 3.0, < 5.0)
...@@ -208,7 +210,7 @@ GEM ...@@ -208,7 +210,7 @@ GEM
jekyll (>= 3.5, < 5.0) jekyll (>= 3.5, < 5.0)
jekyll-feed (~> 0.9) jekyll-feed (~> 0.9)
jekyll-seo-tag (~> 2.1) jekyll-seo-tag (~> 2.1)
minimal-mistakes-jekyll (4.20.1) minimal-mistakes-jekyll (4.20.2)
jekyll (>= 3.7, < 5.0) jekyll (>= 3.7, < 5.0)
jekyll-feed (~> 0.1) jekyll-feed (~> 0.1)
jekyll-gist (~> 1.5) jekyll-gist (~> 1.5)
...@@ -229,9 +231,10 @@ GEM ...@@ -229,9 +231,10 @@ GEM
rb-inotify (0.10.1) rb-inotify (0.10.1)
ffi (~> 1.0) ffi (~> 1.0)
rexml (3.2.4) rexml (3.2.4)
rouge (3.19.0) rouge (3.23.0)
ruby-enum (0.8.0) ruby-enum (0.8.0)
i18n i18n
ruby2_keywords (0.0.2)
rubyzip (2.3.0) rubyzip (2.3.0)
safe_yaml (1.0.5) safe_yaml (1.0.5)
sass (3.7.4) sass (3.7.4)
...@@ -251,7 +254,7 @@ GEM ...@@ -251,7 +254,7 @@ GEM
ethon (>= 0.9.0) ethon (>= 0.9.0)
tzinfo (1.2.7) tzinfo (1.2.7)
thread_safe (~> 0.1) thread_safe (~> 0.1)
tzinfo-data (1.2020.1) tzinfo-data (1.2020.3)
tzinfo (>= 1.0.0) tzinfo (>= 1.0.0)
unf (0.1.4) unf (0.1.4)
unf_ext unf_ext
......
...@@ -125,7 +125,7 @@ combinations, which we call 3D parallelism. ...@@ -125,7 +125,7 @@ combinations, which we call 3D parallelism.
## Communication efficiency ## Communication efficiency
Pipeline parallelism of DeepSpeed reduce communication volume during distributed training, which allows users to train multi-billion-parameter models 2–7x faster on clusters with limited network bandwidth. Pipeline parallelism of DeepSpeed reduce communication volume during distributed training, which allows users to train multi-billion-parameter models 2–7x faster on clusters with limited network bandwidth.
![PP Figure](/assests/images/pp-lowbw-gpt2.png) ![Low-bandwidth GPT-2 Performance](/assets/images/pp-lowbw-gpt2.png)
1-bit Adam reduces communication volume by up to 5x while achieving similar convergence efficiency to Adam, allowing for scaling to different types of GPU clusters and networks. [Read more here](https://www.deepspeed.ai/news/2020/09/08/onebit-adam-blog-post.html). 1-bit Adam reduces communication volume by up to 5x while achieving similar convergence efficiency to Adam, allowing for scaling to different types of GPU clusters and networks. [Read more here](https://www.deepspeed.ai/news/2020/09/08/onebit-adam-blog-post.html).
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment