Merge pull request #3302 from microsoft/v2.0-merge

Merge branch v2.0 into master (no squash)

Merge pull request #3302 from microsoft/v2.0-merge
Merge branch v2.0 into master (no squash)
4784cc6c · liuzhe-lz · GitHub · 25db55ca · 349ead41 · 4784cc6c
Unverified Commit 4784cc6c authored Jan 14, 2021 by liuzhe-lz Committed by GitHub Jan 14, 2021
20 changed files
--- a/.gitignore
+++ b/.gitignore
+/nni/version.py
 /nni_node/
 /toolchain/


--- a/README.md
+++ b/README.md
@@ -12,11 +12,11 @@
 [![Version](https://img.shields.io/github/release/Microsoft/nni.svg)](https://github.com/Microsoft/nni/releases) [![Join the chat at https://gitter.im/Microsoft/nni](https://badges.gitter.im/Microsoft/nni.svg)](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 [![Documentation Status](https://readthedocs.org/projects/nni/badge/?version=latest)](https://nni.readthedocs.io/en/latest/?badge=latest)

-[简体中文](README_zh_CN.md)
+[NNI Doc](https://nni.readthedocs.io/) | [简体中文](README_zh_CN.md)

 **NNI (Neural Network Intelligence)** is a lightweight but powerful toolkit to help users **automate** <a href="docs/en_US/FeatureEngineering/Overview.rst">Feature Engineering</a>, <a href="docs/en_US/NAS/Overview.rst">Neural Architecture Search</a>, <a href="docs/en_US/Tuner/BuiltinTuner.rst">Hyperparameter Tuning</a> and <a href="docs/en_US/Compression/Overview.rst">Model Compression</a>.

-The tool manages automated machine learning (AutoML) experiments, **dispatches and runs** experiments' trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in **different training environments** like <a href="docs/en_US/TrainingService/LocalMode.rst">Local Machine</a>, <a href="docs/en_US/TrainingService/RemoteMachineMode.rst">Remote Servers</a>, <a href="docs/en_US/TrainingService/PaiMode.rst">OpenPAI</a>, <a href="docs/en_US/TrainingService/KubeflowMode.rst">Kubeflow</a>, <a href="docs/en_US/TrainingService/FrameworkControllerMode.rst">FrameworkController on K8S (AKS etc.)</a>, <a href="docs/en_US/TrainingService/DLTSMode.rst">DLWorkspace (aka. DLTS)</a>, <a href="docs/en_US/TrainingService/AMLMode.rst">AML (Azure Machine Learning)</a>, <a href="docs/en_US/TrainingService/AdaptDLMode.rst">AdaptDL (aka. ADL)</a> and other cloud options.
+The tool manages automated machine learning (AutoML) experiments, **dispatches and runs** experiments' trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in **different training environments** like <a href="docs/en_US/TrainingService/LocalMode.rst">Local Machine</a>, <a href="docs/en_US/TrainingService/RemoteMachineMode.rst">Remote Servers</a>, <a href="docs/en_US/TrainingService/PaiMode.rst">OpenPAI</a>, <a href="docs/en_US/TrainingService/KubeflowMode.rst">Kubeflow</a>, <a href="docs/en_US/TrainingService/FrameworkControllerMode.rst">FrameworkController on K8S (AKS etc.)</a>, <a href="docs/en_US/TrainingService/DLTSMode.rst">DLWorkspace (aka. DLTS)</a>, <a href="docs/en_US/TrainingService/AMLMode.rst">AML (Azure Machine Learning)</a>, <a href="docs/en_US/TrainingService/AdaptDLMode.rst">AdaptDL (aka. ADL)</a> , other cloud options and even <a href="docs/en_US/TrainingService/HybridMode.rst">Hybrid mode</a>.

 ## **Who should consider using NNI**

@@ -25,7 +25,8 @@ The tool manages automated machine learning (AutoML) experiments, **dispatches a
 * Researchers and data scientists who want to easily **implement and experiment new AutoML algorithms**, may it be: hyperparameter tuning algorithm, neural architect search algorithm or model compression algorithm.
 * ML Platform owners who want to **support AutoML in their platform**.

-### **[NNI v1.9 has been released!](https://github.com/microsoft/nni/releases) &nbsp;<a href="#nni-released-reminder"><img width="48" src="docs/img/release_icon.png"></a>**
+### **[NNI v2.0 has been released!](https://github.com/microsoft/nni/releases) &nbsp;<a href="#nni-released-reminder"><img width="48" src="docs/img/release_icon.png"></a>**
+

 ## **NNI capabilities in a glance**

@@ -171,6 +172,7 @@ Within the following table, we summarized the current NNI capabilities, we are g
      <ul>
        <li><a href="docs/en_US/TrainingService/LocalMode.rst">Local Machine</a></li>
        <li><a href="docs/en_US/TrainingService/RemoteMachineMode.rst">Remote Servers</a></li>
+        <li><a href="docs/en_US/TrainingService/HybridMode.rst">Hybrid mode</a></li>
        <li><a href="docs/en_US/TrainingService/AMLMode.rst">AML(Azure Machine Learning)</a></li>
        <li><b>Kubernetes based services</b></li>
        <ul>
@@ -244,12 +246,10 @@ Note:

 ### **Verify installation**

-The following example is built on TensorFlow 1.x. Make sure **TensorFlow 1.x is used** when running it.
-
 * Download the examples via clone the source code.

  ```bash
-  git clone -b v1.9 https://github.com/Microsoft/nni.git
+  git clone -b v2.0 https://github.com/Microsoft/nni.git
  ```

 * Run the MNIST example.
@@ -257,13 +257,13 @@ The following example is built on TensorFlow 1.x. Make sure **TensorFlow 1.x is
  Linux or macOS

  ```bash
-  nnictl create --config nni/examples/trials/mnist-tfv1/config.yml
+  nnictl create --config nni/examples/trials/mnist-pytorch/config.yml
  ```

  Windows

-  ```bash
-  nnictl create --config nni\examples\trials\mnist-tfv1\config_windows.yml
+  ```powershell
+  nnictl create --config nni\examples\trials\mnist-pytorch\config_windows.yml
  ```

 * Wait for the message `INFO: Successfully started experiment!` in the command line. This message indicates that your experiment has been successfully started. You can explore the experiment using the `Web UI url`.
@@ -301,51 +301,22 @@ You can use these commands to get more information about the experiment
    <th><img src="./docs/img/webui-img/full-detail.png" alt="drawing" width="410" height="300"/></th>
 </table>

-## **Documentation**
-
-* To learn about what's NNI, read the [NNI Overview](https://nni.readthedocs.io/en/latest/Overview.html).
-* To get yourself familiar with how to use NNI, read the [documentation](https://nni.readthedocs.io/en/latest/index.html).
-* To get started and install NNI on your system, please refer to [Install NNI](https://nni.readthedocs.io/en/latest/installation.html).
-
-## **Contributing**
-This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
+## **Releases and Contributing**
+NNI has a monthly release cycle (major releases). Please let us know if you encounter a bug by [filling an issue](https://github.com/microsoft/nni/issues/new/choose).

-When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
+We appreciate all contributions. If you are planning to contribute any bug-fixes, please do so without further discussions.

-This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the Code of [Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact opencode@microsoft.com with any additional questions or comments.
+If you plan to contribute new features, new tuners, new training services, etc. please first open an issue or reuse an exisiting issue, and discuss the feature with us. We will discuss with you on the issue timely or set up conference calls if needed.

-After getting familiar with contribution agreements, you are ready to create your first PR =), follow the NNI developer tutorials to get start:
+To learn more about making a contribution to NNI, please refer to our [How-to contribution page](https://nni.readthedocs.io/en/stable/contribution.html). 

-* We recommend new contributors to start with simple issues: [good first issue](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) or [help-wanted](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22).
-* [NNI developer environment installation tutorial](docs/en_US/Tutorial/SetupNniDeveloperEnvironment.rst)
-* [How to debug](docs/en_US/Tutorial/HowToDebug.rst)
-* If you have any questions on usage, review [FAQ](https://github.com/microsoft/nni/blob/master/docs/en_US/Tutorial/FAQ.rst) first, if there are no relevant issues and answers to your question, try contact NNI dev team and users in [Gitter](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) or [File an issue](https://github.com/microsoft/nni/issues/new/choose) on GitHub.
-* [Customize your own Tuner](docs/en_US/Tuner/CustomizeTuner.rst)
-* [Implement customized TrainingService](docs/en_US/TrainingService/HowToImplementTrainingService.rst)
-* [Implement a new NAS trainer on NNI](docs/en_US/NAS/Advanced.rst)
-* [Customize your own Advisor](docs/en_US/Tuner/CustomizeAdvisor.rst)
+We appreciate all contributions and thank all the contributors!

-## **External Repositories and References**
-With authors' permission, we listed a set of NNI usage examples and relevant articles.
+<a href="https://github.com/microsoft/nni/graphs/contributors"><img src="docs/img/contributors.png" /></a>

-* ### **External Repositories** ###
-   * Run [ENAS](examples/nas/enas/README.md) with NNI
-   * [Automatic Feature Engineering](examples/feature_engineering/auto-feature-engineering/README.md) with NNI
-   * [Hyperparameter Tuning for Matrix Factorization](https://github.com/microsoft/recommenders/blob/master/examples/04_model_select_and_optimize/nni_surprise_svd.ipynb) with NNI
-   * [scikit-nni](https://github.com/ksachdeva/scikit-nni) Hyper-parameter search for scikit-learn pipelines using NNI
-* ### **Relevant Articles** ###
-  * [Hyper Parameter Optimization Comparison](docs/en_US/CommunitySharings/HpoComparison.rst)
-  * [Neural Architecture Search Comparison](docs/en_US/CommunitySharings/NasComparison.rst)
-  * [Parallelizing a Sequential Algorithm TPE](docs/en_US/CommunitySharings/ParallelizingTpeSearch.rst)
-  * [Automatically tuning SVD with NNI](docs/en_US/CommunitySharings/RecommendersSvd.rst)
-  * [Automatically tuning SPTAG with NNI](docs/en_US/CommunitySharings/SptagAutoTune.rst)
-  * [Find thy hyper-parameters for scikit-learn pipelines using Microsoft NNI](https://towardsdatascience.com/find-thy-hyper-parameters-for-scikit-learn-pipelines-using-microsoft-nni-f1015b1224c1)
-  * **Blog (in Chinese)** - [AutoML tools (Advisor, NNI and Google Vizier) comparison](http://gaocegege.com/Blog/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/katib-new#%E6%80%BB%E7%BB%93%E4%B8%8E%E5%88%86%E6%9E%90) by [@gaocegege](https://github.com/gaocegege) - 总结与分析 section of design and implementation of kubeflow/katib
-  * **Blog (in Chinese)** - [A summary of NNI new capabilities in 2019](https://mp.weixin.qq.com/s/7_KRT-rRojQbNuJzkjFMuA) by @squirrelsc

 ## **Feedback**
 * [File an issue](https://github.com/microsoft/nni/issues/new/choose) on GitHub.
-* Ask a question with NNI tags on [Stack Overflow](https://stackoverflow.com/questions/tagged/nni?sort=Newest&edited=true).
 * Discuss on the NNI [Gitter](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) in NNI.

 Join IM discussion groups:

--- a/deployment/deployment-pipelines.yml
+++ b/deployment/deployment-pipelines.yml
-# Copyright (c) Microsoft Corporation.
-# Licensed under the MIT license.
-
-jobs:
- job: 'version_number_validation'
-  pool:
-    vmImage: 'Ubuntu 16.04'
-  strategy:
-    matrix:
-      Python36:
-        PYTHON_VERSION: '3.6'
-  steps:
-  - script: |
-      echo $(build_version)
-      if [[ $(build_version) =~ ^v[0-9](.[0-9]){1,3}$ ]]; then
-        echo 'valid build version $(build_version)'
-        echo `git describe --tags --abbrev=0`
-      else
-        echo 'invalid build version $(build_version)'
-        exit 1
-      fi
-    condition: eq( variables['build_type'], 'prerelease' )
-    displayName: 'Validate prerelease version number'
-
-  - script: |
-      export BRANCH_TAG=`git describe --tags --abbrev=0`
-      echo $BRANCH_TAG
-      if [[ $BRANCH_TAG = $(build_version) && $BRANCH_TAG =~ ^v[0-9](.[0-9]){1,3}$ ]]; then
-        echo 'Build version match branch tag'
-      else
-        echo 'Build version does not match branch tag'
-        exit 1
-      fi
-    condition: eq( variables['build_type'], 'release' )
-    displayName: 'Validate release version number and branch tag'
-
-
- job: 'Build_upload_nni_ubuntu'
-  dependsOn: version_number_validation
-  condition: succeeded()
-  pool:
-    vmImage: 'Ubuntu 16.04'
-  strategy:
-    matrix:
-      Python36:
-        PYTHON_VERSION: '3.6'
-
-  steps:
-  - script: |
-      python3 -m pip install --upgrade pip setuptools --user
-      python3 -m pip install twine --user
-    displayName: 'Install twine'
-
-  - script: |
-      cd deployment/pypi
-      if [ $(build_type) = 'prerelease' ]
-      then
-        # NNI build scripts (Makefile) uses branch tag as package version number
-        git tag $(build_version)
-        echo 'building prerelease package...'
-        make version_ts=true build
-      else
-        echo 'building release package...'
-        make build
-      fi
-    condition: eq( variables['upload_package'], 'true')
-    displayName: 'build nni bdsit_wheel'
-
-  - script: |
-      cd deployment/pypi
-      if [ $(build_type) = 'prerelease' ]
-      then
-        echo 'uploading prerelease package to testpypi...'
-        python3 -m twine upload -u $(testpypi_user) -p $(testpypi_pwd) --repository-url https://test.pypi.org/legacy/ dist/*
-      else
-        echo 'uploading release package to pypi...'
-        python3 -m twine upload -u $(pypi_user) -p $(pypi_pwd) dist/*
-      fi
-    condition: eq( variables['upload_package'], 'true')
-    displayName: 'upload nni package to pypi/testpypi'
-
-  - script: |
-      cd deployment/docker
-
-      if [ $(build_type) = 'prerelease' ]
-      then
-        docker login -u $(docker_hub_dev_user) -p $(docker_hub_dev_pwd)
-        export IMG_NAME=$(dev_docker_img)
-        export IMG_TAG=`git describe --tags --abbrev=0`.`date -u +%y%m%d%H%M`
-        echo 'updating docker file for testpyi...'
-        sed -ie 's/RUN python3 -m pip --no-cache-dir install nni/RUN python3 -m pip install --no-cache-dir --index-url https:\/\/test.pypi.org\/simple --extra-index-url https:\/\/pypi.org\/simple nni/' Dockerfile
-      else
-        docker login -u $(docker_hub_user) -p $(docker_hub_pwd)
-        export IMG_NAME=msranni/nni
-        export IMG_TAG=`git describe --tags --abbrev=0`
-      fi
-      echo $IMG_NAME:$IMG_TAG
-      cat Dockerfile
-      docker build -f Dockerfile -t $IMG_NAME:$IMG_TAG .
-      docker push $IMG_NAME:$IMG_TAG
-
-      if [ $(update_latest_tag) = 'true' ]
-      then
-        docker tag $IMG_NAME:$IMG_TAG $IMG_NAME:latest
-        docker push $IMG_NAME:latest
-      fi
-
-    condition: eq( variables['build_docker_img'], 'true')
-    displayName: 'build and upload nni docker image'
-
- job: 'Build_upload_nni_macos'
-  dependsOn: version_number_validation
-  condition: succeeded()
-  pool:
-    vmImage: 'macOS-10.15'
-  steps:
-  - script: |
-      python3 -m pip install --upgrade pip setuptools --user
-      python3 -m pip install twine --user
-    displayName: 'Install twine'
-
-  - script: |
-      make install-dependencies
-    displayName: 'Install nni dependencies'
-
-  - script: |
-      cd deployment/pypi
-      if [ $(build_type) = 'prerelease' ]
-      then
-        # NNI build scripts (Makefile) uses branch tag as package version number
-        git tag $(build_version)
-        echo 'building prerelease package...'
-        PATH=$HOME/Library/Python/3.8/bin:$PATH make version_ts=true build
-      else
-        echo 'building release package...'
-        PATH=$HOME/Library/Python/3.8/bin:$PATH make build
-      fi
-    condition: eq( variables['upload_package'], 'true')
-    displayName: 'build nni bdsit_wheel'
-
-  - script: |
-      cd deployment/pypi
-      if [ $(build_type) = 'prerelease' ]
-      then
-        echo 'uploading prerelease package to testpypi...'
-        python3 -m twine upload -u $(testpypi_user) -p $(testpypi_pwd) --repository-url https://test.pypi.org/legacy/ dist/*
-      else
-        echo 'uploading release package to pypi...'
-        python3 -m twine upload -u $(pypi_user) -p $(pypi_pwd) dist/*
-      fi
-    condition: eq( variables['upload_package'], 'true')
-    displayName: 'upload nni package to pypi/testpypi'
-
- job: 'Build_upload_nni_win32'
-  dependsOn: version_number_validation
-  condition: succeeded()
-  pool:
-    vmImage: 'vs2017-win2016'
-  strategy:
-    matrix:
-      Python36:
-        PYTHON_VERSION: '3.6'
-  steps:
-  - powershell: |
-      python -m pip install --upgrade pip setuptools
-      python -m pip install twine
-    displayName: 'Install twine'
-
-  - powershell: |
-      cd deployment\pypi
-      if($env:BUILD_TYPE -eq 'prerelease'){
-        # NNI build scripts (powershell) uses branch tag as package version number
-        git tag $env:BUILD_VERSION
-        Write-Host 'building prerelease package...'
-        .\install.ps1 -version_os 32 -version_ts $True
-      }
-      else{
-        Write-Host 'building release package...'
-        .\install.ps1 -version_os 32 -version_ts $False
-      }
-    condition: eq( variables['upload_package'], 'true')
-    displayName: 'build nni bdsit_wheel'
-
-  - powershell: |
-      cd deployment\pypi
-      if($env:BUILD_TYPE -eq 'prerelease'){
-        Write-Host 'uploading prerelease package to testpypi...'
-        python -m twine upload -u $(testpypi_user) -p $(testpypi_pwd) --repository-url https://test.pypi.org/legacy/ dist/*
-      }
-      else{
-        Write-Host 'uploading release package to pypi...'
-        python -m twine upload -u $(pypi_user) -p $(pypi_pwd) dist/*
-      }
-    condition: eq( variables['upload_package'], 'true')
-    displayName: 'upload nni package to pypi/testpypi'
-
- job: 'Build_upload_nni_win_amd64'
-  dependsOn: version_number_validation
-  condition: succeeded()
-  pool:
-    vmImage: 'vs2017-win2016'
-  strategy:
-    matrix:
-      Python36:
-        PYTHON_VERSION: '3.6'
-  steps:
-  - powershell: |
-      python -m pip install --upgrade pip setuptools
-      python -m pip install twine
-    displayName: 'Install twine'
-
-  - powershell: |
-      cd deployment\pypi
-      if($env:BUILD_TYPE -eq 'prerelease'){
-        # NNI build scripts (powershell) uses branch tag as package version number
-        git tag $env:BUILD_VERSION
-        Write-Host 'building prerelease package...'
-        .\install.ps1 -version_os 64 -version_ts $True
-      }
-      else{
-        Write-Host 'building release package...'
-        .\install.ps1 -version_os 64 -version_ts $False
-      }
-    condition: eq( variables['upload_package'], 'true')
-    displayName: 'build nni bdsit_wheel'
-
-  - powershell: |
-      cd deployment\pypi
-      if($env:BUILD_TYPE -eq 'prerelease'){
-        Write-Host 'uploading prerelease package to testpypi...'
-        python -m twine upload -u $(testpypi_user) -p $(testpypi_pwd) --repository-url https://test.pypi.org/legacy/ dist/*
-      }
-      else{
-        Write-Host 'uploading release package to pypi...'
-        python -m twine upload -u $(pypi_user) -p $(pypi_pwd) dist/*
-      }
-    condition: eq( variables['upload_package'], 'true')
-    displayName: 'upload nni package to pypi/testpypi'
--- a/docs/en_US/Assessor/CustomizeAssessor.rst
+++ b/docs/en_US/Assessor/CustomizeAssessor.rst
@@ -60,8 +60,6 @@ The working directory of your assessor is ``<home>/nni-experiments/<experiment_i

 More detail example you could see:

-..
-
-   * :githublink:`medianstop-assessor <src/sdk/pynni/nni/medianstop_assessor>`
-   * :githublink:`curvefitting-assessor <src/sdk/pynni/nni/curvefitting_assessor>`
+* :githublink:`medianstop-assessor <nni/algorithms/hpo/medianstop_assessor.py>`
+* :githublink:`curvefitting-assessor <nni/algorithms/hpo/curvefitting_assessor/>`

--- a/docs/en_US/CommunitySharings/AutoCompletion.rst
+++ b/docs/en_US/CommunitySharings/AutoCompletion.rst
@@ -25,7 +25,9 @@ Step 1. Download ``bash-completion``
   cd ~
   wget https://raw.githubusercontent.com/microsoft/nni/{nni-version}/tools/bash-completion

-Here, {nni-version} should by replaced by the version of NNI, e.g., ``master``\ , ``v1.9``. You can also check the latest ``bash-completion`` script :githublink:`here <tools/bash-completion>`.
+Here, {nni-version} should by replaced by the version of NNI, e.g., ``master``, ``v2.0``. You can also check the latest ``bash-completion`` script :githublink:`here <tools/bash-completion>`.
+
+.. cannot find :githublink:`here <tools/bash-completion>`.

 Step 2. Install the script
 ^^^^^^^^^^^^^^^^^^^^^^^^^^

--- a/docs/en_US/CommunitySharings/HpoComparison.rst
+++ b/docs/en_US/CommunitySharings/HpoComparison.rst
@@ -229,9 +229,9 @@ Storage performance

 **Latency**\ : each IO request will take some time to complete, this is called the average latency. There are several factors that would affect this time including network connection quality and hard disk IO performance.

-**IOPS**\ :** IO operations per second**\ , which means the amount of *read or write operations* that could be done in one seconds time.
+**IOPS**\ : **IO operations per second**\ , which means the amount of *read or write operations* that could be done in one seconds time.

-**IO size**\ :** the size of each IO request**. Depending on the operating system and the application/service that needs disk access it will issue a request to read or write a certain amount of data at the same time.
+**IO size**\ : **the size of each IO request**. Depending on the operating system and the application/service that needs disk access it will issue a request to read or write a certain amount of data at the same time.

 **Throughput (in MB/s) = Average IO size x IOPS** 


--- a/docs/en_US/CommunitySharings/ModelCompressionComparison.rst
+++ b/docs/en_US/CommunitySharings/ModelCompressionComparison.rst
@@ -42,7 +42,7 @@ Experiment Result

 For each dataset/model/pruner combination, we prune the model to different levels by setting a series of target sparsities for the pruner. 

-Here we plot both **Number of Weights - Performances** curve and** FLOPs - Performance** curve. 
+Here we plot both **Number of Weights - Performances** curve and **FLOPs - Performance** curve. 
 As a reference, we also plot the result declared in the paper `AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates <http://arxiv.org/abs/1907.03141>`__ for models VGG16 and ResNet18 on CIFAR-10.

 The experiment result are shown in the following figures:
@@ -50,24 +50,24 @@ The experiment result are shown in the following figures:
 CIFAR-10, VGG16:


-.. image:: ../../../examples/model_compress/comparison_of_pruners/img/performance_comparison_vgg16.png
-   :target: ../../../examples/model_compress/comparison_of_pruners/img/performance_comparison_vgg16.png
+.. image:: ../../../examples/model_compress/pruning/comparison_of_pruners/img/performance_comparison_vgg16.png
+   :target: ../../../examples/model_compress/pruning/comparison_of_pruners/img/performance_comparison_vgg16.png
   :alt: 


 CIFAR-10, ResNet18:


-.. image:: ../../../examples/model_compress/comparison_of_pruners/img/performance_comparison_resnet18.png
-   :target: ../../../examples/model_compress/comparison_of_pruners/img/performance_comparison_resnet18.png
+.. image:: ../../../examples/model_compress/pruning/comparison_of_pruners/img/performance_comparison_resnet18.png
+   :target: ../../../examples/model_compress/pruning/comparison_of_pruners/img/performance_comparison_resnet18.png
   :alt: 


 CIFAR-10, ResNet50:


-.. image:: ../../../examples/model_compress/comparison_of_pruners/img/performance_comparison_resnet50.png
-   :target: ../../../examples/model_compress/comparison_of_pruners/img/performance_comparison_resnet50.png
+.. image:: ../../../examples/model_compress/pruning/comparison_of_pruners/img/performance_comparison_resnet50.png
+   :target: ../../../examples/model_compress/pruning/comparison_of_pruners/img/performance_comparison_resnet50.png
   :alt: 



--- a/docs/en_US/CommunitySharings/NNI_AutoFeatureEng.rst
+++ b/docs/en_US/CommunitySharings/NNI_AutoFeatureEng.rst
@@ -7,7 +7,7 @@ NNI review article from Zhihu: :raw-html:`<an open source project with highly re

 The article is by a NNI user on Zhihu forum. In the article, Garvin had shared his experience on using NNI for Automatic Feature Engineering. We think this article is very useful for users who are interested in using NNI for feature engineering. With author's permission, we translated the original article into English.  

-**原文(source)**\ : `如何看待微软最新发布的AutoML平台NNI？By Garvin Li <https://www.zhihu.com/question/297982959/answer/964961829?utm_source=wechat_session&utm_medium=social&utm_oi=28812108627968&from=singlemessage&isappinstalled=0>`__
+**source**\ : `如何看待微软最新发布的AutoML平台NNI？By Garvin Li <https://www.zhihu.com/question/297982959/answer/964961829?utm_source=wechat_session&utm_medium=social&utm_oi=28812108627968&from=singlemessage&isappinstalled=0>`__

 01 Overview of AutoML
 ---------------------
@@ -24,7 +24,7 @@ Microsoft, to help users design and tune machine learning models, neural network
 architectures, or a complex system’s parameters in an efficient and automatic
 way.

-Link:\ ` https://github.com/Microsoft/nni <https://github.com/Microsoft/nni>`__
+Link: `https://github.com/Microsoft/nni <https://github.com/Microsoft/nni>`__

 In general, most of Microsoft tools have one prominent characteristic: the
 design is highly reasonable (regardless of the technology innovation degree).
@@ -51,7 +51,7 @@ NNI treats AutoFeatureENG as a two-steps-task, feature generation exploration an
 04 Feature Exploration
 ----------------------

-For feature derivation, NNI offers many operations which could automatically generate new features, which list \ `as following <https://github.com/SpongebBob/tabular_automl_NNI/blob/master/AutoFEOp.rst>`__\  :
+For feature derivation, NNI offers many operations which could automatically generate new features, which list \ `as following <https://github.com/SpongebBob/tabular_automl_NNI/blob/master/AutoFEOp.md>`__\  :

 **count**\ : Count encoding is based on replacing categories with their counts computed on the train set, also named frequency encoding.

@@ -111,7 +111,7 @@ To avoid feature explosion and overfitting, feature selection is necessary. In t
   :alt: 


-If you have used **XGBoost** or** GBDT**\ , you would know the algorithm based on tree structure can easily calculate the importance of each feature on results. LightGBM is able to make feature selection naturally.
+If you have used **XGBoost** or **GBDT**\ , you would know the algorithm based on tree structure can easily calculate the importance of each feature on results. LightGBM is able to make feature selection naturally.

 The issue is that selected features might be applicable to *GBDT* (Gradient Boosting Decision Tree), but not to the linear algorithm like *LR* (Logistic Regression).


--- a/docs/en_US/CommunitySharings/NasComparison.rst
+++ b/docs/en_US/CommunitySharings/NasComparison.rst
@@ -61,7 +61,7 @@ To avoid over-fitting in **CIFAR-10**\ , we also compare the models in the other

 We do not change the default fine-tuning technique in their source code. In order to match each task, the codes of input image shape and output numbers are changed.

-Search phase time for all NAS methods is **two days** as well as the retrain time.  Average results are reported based on** three repeat times**. Our evaluation machines have one Nvidia Tesla P100 GPU, 112GB of RAM and one 2.60GHz CPU (Intel E5-2690).
+Search phase time for all NAS methods is **two days** as well as the retrain time.  Average results are reported based on **three repeat times**. Our evaluation machines have one Nvidia Tesla P100 GPU, 112GB of RAM and one 2.60GHz CPU (Intel E5-2690).

 For NAO, it requires too much computing resources, so we only use NAO-WS which provides the pipeline script.


--- a/docs/en_US/CommunitySharings/ParallelizingTpeSearch.rst
+++ b/docs/en_US/CommunitySharings/ParallelizingTpeSearch.rst
@@ -176,8 +176,8 @@ Note: The total number of samples per test is 240 (ensure that the budget is equ
 References
 ----------

-[1] James Bergstra, Remi Bardenet, Yoshua Bengio, Balazs Kegl. "Algorithms for Hyper-Parameter Optimization". `Link <https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf>`__
+[1] James Bergstra, Remi Bardenet, Yoshua Bengio, Balazs Kegl. `Algorithms for Hyper-Parameter Optimization. <https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf>`__

-[2] Meng-Hiot Lim, Yew-Soon Ong. "Computational Intelligence in Expensive Optimization Problems". `Link <https://link.springer.com/content/pdf/10.1007%2F978-3-642-10701-6.pdf>`__
+[2] Meng-Hiot Lim, Yew-Soon Ong. `Computational Intelligence in Expensive Optimization Problems. <https://link.springer.com/content/pdf/10.1007%2F978-3-642-10701-6.pdf>`__

-[3] M. Jordan, J. Kleinberg, B. Scho¨lkopf. "Pattern Recognition and Machine Learning". `Link <http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf>`__
+[3] M. Jordan, J. Kleinberg, B. Scho¨lkopf. `Pattern Recognition and Machine Learning. <http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf>`__
--- a/docs/en_US/CommunitySharings/RecommendersSvd.rst
+++ b/docs/en_US/CommunitySharings/RecommendersSvd.rst
@@ -4,12 +4,12 @@ Automatically tuning SVD (NNI in Recommenders)
 In this tutorial, we first introduce a github repo `Recommenders <https://github.com/Microsoft/Recommenders>`__. It is a repository that provides examples and best practices for building recommendation systems, provided as Jupyter notebooks. It has various models that are popular and widely deployed in recommendation systems. To provide a complete end-to-end experience, they present each example in five key tasks, as shown below:


-* `Prepare Data <https://github.com/Microsoft/Recommenders/blob/master/notebooks/01_prepare_data/README.rst>`__\ : Preparing and loading data for each recommender algorithm.
-* `Model <https://github.com/Microsoft/Recommenders/blob/master/notebooks/02_model/README.rst>`__\ : Building models using various classical and deep learning recommender algorithms such as Alternating Least Squares (\ `ALS <https://spark.apache.org/docs/latest/api/python/_modules/pyspark/ml/recommendation.html#ALS>`__\ ) or eXtreme Deep Factorization Machines (\ `xDeepFM <https://arxiv.org/abs/1803.05170>`__\ ).
-* `Evaluate <https://github.com/Microsoft/Recommenders/blob/master/notebooks/03_evaluate/README.rst>`__\ : Evaluating algorithms with offline metrics.
-* `Model Select and Optimize <https://github.com/Microsoft/Recommenders/blob/master/notebooks/04_model_select_and_optimize/README.rst>`__\ : Tuning and optimizing hyperparameters for recommender models.
-* `Operationalize <https://github.com/Microsoft/Recommenders/blob/master/notebooks/05_operationalize/README.rst>`__\ : Operationalizing models in a production environment on Azure.
+* `Prepare Data <https://github.com/microsoft/recommenders/tree/master/examples/01_prepare_data>`__\ : Preparing and loading data for each recommender algorithm.
+* Model(`collaborative filtering algorithms <https://github.com/microsoft/recommenders/tree/master/examples/02_model_collaborative_filtering>`__\ , `content-based filtering algorithms <https://github.com/microsoft/recommenders/tree/master/examples/02_model_content_based_filtering>`__\ , `hybrid algorithms <https://github.com/microsoft/recommenders/tree/master/examples/02_model_hybrid>`__\ ): Building models using various classical and deep learning recommender algorithms such as Alternating Least Squares (\ `ALS <https://spark.apache.org/docs/latest/api/python/_modules/pyspark/ml/recommendation.html#ALS>`__\ ) or eXtreme Deep Factorization Machines (\ `xDeepFM <https://arxiv.org/abs/1803.05170>`__\ ).
+* `Evaluate <https://github.com/microsoft/recommenders/tree/master/examples/03_evaluate>`__\ : Evaluating algorithms with offline metrics.
+* `Model Select and Optimize <https://github.com/microsoft/recommenders/tree/master/examples/04_model_select_and_optimize>`__\ : Tuning and optimizing hyperparameters for recommender models.
+* `Operationalize <https://github.com/microsoft/recommenders/tree/master/examples/05_operationalize>`__\ : Operationalizing models in a production environment on Azure.

-The fourth task is tuning and optimizing the model's hyperparameters, this is where NNI could help. To give a concrete example that NNI tunes the models in Recommenders, let's demonstrate with the model `SVD <https://github.com/Microsoft/Recommenders/blob/master/notebooks/02_model/surprise_svd_deep_dive.ipynb>`__\ , and data Movielens100k. There are more than 10 hyperparameters to be tuned in this model.
+The fourth task is tuning and optimizing the model's hyperparameters, this is where NNI could help. To give a concrete example that NNI tunes the models in Recommenders, let's demonstrate with the model `SVD <https://github.com/microsoft/recommenders/blob/master/examples/02_model_collaborative_filtering/surprise_svd_deep_dive.ipynb>`__\ , and data Movielens100k. There are more than 10 hyperparameters to be tuned in this model.

-`This Jupyter notebook <https://github.com/Microsoft/Recommenders/blob/master/notebooks/04_model_select_and_optimize/nni_surprise_svd.ipynb>`__ provided by Recommenders is a very detailed step-by-step tutorial for this example. It uses different built-in tuning algorithms in NNI, including ``Annealing``\ , ``SMAC``\ , ``Random Search``\ , ``TPE``\ , ``Hyperband``\ , ``Metis`` and ``Evolution``. Finally, the results of different tuning algorithms are compared. Please go through this notebook to learn how to use NNI to tune SVD model, then you could further use NNI to tune other models in Recommenders.
+This `Jupyter notebook <https://github.com/microsoft/recommenders/blob/master/examples/04_model_select_and_optimize/nni_surprise_svd.ipynb>`__ provided by Recommenders is a very detailed step-by-step tutorial for this example. It uses different built-in tuning algorithms in NNI, including ``Annealing``\ , ``SMAC``\ , ``Random Search``\ , ``TPE``\ , ``Hyperband``\ , ``Metis`` and ``Evolution``. Finally, the results of different tuning algorithms are compared. Please go through this notebook to learn how to use NNI to tune SVD model, then you could further use NNI to tune other models in Recommenders.
--- a/docs/en_US/CommunitySharings/SptagAutoTune.rst
+++ b/docs/en_US/CommunitySharings/SptagAutoTune.rst
@@ -6,4 +6,4 @@ Automatically tuning SPTAG with NNI
 This library assumes that the samples are represented as vectors and that the vectors can be compared by L2 distances or cosine distances. Vectors returned for a query vector are the vectors that have smallest L2 distance or cosine distances with the query vector.
 SPTAG provides two methods: kd-tree and relative neighborhood graph (SPTAG-KDT) and balanced k-means tree and relative neighborhood graph (SPTAG-BKT). SPTAG-KDT is advantageous in index building cost, and SPTAG-BKT is advantageous in search accuracy in very high-dimensional data.

-In SPTAG, there are tens of parameters that can be tuned for specified scenarios or datasets. NNI is a great tool for automatically tuning those parameters. The authors of SPTAG tried NNI for the auto tuning and found good-performing parameters easily, thus, they shared the practice of tuning SPTAG on NNI in their document `here <https://github.com/microsoft/SPTAG/blob/master/docs/Parameters.rst>`__. Please refer to it for detailed tutorial.
+In SPTAG, there are tens of parameters that can be tuned for specified scenarios or datasets. NNI is a great tool for automatically tuning those parameters. The authors of SPTAG tried NNI for the auto tuning and found good-performing parameters easily, thus, they shared the practice of tuning SPTAG on NNI in their document `here <https://github.com/microsoft/SPTAG/blob/master/docs/Parameters.md>`__. Please refer to it for detailed tutorial.
--- a/docs/en_US/CommunitySharings/autosys.rst
+++ b/docs/en_US/CommunitySharings/autosys.rst
@@ -2,7 +2,7 @@
 Automatic System Tuning
 #######################

-The performance of systems, such as database, tensor operator implementaion, often need to be tuned to adapt to specific hardware configuration, targeted workload, etc. Manually tuning a system is complicated and often requires detailed understanding of hardware and workload. NNI can make such tasks much easier and help system owners find the best configuration to the system automatically. The detailed design philosophy of automatic system tuning can be found in [this paper](https://dl.acm.org/doi/10.1145/3352020.3352031). The following are some typical cases that NNI can help.
+The performance of systems, such as database, tensor operator implementaion, often need to be tuned to adapt to specific hardware configuration, targeted workload, etc. Manually tuning a system is complicated and often requires detailed understanding of hardware and workload. NNI can make such tasks much easier and help system owners find the best configuration to the system automatically. The detailed design philosophy of automatic system tuning can be found in this `paper <https://dl.acm.org/doi/10.1145/3352020.3352031>`__\ . The following are some typical cases that NNI can help.

 ..  toctree::
    :maxdepth: 1

--- a/docs/en_US/CommunitySharings/community_sharings.rst
+++ b/docs/en_US/CommunitySharings/community_sharings.rst
@@ -4,6 +4,8 @@ Use Cases and Solutions

 Different from the tutorials and examples in the rest of the document which show the usage of a feature, this part mainly introduces end-to-end scenarios and use cases to help users further understand how NNI can help them. NNI can be widely adopted in various scenarios. We also encourage community contributors to share their AutoML practices especially the NNI usage practices from their experience.

+Use Cases and Solutions
+=======================
 ..  toctree::
    :maxdepth: 2

@@ -14,3 +16,24 @@ Different from the tutorials and examples in the rest of the document which show
    Performance measurement, comparison and analysis <perf_compare>
    Use NNI on Google Colab <NNI_colab_support>
    Auto Completion for nnictl Commands <AutoCompletion>
+
+External Repositories and References
+====================================
+With authors' permission, we listed a set of NNI usage examples and relevant articles.
+
+External Repositories
+===================== 
+   * `Hyperparameter Tuning for Matrix Factorization <https://github.com/microsoft/recommenders/blob/master/examples/04_model_select_and_optimize/nni_surprise_svd.ipynb>`__ with NNI
+   * `scikit-nni <https://github.com/ksachdeva/scikit-nni>`__ Hyper-parameter search for scikit-learn pipelines using NNI
+
+Relevant Articles
+=================
+  * `Hyper Parameter Optimization Comparison <./HpoComparison.rst>`__ 
+  * `Neural Architecture Search Comparison <./NasComparison.rst>`__ 
+  * `Parallelizing a Sequential Algorithm TPE <./ParallelizingTpeSearch.rst>`__ 
+  * `Automatically tuning SVD with NNI <./RecommendersSvd.rst>`__ 
+  * `Automatically tuning SPTAG with NNI <./SptagAutoTune.rst>`__ 
+  * `Find thy hyper-parameters for scikit-learn pipelines using Microsoft NNI <https://towardsdatascience.com/find-thy-hyper-parameters-for-scikit-learn-pipelines-using-microsoft-nni-f1015b1224c1>`__ 
+  * `(in Chinese) AutoML tools (Advisor, NNI and Google Vizier) comparison <http://gaocegege.com/Blog/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/katib-new#%E6%80%BB%E7%BB%93%E4%B8%8E%E5%88%86%E6%9E%90>`__  
+  * `(in Chinese) A summary of NNI new capabilities in 2019 <https://mp.weixin.qq.com/s/7_KRT-rRojQbNuJzkjFMuA>`__
+
--- a/docs/en_US/Compression/AutoPruningUsingTuners.rst
+++ b/docs/en_US/Compression/AutoPruningUsingTuners.rst
@@ -15,7 +15,7 @@ You can easily compress a model with NNI compression. Take pruning for example,
   pruner = LevelPruner(model, config_list)
   pruner.compress()

-The 'default' op_type stands for the module types defined in :githublink:`default_layers.py <src/sdk/pynni/nni/compression/pytorch/default_layers.py>` for pytorch.
+The 'default' op_type stands for the module types defined in :githublink:`default_layers.py <nni/compression/pytorch/default_layers.py>` for pytorch.

 Therefore ``{ 'sparsity': 0.8, 'op_types': ['default'] }``\ means that **all layers with specified op_types will be compressed with the same 0.8 sparsity**. When ``pruner.compress()`` called, the model is compressed with masks and after that you can normally fine tune this model and **pruned weights won't be updated** which have been masked.


--- a/docs/en_US/Compression/CustomizeCompressor.rst
+++ b/docs/en_US/Compression/CustomizeCompressor.rst
@@ -5,7 +5,7 @@ Customize New Compression Algorithm

 In order to simplify the process of writing new compression algorithms, we have designed simple and flexible programming interface, which covers pruning and quantization. Below, we first demonstrate how to customize a new pruning algorithm and then demonstrate how to customize a new quantization algorithm.

-**Important Note** To better understand how to customize new pruning/quantization algorithms, users should first understand the framework that supports various pruning algorithms in NNI. Reference `Framework overview of model compression </Compression/Framework.html>`__
+**Important Note** To better understand how to customize new pruning/quantization algorithms, users should first understand the framework that supports various pruning algorithms in NNI. Reference `Framework overview of model compression <../Compression/Framework.rst>`__

 Customize a new pruning algorithm
 ---------------------------------
@@ -28,7 +28,7 @@ An implementation of ``weight masker`` may look like this:
           # mask = ...
           return {'weight_mask': mask}

-You can reference nni provided :githublink:`weight masker <src/sdk/pynni/nni/compression/pytorch/pruning/structured_pruning.py>` implementations to implement your own weight masker.
+You can reference nni provided :githublink:`weight masker <nni/algorithms/compression/pytorch/pruning/structured_pruning.py>` implementations to implement your own weight masker.

 A basic ``pruner`` looks likes this:

@@ -52,7 +52,7 @@ A basic ``pruner`` looks likes this:
               wrapper.if_calculated = True
               return masks

-Reference nni provided :githublink:`pruner <src/sdk/pynni/nni/compression/pytorch/pruning/one_shot.py>` implementations to implement your own pruner class.
+Reference nni provided :githublink:`pruner <nni/algorithms/compression/pytorch/pruning/one_shot.py>` implementations to implement your own pruner class.

 ----


--- a/docs/en_US/Compression/DependencyAware.rst
+++ b/docs/en_US/Compression/DependencyAware.rst
@@ -3,7 +3,7 @@ Dependency-aware Mode for Filter Pruning

 Currently, we have several filter pruning algorithm for the convolutional layers: FPGM Pruner, L1Filter Pruner, L2Filter Pruner, Activation APoZ Rank Filter Pruner, Activation Mean Rank Filter Pruner, Taylor FO On Weight Pruner. In these filter pruning algorithms, the pruner will prune each convolutional layer separately. While pruning a convolution layer, the algorithm will quantify the importance of each filter based on some specific rules(such as l1-norm), and prune the less important filters.

-As `dependency analysis utils <./CompressionUtils.md>`__ shows, if the output channels of two convolutional layers(conv1, conv2) are added together, then these two conv layers have channel dependency with each other(more details please see `Compression Utils <./CompressionUtils.rst>`__\ ). Take the following figure as an example.
+As `dependency analysis utils <./CompressionUtils.rst>`__ shows, if the output channels of two convolutional layers(conv1, conv2) are added together, then these two conv layers have channel dependency with each other(more details please see `Compression Utils <./CompressionUtils.rst>`__\ ). Take the following figure as an example.


 .. image:: ../../img/mask_conflict.jpg

--- a/docs/en_US/Compression/ModelSpeedup.rst
+++ b/docs/en_US/Compression/ModelSpeedup.rst
@@ -37,7 +37,7 @@ Usage
   out = model(dummy_input)
   print('elapsed time: ', time.time() - start)

-For complete examples please refer to :githublink:`the code <examples/model_compress/model_speedup.py>`
+For complete examples please refer to :githublink:`the code <examples/model_compress/pruning/model_speedup.py>`

 NOTE: The current implementation supports PyTorch 1.3.1 or newer.

@@ -51,7 +51,7 @@ For PyTorch we can only replace modules, if functions in ``forward`` should be r
 Speedup Results of Examples
 ---------------------------

-The code of these experiments can be found :githublink:`here <examples/model_compress/model_speedup.py>`.
+The code of these experiments can be found :githublink:`here <examples/model_compress/pruning/model_speedup.py>`.

 slim pruner example
 ^^^^^^^^^^^^^^^^^^^
@@ -188,3 +188,13 @@ input tensor: ``torch.randn(64, 3, 32, 32)``
     - 0.12421
     - 0.087113

+
+SimulatedAnnealing pruner example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In this experiment, we use SimulatedAnnealing pruner to prune the resnet18 on the cifar10 dataset.
+We measure the latencies and accuracies of the pruned model under different sparsity ratios, as shown in the following figure.
+The latency is measured on one V100 GPU and the input tensor is  ``torch.randn(128, 3, 32, 32)``.
+
+
+.. image:: ../../img/SA_latency_accuracy.png
\ No newline at end of file
--- a/docs/en_US/Compression/Overview.rst
+++ b/docs/en_US/Compression/Overview.rst
@@ -32,39 +32,39 @@ Pruning algorithms compress the original network by removing redundant weights o

   * - Name
     - Brief Introduction of Algorithm
-   * - `Level Pruner </Compression/Pruner.html#level-pruner>`__
+   * - `Level Pruner <Pruner.rst#level-pruner>`__
     - Pruning the specified ratio on each weight based on absolute values of weights
-   * - `AGP Pruner </Compression/Pruner.html#agp-pruner>`__
+   * - `AGP Pruner <../Compression/Pruner.rst#agp-pruner>`__
     - Automated gradual pruning (To prune, or not to prune: exploring the efficacy of pruning for model compression) `Reference Paper <https://arxiv.org/abs/1710.01878>`__
-   * - `Lottery Ticket Pruner </Compression/Pruner.html#lottery-ticket-hypothesis>`__
+   * - `Lottery Ticket Pruner <../Compression/Pruner.rst#lottery-ticket-hypothesis>`__
     - The pruning process used by "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks". It prunes a model iteratively. `Reference Paper <https://arxiv.org/abs/1803.03635>`__
-   * - `FPGM Pruner </Compression/Pruner.html#fpgm-pruner>`__
+   * - `FPGM Pruner <../Compression/Pruner.rst#fpgm-pruner>`__
     - Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration `Reference Paper <https://arxiv.org/pdf/1811.00250.pdf>`__
-   * - `L1Filter Pruner </Compression/Pruner.html#l1filter-pruner>`__
+   * - `L1Filter Pruner <../Compression/Pruner.rst#l1filter-pruner>`__
     - Pruning filters with the smallest L1 norm of weights in convolution layers (Pruning Filters for Efficient Convnets) `Reference Paper <https://arxiv.org/abs/1608.08710>`__
-   * - `L2Filter Pruner </Compression/Pruner.html#l2filter-pruner>`__
+   * - `L2Filter Pruner <../Compression/Pruner.rst#l2filter-pruner>`__
     - Pruning filters with the smallest L2 norm of weights in convolution layers
-   * - `ActivationAPoZRankFilterPruner </Compression/Pruner.html#activationapozrankfilterpruner>`__
+   * - `ActivationAPoZRankFilterPruner <../Compression/Pruner.rst#activationapozrankfilter-pruner>`__
     - Pruning filters based on the metric APoZ (average percentage of zeros) which measures the percentage of zeros in activations of (convolutional) layers. `Reference Paper <https://arxiv.org/abs/1607.03250>`__
-   * - `ActivationMeanRankFilterPruner </Compression/Pruner.html#activationmeanrankfilterpruner>`__
+   * - `ActivationMeanRankFilterPruner <../Compression/Pruner.rst#activationmeanrankfilter-pruner>`__
     - Pruning filters based on the metric that calculates the smallest mean value of output activations
-   * - `Slim Pruner </Compression/Pruner.html#slim-pruner>`__
+   * - `Slim Pruner <../Compression/Pruner.rst#slim-pruner>`__
     - Pruning channels in convolution layers by pruning scaling factors in BN layers(Learning Efficient Convolutional Networks through Network Slimming) `Reference Paper <https://arxiv.org/abs/1708.06519>`__
-   * - `TaylorFO Pruner </Compression/Pruner.html#taylorfoweightfilterpruner>`__
+   * - `TaylorFO Pruner <../Compression/Pruner.rst#taylorfoweightfilter-pruner>`__
     - Pruning filters based on the first order taylor expansion on weights(Importance Estimation for Neural Network Pruning) `Reference Paper <http://jankautz.com/publications/Importance4NNPruning_CVPR19.pdf>`__
-   * - `ADMM Pruner </Compression/Pruner.html#admm-pruner>`__
+   * - `ADMM Pruner <../Compression/Pruner.rst#admm-pruner>`__
     - Pruning based on ADMM optimization technique `Reference Paper <https://arxiv.org/abs/1804.03294>`__
-   * - `NetAdapt Pruner </Compression/Pruner.html#netadapt-pruner>`__
+   * - `NetAdapt Pruner <../Compression/Pruner.rst#netadapt-pruner>`__
     - Automatically simplify a pretrained network to meet the resource budget by iterative pruning  `Reference Paper <https://arxiv.org/abs/1804.03230>`__
-   * - `SimulatedAnnealing Pruner </Compression/Pruner.html#simulatedannealing-pruner>`__
+   * - `SimulatedAnnealing Pruner <../Compression/Pruner.rst#simulatedannealing-pruner>`__
     - Automatic pruning with a guided heuristic search method, Simulated Annealing algorithm `Reference Paper <https://arxiv.org/abs/1907.03141>`__
-   * - `AutoCompress Pruner </Compression/Pruner.html#autocompress-pruner>`__
+   * - `AutoCompress Pruner <../Compression/Pruner.rst#autocompress-pruner>`__
     - Automatic pruning by iteratively call SimulatedAnnealing Pruner and ADMM Pruner `Reference Paper <https://arxiv.org/abs/1907.03141>`__
-   * - `AMC Pruner </Compression/Pruner.html#amc-pruner>`__
+   * - `AMC Pruner <../Compression/Pruner.rst#amc-pruner>`__
     - AMC: AutoML for Model Compression and Acceleration on Mobile Devices `Reference Paper <https://arxiv.org/pdf/1802.03494.pdf>`__


-You can refer to this :githublink:`benchmark <docs/en_US/CommunitySharings/ModelCompressionComparison.rst>` for the performance of these pruners on some benchmark problems.
+You can refer to this `benchmark <../CommunitySharings/ModelCompressionComparison.rst>`__ for the performance of these pruners on some benchmark problems.

 Quantization Algorithms
 ^^^^^^^^^^^^^^^^^^^^^^^
@@ -77,13 +77,13 @@ Quantization algorithms compress the original network by reducing the number of

   * - Name
     - Brief Introduction of Algorithm
-   * - `Naive Quantizer </Compression/Quantizer.html#naive-quantizer>`__
+   * - `Naive Quantizer <../Compression/Quantizer.rst#naive-quantizer>`__
     - Quantize weights to default 8 bits
-   * - `QAT Quantizer </Compression/Quantizer.html#qat-quantizer>`__
+   * - `QAT Quantizer <../Compression/Quantizer.rst#qat-quantizer>`__
     - Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. `Reference Paper <http://openaccess.thecvf.com/content_cvpr_2018/papers/Jacob_Quantization_and_Training_CVPR_2018_paper.pdf>`__
-   * - `DoReFa Quantizer </Compression/Quantizer.html#dorefa-quantizer>`__
+   * - `DoReFa Quantizer <../Compression/Quantizer.rst#dorefa-quantizer>`__
     - DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients. `Reference Paper <https://arxiv.org/abs/1606.06160>`__
-   * - `BNN Quantizer </Compression/Quantizer.html#bnn-quantizer>`__
+   * - `BNN Quantizer <../Compression/Quantizer.rst#bnn-quantizer>`__
     - Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1. `Reference Paper <https://arxiv.org/abs/1602.02830>`__



--- a/docs/en_US/Compression/Pruner.rst
+++ b/docs/en_US/Compression/Pruner.rst
@@ -113,7 +113,7 @@ User configuration for Slim Pruner
 Reproduced Experiment
 ^^^^^^^^^^^^^^^^^^^^^

-We implemented one of the experiments in `'Learning Efficient Convolutional Networks through Network Slimming' <https://arxiv.org/pdf/1708.06519.pdf>`__\ , we pruned $70\%$ channels in the **VGGNet** for CIFAR-10 in the paper, in which $88.5\%$ parameters are pruned. Our experiments results are as follows:
+We implemented one of the experiments in `Learning Efficient Convolutional Networks through Network Slimming <https://arxiv.org/pdf/1708.06519.pdf>`__\ , we pruned ``70%`` channels in the **VGGNet** for CIFAR-10 in the paper, in which ``88.5%`` parameters are pruned. Our experiments results are as follows:

 .. list-table::
   :header-rows: 1
@@ -133,7 +133,7 @@ We implemented one of the experiments in `'Learning Efficient Convolutional Netw
     - 88.5%


-The experiments code can be found at :githublink:`examples/model_compress <examples/model_compress/>`
+The experiments code can be found at :githublink:`examples/model_compress/pruning/reproduced/slim_torch_cifar10.py <examples/model_compress/pruning/reproduced/slim_torch_cifar10.py>`

 ----

@@ -182,7 +182,7 @@ User configuration for FPGM Pruner
 L1Filter Pruner
 ---------------

-This is an one-shot pruner, In `'PRUNING FILTERS FOR EFFICIENT CONVNETS' <https://arxiv.org/abs/1608.08710>`__\ , authors Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet and Hans Peter Graf.
+This is an one-shot pruner, In `PRUNING FILTERS FOR EFFICIENT CONVNETS <https://arxiv.org/abs/1608.08710>`__\ , authors Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet and Hans Peter Graf.


 .. image:: ../../img/l1filter_pruner.png
@@ -232,7 +232,7 @@ User configuration for L1Filter Pruner
 Reproduced Experiment
 ^^^^^^^^^^^^^^^^^^^^^

-We implemented one of the experiments in `'PRUNING FILTERS FOR EFFICIENT CONVNETS' <https://arxiv.org/abs/1608.08710>`__ with **L1FilterPruner**\ , we pruned** VGG-16** for CIFAR-10 to** VGG-16-pruned-A** in the paper, in which $64\%$ parameters are pruned. Our experiments results are as follows:
+We implemented one of the experiments in `PRUNING FILTERS FOR EFFICIENT CONVNETS <https://arxiv.org/abs/1608.08710>`__ with **L1FilterPruner**\ , we pruned **VGG-16** for CIFAR-10 to **VGG-16-pruned-A** in the paper, in which ``64%`` parameters are pruned. Our experiments results are as follows:

 .. list-table::
   :header-rows: 1
@@ -252,7 +252,7 @@ We implemented one of the experiments in `'PRUNING FILTERS FOR EFFICIENT CONVNET
     - 64.0%


-The experiments code can be found at :githublink:`examples/model_compress <examples/model_compress/>`
+The experiments code can be found at :githublink:`examples/model_compress/pruning/reproduced/L1_torch_cifar10.py <examples/model_compress/pruning/reproduced/L1_torch_cifar10.py>`

 ----

@@ -316,7 +316,7 @@ PyTorch code

 Note: ActivationAPoZRankFilterPruner is used to prune convolutional layers within deep neural networks, therefore the ``op_types`` field supports only convolutional layers.

-You can view :githublink:`example <examples/model_compress/model_prune_torch.py>` for more information.
+You can view :githublink:`example <examples/model_compress/pruning/model_prune_torch.py>` for more information.

 User configuration for ActivationAPoZRankFilter Pruner
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -330,7 +330,7 @@ User configuration for ActivationAPoZRankFilter Pruner
 ActivationMeanRankFilter Pruner
 -------------------------------

-ActivationMeanRankFilterPruner is a pruner which prunes the filters with the smallest importance criterion ``mean activation`` calculated from the output activations of convolution layers to achieve a preset level of network sparsity. The pruning criterion ``mean activation`` is explained in section 2.2 of the paper\ `Pruning Convolutional Neural Networks for Resource Efficient Inference <https://arxiv.org/abs/1611.06440>`__. Other pruning criteria mentioned in this paper will be supported in future release.
+ActivationMeanRankFilterPruner is a pruner which prunes the filters with the smallest importance criterion ``mean activation`` calculated from the output activations of convolution layers to achieve a preset level of network sparsity. The pruning criterion ``mean activation`` is explained in section 2.2 of the paper `Pruning Convolutional Neural Networks for Resource Efficient Inference <https://arxiv.org/abs/1611.06440>`__. Other pruning criteria mentioned in this paper will be supported in future release.

 We also provide a dependency-aware mode for this pruner to get better speedup from the pruning. Please reference `dependency-aware <./DependencyAware.rst>`__ for more details.

@@ -351,7 +351,7 @@ PyTorch code

 Note: ActivationMeanRankFilterPruner is used to prune convolutional layers within deep neural networks, therefore the ``op_types`` field supports only convolutional layers.

-You can view :githublink:`example <examples/model_compress/model_prune_torch.py>` for more information.
+You can view :githublink:`example <examples/model_compress/pruning/model_prune_torch.py>` for more information.

 User configuration for ActivationMeanRankFilterPruner
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -471,7 +471,7 @@ PyTorch code

   pruner.update_epoch(epoch)

-You can view :githublink:`example <examples/model_compress/model_prune_torch.py>` for more information.
+You can view :githublink:`example <examples/model_compress/pruning/model_prune_torch.py>` for more information.

 User configuration for AGP Pruner
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -511,7 +511,7 @@ PyTorch code
   pruner = NetAdaptPruner(model, config_list, short_term_fine_tuner=short_term_fine_tuner, evaluator=evaluator,base_algo='l1', experiment_data_dir='./')
   pruner.compress()

-You can view :githublink:`example <examples/model_compress/auto_pruners_torch.py>` for more information.
+You can view :githublink:`example <examples/model_compress/pruning/auto_pruners_torch.py>` for more information.

 User configuration for NetAdapt Pruner
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -552,7 +552,7 @@ PyTorch code
   pruner = SimulatedAnnealingPruner(model, config_list, evaluator=evaluator, base_algo='l1', cool_down_rate=0.9, experiment_data_dir='./')
   pruner.compress()

-You can view :githublink:`example <examples/model_compress/auto_pruners_torch.py>` for more information.
+You can view :githublink:`example <examples/model_compress/pruning/auto_pruners_torch.py>` for more information.

 User configuration for SimulatedAnnealing Pruner
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -582,7 +582,7 @@ PyTorch code

 .. code-block:: python

-   from nni.algorithms.compression.pytorch.pruning import ADMMPruner
+   from nni.algorithms.compression.pytorch.pruning import AutoCompressPruner
   config_list = [{
           'sparsity': 0.5,
           'op_types': ['Conv2d']
@@ -593,7 +593,7 @@ PyTorch code
               cool_down_rate=0.9, admm_num_iterations=30, admm_training_epochs=5, experiment_data_dir='./')
   pruner.compress()

-You can view :githublink:`example <examples/model_compress/auto_pruners_torch.py>` for more information.
+You can view :githublink:`example <examples/model_compress/pruning/auto_pruners_torch.py>` for more information.

 User configuration for AutoCompress Pruner
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -631,9 +631,9 @@ PyTorch code
   pruner = AMCPruner(model, config_list, evaluator, val_loader, flops_ratio=0.5)
   pruner.compress()

-You can view :githublink:`example <examples/model_compress/amc/>` for more information.
+You can view :githublink:`example <examples/model_compress/pruning/amc/>` for more information.

-User configuration for AutoCompress Pruner
+User configuration for AMC Pruner
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 **PyTorch**
@@ -659,7 +659,7 @@ We implemented one of the experiments in `AMC: AutoML for Model Compression and
     - 50%


-The experiments code can be found at :githublink:`examples/model_compress <examples/model_compress/amc/>`
+The experiments code can be found at :githublink:`examples/model_compress/pruning/ <examples/model_compress/pruning/amc/>`

 ADMM Pruner
 -----------
@@ -693,7 +693,7 @@ PyTorch code
   pruner = ADMMPruner(model, config_list, trainer=trainer, num_iterations=30, epochs=5)
   pruner.compress()

-You can view :githublink:`example <examples/model_compress/auto_pruners_torch.py>` for more information.
+You can view :githublink:`example <examples/model_compress/pruning/auto_pruners_torch.py>` for more information.

 User configuration for ADMM Pruner
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -754,7 +754,7 @@ User configuration for LotteryTicket Pruner
 Reproduced Experiment
 ^^^^^^^^^^^^^^^^^^^^^

-We try to reproduce the experiment result of the fully connected network on MNIST using the same configuration as in the paper. The code can be referred :githublink:`here <examples/model_compress/lottery_torch_mnist_fc.py>`. In this experiment, we prune 10 times, for each pruning we train the pruned model for 50 epochs.
+We try to reproduce the experiment result of the fully connected network on MNIST using the same configuration as in the paper. The code can be referred :githublink:`here <examples/model_compress/pruning/reproduced/lottery_torch_mnist_fc.py>`. In this experiment, we prune 10 times, for each pruning we train the pruned model for 50 epochs.


 .. image:: ../../img/lottery_ticket_mnist_fc.png