Doc for installation (#4630)

docs/source/installation_zh.rst → docs/_removed/installation.rst

-.. c62173d7147a43a13bf2cdf945b82d07
-
 ############
-安装
+Installation
 ############
 
-当前支持在 Linux，macOS 和 Windows 下安装。 还可使用 Docker。
+Currently we support installation on Linux, Mac and Windows. We also allow you to use docker.
 
 ..  toctree::
     :maxdepth: 2
 
-    Linux 和 macOS <Tutorial/InstallationLinux>
+    Linux & Mac <Tutorial/InstallationLinux>
     Windows <Tutorial/InstallationWin>
-    使用 Docker <Tutorial/HowToUseDocker>
\ No newline at end of file
+    Use Docker <Tutorial/HowToUseDocker>
\ No newline at end of file

docs/source/Tutorial/QuickStart.rst

@@ -24,8 +24,6 @@ Windows
 
 .. Note:: If there is an error like ``Segmentation fault``, please refer to the :doc:`FAQ <FAQ>`.
 
-.. Note:: For the system requirements of NNI, please refer to :doc:`Install NNI on Linux & Mac <InstallationLinux>` or :doc:`Windows <InstallationWin>`. If you want to use docker, refer to :doc:`HowToUseDocker <HowToUseDocker>`.
-
 
 "Hello World" example on MNIST
 ------------------------------

docs/source/Tutorial/QuickStart_zh.rst

-.. 8eaeffaa088fa022c601eb18bebd0736
+.. 3b483d31655ac74e05b4c43b70452e08
 
 快速入门
 ==========
@@ -26,8 +26,6 @@ Windows
 
 .. Note:: 如果出现 ``Segmentation fault`` 这样的错误，参考 :doc:`常见问题 <FAQ>` 。
 
-.. Note:: NNI 的系统需求，参考 :doc:`Linux & Mac <InstallationLinux>` 或者 :doc:`Windows <InstallationWin>` 的安装教程。如果想要使用 docker, 参考 :doc:`如何使用 Docker <HowToUseDocker>` 。
-
 
 MNIST 上的 "Hello World"
 ------------------------

docs/source/experiment/adaptdl.rst

@@ -14,7 +14,7 @@ Before starting to use NNI AdaptDL training service, you should have a Kubernete
 #. Prepare a **kubeconfig** file, which will be used by NNI to interact with your Kubernetes API server. By default, NNI manager will use ``$(HOME)/.kube/config`` as kubeconfig file's path. You can also specify other kubeconfig files by setting the **KUBECONFIG** environment variable. Refer this `guideline <https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig>`__ to learn more about kubeconfig.
 #. If your NNI trial job needs GPU resource, you should follow this `guideline <https://github.com/NVIDIA/k8s-device-plugin>`__ to configure **Nvidia device plugin for Kubernetes**.
 #. (Optional) Prepare a **NFS server** and export a general purpose mount as external storage.
-#. Install **NNI**\ , follow the install guide :doc:`../installation`.
+#. Install **NNI**.
 
 Verify the Prerequisites
 ^^^^^^^^^^^^^^^^^^^^^^^^

docs/source/index.rst

@@ -40,6 +40,7 @@ Neural Network Intelligence
     Use Cases and Solutions <CommunitySharings/community_sharings>
     Research and Publications <ResearchPublications>
     FAQ <Tutorial/FAQ>
+    notes/build_from_source
     How to Contribute <contribution>
     Change Log <Release>
 

docs/source/index_zh.rst

-.. c20f29083a3327d6ecad5f73901e7f99
+.. 667579cc0df56e2a258a2222f0f6cd36
 
 ###########################
 Neural Network Intelligence
@@ -10,8 +10,8 @@ Neural Network Intelligence
     :titlesonly:
     :hidden:
 
-    安装 <installation>
     入门<Tutorial/QuickStart>
+    安装 <installation>
     教程<tutorials>
     自动（超参数）调优 <hpo/index>
     神经网络架构搜索<nas/index>
@@ -23,6 +23,7 @@ Neural Network Intelligence
     示例与解决方案<CommunitySharings/community_sharings>
     研究和出版物 <ResearchPublications>
     常见问题 <Tutorial/FAQ>
+    从源代码安装 <notes/build_from_source>
     如何贡献 <contribution>
     更改日志 <Release>
 

docs/source/installation.rst

-############
-Installation
-############
+Install NNI
+===========
 
-Currently we support installation on Linux, Mac and Windows. We also allow you to use docker.
+NNI requires Python >= 3.7.
+It is tested and supported on Ubuntu >= 18.04,
+Windows 10 >= 21H2, and macOS >= 11.
 
-..  toctree::
-    :maxdepth: 2
+Using pip
+---------
 
-    Linux & Mac <Tutorial/InstallationLinux>
-    Windows <Tutorial/InstallationWin>
-    Use Docker <Tutorial/HowToUseDocker>
\ No newline at end of file
+NNI provides official packages for x86-64 CPUs. They can be installed with pip:
+
+.. code-block::
+
+    python -m pip install --upgrade nni
+
+You can check installation with:
+
+.. code-block::
+
+    nnictl --version
+
+On Linux systems without Conda, you may encounter ``bash: nnictl: command not found``.
+In this case you need to add pip script directory to ``PATH``:
+
+.. code-block:: bash
+
+    echo 'export PATH=${PATH}:${HOME}/.local/bin' >> ~/.bashrc
+    source ~/.bashrc
+
+Installing from Source Code
+---------------------------
+
+NNI hosts source code on `GitHub <https://github.com/microsoft/nni>`__.
+
+NNI has experimental support for ARM64 CPUs, including Apple M1.
+It requires to install from source code.
+
+See :doc:`/notes/build_from_source`.
+
+Using Docker
+------------
+
+NNI provides official Docker image on `Docker Hub <https://hub.docker.com/r/msranni/nni>`__.
+
+.. code-block::
+
+    docker pull msranni/nni
+
+Installing Extra Dependencies
+-----------------------------
+
+Some built-in algorithms of NNI requires extra packages.
+Use ``nni[<algorithm-name>]`` to install their dependencies.
+
+For example, to install dependencies of :class:`DNGO tuner<nni.algorithms.hpo.dngo_tuner.DNGOTuner>` :
+
+.. code-block::
+
+    python -m pip install nni[DNGO]
+
+This command will not reinstall NNI itself, even if it was installed in development mode.
+
+Alternatively, you may install all extra dependencies at once:
+
+.. code-block::
+
+    python -m pip install nni[all]
+
+**NOTE**: SMAC tuner depends on swig3, which requires a manual downgrade on Ubuntu:
+
+.. code-block::
+
+    sudo apt install swig3.0
+    sudo rm /usr/bin/swig
+    sudo ln -s swig3.0 /usr/bin/swig

docs/source/notes/build_from_source.rst

+Build from Source
+=================
+
+This article describes how to build and install NNI from `source code`_.
+
+We recommend using latest setuptools:
+
+.. code-block::
+
+    python -m pip install --upgrade setuptools pip wheel
+
+.. _source code: https://github.com/microsoft/nni
+
+Development Build
+-----------------
+
+If you want to build NNI for your own use, we recommend using `development mode`_.
+
+.. code-block::
+
+    python setup.py develop
+
+This will install NNI as symlink, and the version number will be ``999.dev0``.
+
+.. _development mode: https://setuptools.pypa.io/en/latest/userguide/development_mode.html
+
+Release Build
+-------------
+
+To install in release mode, you must first build a wheel.
+NNI does not support setuptools' "install" command.
+
+A release package requires jupyterlab to build the extension:
+
+.. code-block::
+
+    python -m pip install jupyterlab
+
+And you need to set ``NNI_RELEASE`` environment variable, and compile TypeScript modules before "bdist_wheel".
+
+In bash:
+
+.. code-block::
+
+    export NNI_RELEASE=2.7
+    python setup.py build_ts
+    python bdist_wheel
+
+In PowerShell:
+
+.. code-block::
+
+    $env:NNI_RELEASE=2.7
+    python setup.py build_ts
+    python bdist_wheel
+
+If successful, you will find the wheel in ``dist`` directory.
+
+.. note::
+
+    NNI's build process is somewhat complicated.
+    This is due to setuptools and TypeScript not working well together.
+
+    Setuptools require to provide ``package_data``, the full list of package files, before running any command.
+    However it is nearly impossible to predict what files will be generated before invoking TypeScript compiler.
+
+    If you have any solution for this problem, please open an issue to let us know.
+
+Build Docker Image
+------------------
+
+You can build a Docker image with Dockerfile:
+
+.. code-block::
+
+    export NNI_RELEASE=2.7
+    python setup.py build_ts
+    python setup.py bdist_wheel -p manylinux1_x86_64
+    docker build --build-arg NNI_RELEASE=${NNI_RELEASE} -t my/nni .
+
+To build image for other platforms, please edit Dockerfile yourself.
+
+Other Commands and Options
+--------------------------
+
+Clean
+^^^^^
+
+If the build fails, please clean up and try again:
+
+.. code::
+
+    python setup.py clean
+
+Skip compiling TypeScript modules
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This is useful when you have uninstalled NNI from development mode and want to install again.
+
+It will not work if you have never built TypeScript modules before.
+
+.. code::
+
+    python setup.py develop --skip-ts

setup.py

@@ -27,7 +27,7 @@ Uninstall:
 
   $ pip uninstall nni
 
-Remove generated files: (use "--all" to remove toolchain and built wheel)
+Remove generated files: (use "--all" to remove built wheel)
 
   $ python setup.py clean [--all]
 
@@ -125,6 +125,7 @@ def _setup():
             'BOHB': _read_requirements_txt('dependencies/required_extra.txt', 'BOHB'),
             'PPOTuner': _read_requirements_txt('dependencies/required_extra.txt', 'PPOTuner'),
             'DNGO': _read_requirements_txt('dependencies/required_extra.txt', 'DNGO'),
+            'all': _read_requirements_txt('dependencies/required_extra.txt'),
         },
         setup_requires = ['requests'],
 
@@ -230,7 +231,7 @@ class Build(build):
         check_jupyter_lab_version()
 
         if os.path.islink('nni_node/main.js'):
-            sys.exit('A development build already exists. Please uninstall NNI and run "python3 setup.py clean --all".')
+            sys.exit('A development build already exists. Please uninstall NNI and run "python3 setup.py clean".')
         open('nni/version.py', 'w').write(f"__version__ = '{release}'")
         super().run()
 
@@ -268,7 +269,7 @@ class Clean(clean):
 
     def run(self):
         super().run()
-        setup_ts.clean(self._all)
+        setup_ts.clean()
         _clean_temp_files()
         shutil.rmtree('nni.egg-info', ignore_errors=True)
         if self._all:

setup_ts.py

@@ -15,6 +15,7 @@ from io import BytesIO
 import json
 import os
 from pathlib import Path
+import platform
 import shutil
 import subprocess
 import sys
@@ -56,12 +57,13 @@ def build(release):
         symlink_nni_node()
     restore_package()
 
-def clean(clean_all=False):
+def clean():
     """
     Remove TypeScript-related intermediate files.
     Python intermediate files are not touched here.
     """
     shutil.rmtree('nni_node', ignore_errors=True)
+    shutil.rmtree('toolchain', ignore_errors=True)
 
     for file_or_dir in generated_files:
         path = Path(file_or_dir)
@@ -70,13 +72,11 @@ def clean(clean_all=False):
         elif path.is_dir():
             shutil.rmtree(path)
 
-    if clean_all:
-        shutil.rmtree('toolchain', ignore_errors=True)
-
 
 if sys.platform == 'linux' or sys.platform == 'darwin':
     node_executable = 'node'
-    node_spec = f'node-{node_version}-{sys.platform}-x64'
+    _arch = 'x64' if platform.machine() == 'x86_64' else platform.machine()
+    node_spec = f'node-{node_version}-{sys.platform}-' + _arch
     node_download_url = f'https://nodejs.org/dist/{node_version}/{node_spec}.tar.xz'
     node_extractor = lambda data: tarfile.open(fileobj=BytesIO(data), mode='r:xz')
     node_executable_in_tarball = 'bin/node'
@@ -183,16 +183,14 @@ def compile_ts(release):
         _yarn('ts/webui', 'build')
 
     _print('Building JupyterLab extension')
-    if release:
+    try:
         _yarn('ts/jupyter_extension')
         _yarn('ts/jupyter_extension', 'build')
-    else:
-        try:
-            _yarn('ts/jupyter_extension')
-            _yarn('ts/jupyter_extension', 'build')
-        except Exception:
-            _print('Failed to build JupyterLab extension, skip for develop mode', color='yellow')
-            _print(traceback.format_exc(), color='yellow')
+    except Exception:
+        if release:
+            raise
+        _print('Failed to build JupyterLab extension, skip for develop mode', color='yellow')
+        _print(traceback.format_exc(), color='yellow')
 
 
 def symlink_nni_node():