[](https://github.com/Microsoft/nni/releases) [](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[](https://github.com/Microsoft/nni/releases) [](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
**NNI (Neural Network Intelligence)** is a lightweight but powerful toolkit to help users **automate**<ahref="docs/en_US/FeatureEngineering/Overview.rst">Feature Engineering</a>, <ahref="docs/en_US/NAS/Overview.rst">Neural Architecture Search</a>, <ahref="docs/en_US/Tuner/BuiltinTuner.rst">Hyperparameter Tuning</a> and <ahref="docs/en_US/Compression/Overview.rst">Model Compression</a>.
**NNI (Neural Network Intelligence)** is a lightweight but powerful toolkit to help users **automate**<ahref="https://nni.readthedocs.io/en/stable/FeatureEngineering/Overview.html">Feature Engineering</a>, <ahref="https://nni.readthedocs.io/en/stable/NAS/Overview.html">Neural Architecture Search</a>, <ahref="https://nni.readthedocs.io/en/stable/Tuner/BuiltinTuner.html">Hyperparameter Tuning</a> and <ahref="https://nni.readthedocs.io/en/stable/Compression/Overview.html">Model Compression</a>.
The tool manages automated machine learning (AutoML) experiments, **dispatches and runs** experiments' trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in **different training environments** like <ahref="docs/en_US/TrainingService/LocalMode.rst">Local Machine</a>, <ahref="docs/en_US/TrainingService/RemoteMachineMode.rst">Remote Servers</a>, <ahref="docs/en_US/TrainingService/PaiMode.rst">OpenPAI</a>, <ahref="docs/en_US/TrainingService/KubeflowMode.rst">Kubeflow</a>, <ahref="docs/en_US/TrainingService/FrameworkControllerMode.rst">FrameworkController on K8S (AKS etc.)</a>, <ahref="docs/en_US/TrainingService/DLTSMode.rst">DLWorkspace (aka. DLTS)</a>, <ahref="docs/en_US/TrainingService/AMLMode.rst">AML (Azure Machine Learning)</a>, <ahref="docs/en_US/TrainingService/AdaptDLMode.rst">AdaptDL (aka. ADL)</a> , other cloud options and even <ahref="docs/en_US/TrainingService/HybridMode.rst">Hybrid mode</a>.
The tool manages automated machine learning (AutoML) experiments, **dispatches and runs** experiments' trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in **different training environments** like <ahref="https://nni.readthedocs.io/en/stable/TrainingService/LocalMode.html">Local Machine</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/RemoteMachineMode.html">Remote Servers</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/PaiMode.html">OpenPAI</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/KubeflowMode.html">Kubeflow</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/FrameworkControllerMode.html">FrameworkController on K8S (AKS etc.)</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/DLTSMode.html">DLWorkspace (aka. DLTS)</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/AMLMode.html">AML (Azure Machine Learning)</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/AdaptDLMode.html">AdaptDL (aka. ADL)</a> , other cloud options and even <ahref="https://nni.readthedocs.io/en/stable/TrainingService/HybridMode.html">Hybrid mode</a>.
## **Who should consider using NNI**
## **Who should consider using NNI**
...
@@ -72,7 +72,7 @@ Within the following table, we summarized the current NNI capabilities, we are g
...
@@ -72,7 +72,7 @@ Within the following table, we summarized the current NNI capabilities, we are g
If you want to try latest code, please [install NNI](https://nni.readthedocs.io/en/latest/installation.html) from source code.
If you want to try latest code, please [install NNI](https://nni.readthedocs.io/en/stable/installation.html) from source code.
For detail system requirements of NNI, please refer to [here](https://nni.readthedocs.io/en/latest/Tutorial/InstallationLinux.html#system-requirements) for Linux & macOS, and [here](https://nni.readthedocs.io/en/latest/Tutorial/InstallationWin.html#system-requirements) for Windows.
For detail system requirements of NNI, please refer to [here](https://nni.readthedocs.io/en/stable/Tutorial/InstallationLinux.html#system-requirements) for Linux & macOS, and [here](https://nni.readthedocs.io/en/stable/Tutorial/InstallationWin.html#system-requirements) for Windows.
Note:
Note:
* If there is any privilege issue, add `--user` to install NNI in the user directory.
* If there is any privilege issue, add `--user` to install NNI in the user directory.
* Currently NNI on Windows supports local, remote and pai mode. Anaconda or Miniconda is highly recommended to install [NNI on Windows](docs/en_US/Tutorial/InstallationWin.rst).
* Currently NNI on Windows supports local, remote and pai mode. Anaconda or Miniconda is highly recommended to install [NNI on Windows](https://nni.readthedocs.io/en/stable/Tutorial/InstallationWin.html).
* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/en_US/Tutorial/FAQ.rst). For FAQ on Windows, please refer to [NNI on Windows](docs/en_US/Tutorial/InstallationWin.rst#faq).
* If there is any error like `Segmentation fault`, please refer to [FAQ](https://nni.readthedocs.io/en/stable/Tutorial/FAQ.html). For FAQ on Windows, please refer to [NNI on Windows](https://nni.readthedocs.io/en/stable/Tutorial/InstallationWin.html#faq).
### **Verify installation**
### **Verify installation**
...
@@ -297,7 +297,7 @@ You can use these commands to get more information about the experiment
...
@@ -297,7 +297,7 @@ You can use these commands to get more information about the experiment
* Open the `Web UI url` in your browser, you can view detail information of the experiment and all the submitted trial jobs as shown below. [Here](docs/en_US/Tutorial/WebUI.rst) are more Web UI pages.
* Open the `Web UI url` in your browser, you can view detailed information of the experiment and all the submitted trial jobs as shown below. [Here](https://nni.readthedocs.io/en/stable/Tutorial/WebUI.html) are more Web UI pages.
This avoids potential issues of counting them of masked models.
This avoids potential issues of counting them of masked models.
*
*
The experiment code can be found :githublink:`here <examples/model_compress/auto_pruners_torch.py>`.
The experiment code can be found :githublink:`here <examples/model_compress/pruning/auto_pruners_torch.py>`.
Experiment Result Rendering
Experiment Result Rendering
^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^
*
*
If you follow the practice in the :githublink:`example <examples/model_compress/auto_pruners_torch.py>`\ , for every single pruning experiment, the experiment result will be saved in JSON format as follows:
If you follow the practice in the :githublink:`example <examples/model_compress/pruning/auto_pruners_torch.py>`\ , for every single pruning experiment, the experiment result will be saved in JSON format as follows:
.. code-block:: json
.. code-block:: json
...
@@ -114,8 +114,8 @@ Experiment Result Rendering
...
@@ -114,8 +114,8 @@ Experiment Result Rendering
}
}
*
*
The experiment results are saved :githublink:`here <examples/model_compress/comparison_of_pruners>`.
The experiment results are saved :githublink:`here <examples/model_compress/pruning/comparison_of_pruners>`.
You can refer to :githublink:`analyze <examples/model_compress/comparison_of_pruners/analyze.py>` to plot new performance comparison figures.
You can refer to :githublink:`analyze <examples/model_compress/pruning/comparison_of_pruners/analyze.py>` to plot new performance comparison figures.
@@ -14,7 +14,9 @@ NNI provides a model compression toolkit to help user compress and speed up thei
...
@@ -14,7 +14,9 @@ NNI provides a model compression toolkit to help user compress and speed up thei
* Provide friendly and easy-to-use compression utilities for users to dive into the compression process and results.
* Provide friendly and easy-to-use compression utilities for users to dive into the compression process and results.
* Concise interface for users to customize their own compression algorithms.
* Concise interface for users to customize their own compression algorithms.
*Note that the interface and APIs are unified for both PyTorch and TensorFlow, currently only PyTorch version has been supported, TensorFlow version will be supported in future.*
.. note::
Since NNI compression algorithms are not meant to compress model while NNI speedup tool can truly compress model and reduce latency. To obtain a truly compact model, users should conduct `model speedup <./ModelSpeedup.rst>`__. The interface and APIs are unified for both PyTorch and TensorFlow, currently only PyTorch version has been supported, TensorFlow version will be supported in future.
@@ -17,18 +17,26 @@ The ``dict``\ s in the ``list`` are applied one by one, that is, the configurati
...
@@ -17,18 +17,26 @@ The ``dict``\ s in the ``list`` are applied one by one, that is, the configurati
There are different keys in a ``dict``. Some of them are common keys supported by all the compression algorithms:
There are different keys in a ``dict``. Some of them are common keys supported by all the compression algorithms:
* **op_types**\ : This is to specify what types of operations to be compressed. 'default' means following the algorithm's default setting.
* **op_types**\ : This is to specify what types of operations to be compressed. 'default' means following the algorithm's default setting. All suported module types are defined in :githublink:`default_layers.py <nni/compression/pytorch/default_layers.py>` for pytorch.
* **op_names**\ : This is to specify by name what operations to be compressed. If this field is omitted, operations will not be filtered by it.
* **op_names**\ : This is to specify by name what operations to be compressed. If this field is omitted, operations will not be filtered by it.
* **exclude**\ : Default is False. If this field is True, it means the operations with specified types and names will be excluded from the compression.
* **exclude**\ : Default is False. If this field is True, it means the operations with specified types and names will be excluded from the compression.
Some other keys are often specific to a certain algorithm, users can refer to `pruning algorithms <./Pruner.rst>`__ and `quantization algorithms <./Quantizer.rst>`__ for the keys allowed by each algorithm.
Some other keys are often specific to a certain algorithm, users can refer to `pruning algorithms <./Pruner.rst>`__ and `quantization algorithms <./Quantizer.rst>`__ for the keys allowed by each algorithm.
A simple example of configuration is shown below:
To prune all ``Conv2d`` layers with the sparsity of 0.6, the configuration can be written as:
.. code-block:: python
.. code-block:: python
[
[{
{
'sparsity': 0.6,
'op_types': ['Conv2d']
}]
To control the sparsity of specific layers, the configuration can be written as:
.. code-block:: python
[{
'sparsity': 0.8,
'sparsity': 0.8,
'op_types': ['default']
'op_types': ['default']
},
},
...
@@ -39,8 +47,7 @@ A simple example of configuration is shown below:
...
@@ -39,8 +47,7 @@ A simple example of configuration is shown below:
{
{
'exclude': True,
'exclude': True,
'op_names': ['op_name3']
'op_names': ['op_name3']
}
}]
]
It means following the algorithm's default setting for compressed operations with sparsity 0.8, but for ``op_name1`` and ``op_name2`` use sparsity 0.6, and do not compress ``op_name3``.
It means following the algorithm's default setting for compressed operations with sparsity 0.8, but for ``op_name1`` and ``op_name2`` use sparsity 0.6, and do not compress ``op_name3``.
...
@@ -84,12 +91,14 @@ The following example shows a more complete ``config_list``\ , it uses ``op_name
...
@@ -84,12 +91,14 @@ The following example shows a more complete ``config_list``\ , it uses ``op_name
'quant_types': ['weight'],
'quant_types': ['weight'],
'quant_bits': 8,
'quant_bits': 8,
'op_names': ['conv1']
'op_names': ['conv1']
}, {
},
{
'quant_types': ['weight'],
'quant_types': ['weight'],
'quant_bits': 4,
'quant_bits': 4,
'quant_start_step': 0,
'quant_start_step': 0,
'op_names': ['conv2']
'op_names': ['conv2']
}, {
},
{
'quant_types': ['weight'],
'quant_types': ['weight'],
'quant_bits': 3,
'quant_bits': 3,
'op_names': ['fc1']
'op_names': ['fc1']
...
@@ -98,8 +107,7 @@ The following example shows a more complete ``config_list``\ , it uses ``op_name
...
@@ -98,8 +107,7 @@ The following example shows a more complete ``config_list``\ , it uses ``op_name
'quant_types': ['weight'],
'quant_types': ['weight'],
'quant_bits': 2,
'quant_bits': 2,
'op_names': ['fc2']
'op_names': ['fc2']
}
}]
]
In this example, 'op_names' is the name of layer and four layers will be quantized to different quant_bits.
In this example, 'op_names' is the name of layer and four layers will be quantized to different quant_bits.
