[](https://github.com/Microsoft/nni/releases) [](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
**NNI (Neural Network Intelligence)** is a lightweight but powerful toolkit to help users **automate**<ahref="docs/en_US/FeatureEngineering/Overview.rst">Feature Engineering</a>, <ahref="docs/en_US/NAS/Overview.rst">Neural Architecture Search</a>, <ahref="docs/en_US/Tuner/BuiltinTuner.rst">Hyperparameter Tuning</a> and <ahref="docs/en_US/Compression/Overview.rst">Model Compression</a>.
**NNI (Neural Network Intelligence)** is a lightweight but powerful toolkit to help users **automate**<ahref="https://nni.readthedocs.io/en/stable/FeatureEngineering/Overview.html">Feature Engineering</a>, <ahref="https://nni.readthedocs.io/en/stable/NAS/Overview.html">Neural Architecture Search</a>, <ahref="https://nni.readthedocs.io/en/stable/Tuner/BuiltinTuner.html">Hyperparameter Tuning</a> and <ahref="https://nni.readthedocs.io/en/stable/Compression/Overview.html">Model Compression</a>.
The tool manages automated machine learning (AutoML) experiments, **dispatches and runs** experiments' trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in **different training environments** like <ahref="docs/en_US/TrainingService/LocalMode.rst">Local Machine</a>, <ahref="docs/en_US/TrainingService/RemoteMachineMode.rst">Remote Servers</a>, <ahref="docs/en_US/TrainingService/PaiMode.rst">OpenPAI</a>, <ahref="docs/en_US/TrainingService/KubeflowMode.rst">Kubeflow</a>, <ahref="docs/en_US/TrainingService/FrameworkControllerMode.rst">FrameworkController on K8S (AKS etc.)</a>, <ahref="docs/en_US/TrainingService/DLTSMode.rst">DLWorkspace (aka. DLTS)</a>, <ahref="docs/en_US/TrainingService/AMLMode.rst">AML (Azure Machine Learning)</a>, <ahref="docs/en_US/TrainingService/AdaptDLMode.rst">AdaptDL (aka. ADL)</a> , other cloud options and even <ahref="docs/en_US/TrainingService/HybridMode.rst">Hybrid mode</a>.
The tool manages automated machine learning (AutoML) experiments, **dispatches and runs** experiments' trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in **different training environments** like <ahref="https://nni.readthedocs.io/en/stable/TrainingService/LocalMode.html">Local Machine</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/RemoteMachineMode.html">Remote Servers</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/PaiMode.html">OpenPAI</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/KubeflowMode.html">Kubeflow</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/FrameworkControllerMode.html">FrameworkController on K8S (AKS etc.)</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/DLTSMode.html">DLWorkspace (aka. DLTS)</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/AMLMode.html">AML (Azure Machine Learning)</a>, <ahref="https://nni.readthedocs.io/en/stable/TrainingService/AdaptDLMode.html">AdaptDL (aka. ADL)</a> , other cloud options and even <ahref="https://nni.readthedocs.io/en/stable/TrainingService/HybridMode.html">Hybrid mode</a>.
## **Who should consider using NNI**
...
...
@@ -72,7 +72,7 @@ Within the following table, we summarized the current NNI capabilities, we are g
If you want to try latest code, please [install NNI](https://nni.readthedocs.io/en/latest/installation.html) from source code.
If you want to try latest code, please [install NNI](https://nni.readthedocs.io/en/stable/installation.html) from source code.
For detail system requirements of NNI, please refer to [here](https://nni.readthedocs.io/en/latest/Tutorial/InstallationLinux.html#system-requirements) for Linux & macOS, and [here](https://nni.readthedocs.io/en/latest/Tutorial/InstallationWin.html#system-requirements) for Windows.
For detail system requirements of NNI, please refer to [here](https://nni.readthedocs.io/en/stable/Tutorial/InstallationLinux.html#system-requirements) for Linux & macOS, and [here](https://nni.readthedocs.io/en/stable/Tutorial/InstallationWin.html#system-requirements) for Windows.
Note:
* If there is any privilege issue, add `--user` to install NNI in the user directory.
* Currently NNI on Windows supports local, remote and pai mode. Anaconda or Miniconda is highly recommended to install [NNI on Windows](docs/en_US/Tutorial/InstallationWin.rst).
* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/en_US/Tutorial/FAQ.rst). For FAQ on Windows, please refer to [NNI on Windows](docs/en_US/Tutorial/InstallationWin.rst#faq).
* Currently NNI on Windows supports local, remote and pai mode. Anaconda or Miniconda is highly recommended to install [NNI on Windows](https://nni.readthedocs.io/en/stable/Tutorial/InstallationWin.html).
* If there is any error like `Segmentation fault`, please refer to [FAQ](https://nni.readthedocs.io/en/stable/Tutorial/FAQ.html). For FAQ on Windows, please refer to [NNI on Windows](https://nni.readthedocs.io/en/stable/Tutorial/InstallationWin.html#faq).
### **Verify installation**
...
...
@@ -297,7 +297,7 @@ You can use these commands to get more information about the experiment
* Open the `Web UI url` in your browser, you can view detail information of the experiment and all the submitted trial jobs as shown below. [Here](docs/en_US/Tutorial/WebUI.rst) are more Web UI pages.
* Open the `Web UI url` in your browser, you can view detailed information of the experiment and all the submitted trial jobs as shown below. [Here](https://nni.readthedocs.io/en/stable/Tutorial/WebUI.html) are more Web UI pages.
This avoids potential issues of counting them of masked models.
*
The experiment code can be found :githublink:`here <examples/model_compress/auto_pruners_torch.py>`.
The experiment code can be found :githublink:`here <examples/model_compress/pruning/auto_pruners_torch.py>`.
Experiment Result Rendering
^^^^^^^^^^^^^^^^^^^^^^^^^^^
*
If you follow the practice in the :githublink:`example <examples/model_compress/auto_pruners_torch.py>`\ , for every single pruning experiment, the experiment result will be saved in JSON format as follows:
If you follow the practice in the :githublink:`example <examples/model_compress/pruning/auto_pruners_torch.py>`\ , for every single pruning experiment, the experiment result will be saved in JSON format as follows:
.. code-block:: json
...
...
@@ -114,8 +114,8 @@ Experiment Result Rendering
}
*
The experiment results are saved :githublink:`here <examples/model_compress/comparison_of_pruners>`.
You can refer to :githublink:`analyze <examples/model_compress/comparison_of_pruners/analyze.py>` to plot new performance comparison figures.
The experiment results are saved :githublink:`here <examples/model_compress/pruning/comparison_of_pruners>`.
You can refer to :githublink:`analyze <examples/model_compress/pruning/comparison_of_pruners/analyze.py>` to plot new performance comparison figures.
@@ -14,7 +14,9 @@ NNI provides a model compression toolkit to help user compress and speed up thei
* Provide friendly and easy-to-use compression utilities for users to dive into the compression process and results.
* Concise interface for users to customize their own compression algorithms.
*Note that the interface and APIs are unified for both PyTorch and TensorFlow, currently only PyTorch version has been supported, TensorFlow version will be supported in future.*
.. note::
Since NNI compression algorithms are not meant to compress model while NNI speedup tool can truly compress model and reduce latency. To obtain a truly compact model, users should conduct `model speedup <./ModelSpeedup.rst>`__. The interface and APIs are unified for both PyTorch and TensorFlow, currently only PyTorch version has been supported, TensorFlow version will be supported in future.
Supported Algorithms
--------------------
...
...
@@ -24,7 +26,7 @@ The algorithms include pruning algorithms and quantization algorithms.
Pruning Algorithms
^^^^^^^^^^^^^^^^^^
Pruning algorithms compress the original network by removing redundant weights or channels of layers, which can reduce model complexity and address the over-fitting issue.
Pruning algorithms compress the original network by removing redundant weights or channels of layers, which can reduce model complexity and address the over-fitting issue.
@@ -17,30 +17,37 @@ The ``dict``\ s in the ``list`` are applied one by one, that is, the configurati
There are different keys in a ``dict``. Some of them are common keys supported by all the compression algorithms:
* **op_types**\ : This is to specify what types of operations to be compressed. 'default' means following the algorithm's default setting.
* **op_types**\ : This is to specify what types of operations to be compressed. 'default' means following the algorithm's default setting. All suported module types are defined in :githublink:`default_layers.py <nni/compression/pytorch/default_layers.py>` for pytorch.
* **op_names**\ : This is to specify by name what operations to be compressed. If this field is omitted, operations will not be filtered by it.
* **exclude**\ : Default is False. If this field is True, it means the operations with specified types and names will be excluded from the compression.
Some other keys are often specific to a certain algorithm, users can refer to `pruning algorithms <./Pruner.rst>`__ and `quantization algorithms <./Quantizer.rst>`__ for the keys allowed by each algorithm.
A simple example of configuration is shown below:
To prune all ``Conv2d`` layers with the sparsity of 0.6, the configuration can be written as:
.. code-block:: python
[
{
'sparsity': 0.8,
'op_types': ['default']
},
{
'sparsity': 0.6,
'op_names': ['op_name1', 'op_name2']
},
{
'exclude': True,
'op_names': ['op_name3']
}
]
[{
'sparsity': 0.6,
'op_types': ['Conv2d']
}]
To control the sparsity of specific layers, the configuration can be written as:
.. code-block:: python
[{
'sparsity': 0.8,
'op_types': ['default']
},
{
'sparsity': 0.6,
'op_names': ['op_name1', 'op_name2']
},
{
'exclude': True,
'op_names': ['op_name3']
}]
It means following the algorithm's default setting for compressed operations with sparsity 0.8, but for ``op_name1`` and ``op_name2`` use sparsity 0.6, and do not compress ``op_name3``.
...
...
@@ -62,10 +69,10 @@ bits length of quantization, key is the quantization type, value is the quantiza
.. code-block:: bash
{
quant_bits: {
'weight': 8,
'output': 4,
},
quant_bits: {
'weight': 8,
'output': 4,
},
}
when the value is int type, all quantization types share same bits length. eg.
...
...
@@ -73,7 +80,7 @@ when the value is int type, all quantization types share same bits length. eg.
.. code-block:: bash
{
quant_bits: 8, # weight or output quantization are all 8 bits
quant_bits: 8, # weight or output quantization are all 8 bits
}
The following example shows a more complete ``config_list``\ , it uses ``op_names`` (or ``op_types``\ ) to specify the target layers along with the quantization bits for those layers.
...
...
@@ -81,25 +88,26 @@ The following example shows a more complete ``config_list``\ , it uses ``op_name
.. code-block:: bash
config_list = [{
'quant_types': ['weight'],
'quant_bits': 8,
'op_names': ['conv1']
}, {
'quant_types': ['weight'],
'quant_bits': 4,
'quant_start_step': 0,
'op_names': ['conv2']
}, {
'quant_types': ['weight'],
'quant_bits': 3,
'op_names': ['fc1']
},
{
'quant_types': ['weight'],
'quant_bits': 2,
'op_names': ['fc2']
}
]
'quant_types': ['weight'],
'quant_bits': 8,
'op_names': ['conv1']
},
{
'quant_types': ['weight'],
'quant_bits': 4,
'quant_start_step': 0,
'op_names': ['conv2']
},
{
'quant_types': ['weight'],
'quant_bits': 3,
'op_names': ['fc1']
},
{
'quant_types': ['weight'],
'quant_bits': 2,
'op_names': ['fc2']
}]
In this example, 'op_names' is the name of layer and four layers will be quantized to different quant_bits.
