NNI (Neural Network Intelligence) is a toolkit to help users run automated machine learning (AutoML) experiments.
NNI (Neural Network Intelligence) is a toolkit to help users run automated machine learning (AutoML) experiments.
The tool dispatches and runs trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in different environments like local machine, remote servers and cloud.
The tool dispatches and runs trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in different environments like local machine, remote servers and cloud.
### **NNI [v0.5](https://github.com/Microsoft/nni/releases) has been released!**
<li><ahref="docs/KubeflowMode.md">FrameworkController on K8S (AKS etc.)</a></li>
</ul>
</td>
</tr>
</tbody>
</table>
## **Who should consider using NNI**
## **Who should consider using NNI**
* Those who want to try different AutoML algorithms in their training code (model) at their local machine.
* Those who want to try different AutoML algorithms in their training code (model) at their local machine.
...
@@ -35,12 +97,14 @@ We encourage researchers and students leverage these projects to accelerate the
...
@@ -35,12 +97,14 @@ We encourage researchers and students leverage these projects to accelerate the
**Install through pip**
**Install through pip**
* We support Linux and MacOS in current stage, Ubuntu 16.04 or higher, along with MacOS 10.14.1 are tested and supported. Simply run the following `pip install` in an environment that has `python >= 3.5`.
* We support Linux and MacOS in current stage, Ubuntu 16.04 or higher, along with MacOS 10.14.1 are tested and supported. Simply run the following `pip install` in an environment that has `python >= 3.5`.
```bash
```bash
python3 -m pip install--upgrade nni
python3 -m pip install--upgrade nni
```
```
* Note:
Note:
* If you are in docker container (as root), please remove `--user` from the installation command.
* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/FAQ.md)
*`--user` can be added if you want to install NNI in your home directory, which does not require any special privileges.
* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/FAQ.md)
**Install through source code**
**Install through source code**
* We support Linux (Ubuntu 16.04 or higher), MacOS (10.14.1) in our current stage.
* We support Linux (Ubuntu 16.04 or higher), MacOS (10.14.1) in our current stage.
Typically each trial job gets single set of configuration (e.g. hyper parameters) from tuner and do some kind of experiment, let's say train a model with that hyper parameter and reports its result to tuner. Sometimes you may want to train multiple models within one trial job to share information between models or saving system resource by creating less trial jobs, for example:
1. Train multiple models sequentially in one trial job, so that later models can leverage the weights or other information of prior models and may use different hyper parameters.
2. Train large amount of models on limited system resource, combine multiple models together to save system resource to create large amount of trial jobs.
3. Any other scenario that you would like to train multiple models with different hyper parameters in one trial job, be aware that if you allocate multiple GPUs to a trial job and you train multiple models concurrently within on trial job, you need to allocate GPU resource properly by your trial code.
In above cases, you can leverage NNI multi-phase experiment to train multiple models with different hyper parameters within each trial job.
Multi-phase experiments refer to experiments whose trial jobs request multiple hyper parameters from tuner and report multiple final results to NNI.
To use multi-phase experiment, please follow below steps:
1. Implement nni.multi_phase.MultiPhaseTuner. For example, this [ENAS tuner](https://github.com/countif/enas_nni/blob/master/nni/examples/tuners/enas/nni_controller_ptb.py) is a multi-phase Tuner which implements nni.multi_phase.MultiPhaseTuner. While implementing your MultiPhaseTuner, you may want to use the trial_job_id parameter of generate_parameters method to generate hyper parameters for each trial job.
2. Set ```multiPhase``` field to ```true```, and configure your tuner implemented in step 1 as customized tuner in configuration file, for example:
```yml
...
multiPhase:true
tuner:
codeDir:tuners/enas
classFileName:nni_controller_ptb.py
className:ENASTuner
classArgs:
say_hello:"hello"
...
```
3. Invoke nni.get_next_parameter() API for multiple times as needed in a trial, for example:
```python
foriinrange(5):
# get parameter from tuner
tuner_param=nni.get_next_parameter()
# consume the params
# ...
# report final result somewhere for the parameter retrieved above
