README.md 16.7 KB
Newer Older
1
<p align="center">
2
<img src="docs/img/nni_logo.png" width="300"/>
3
4
5
</p>

-----------
6

7
[![MIT licensed](https://img.shields.io/badge/license-MIT-brightgreen.svg)](LICENSE)
8
[![Build Status](https://msrasrg.visualstudio.com/NNIOpenSource/_apis/build/status/Microsoft.nni)](https://msrasrg.visualstudio.com/NNIOpenSource/_build/latest?definitionId=6)
Gems Guo's avatar
Gems Guo committed
9
10
11
[![Issues](https://img.shields.io/github/issues-raw/Microsoft/nni.svg)](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen)
[![Bugs](https://img.shields.io/github/issues/Microsoft/nni/bug.svg)](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3Abug)
[![Pull Requests](https://img.shields.io/github/issues-pr-raw/Microsoft/nni.svg)](https://github.com/Microsoft/nni/pulls?q=is%3Apr+is%3Aopen)
The Gitter Badger's avatar
The Gitter Badger committed
12
[![Version](https://img.shields.io/github/release/Microsoft/nni.svg)](https://github.com/Microsoft/nni/releases) [![Join the chat at https://gitter.im/Microsoft/nni](https://badges.gitter.im/Microsoft/nni.svg)](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
Yan Ni's avatar
Yan Ni committed
13
[![Documentation Status](https://readthedocs.org/projects/nni/badge/?version=latest)](https://nni.readthedocs.io/en/latest/?badge=latest)
Microsoft Open Source's avatar
Microsoft Open Source committed
14

15
[简体中文](README_zh_CN.md)
Chi Song's avatar
Chi Song committed
16

17
NNI (Neural Network Intelligence) is a toolkit to help users run automated machine learning (AutoML) experiments.
Irene Tenison's avatar
Irene Tenison committed
18
The tool dispatches and runs trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in different environments like local machine, remote servers and cloud.
19

Lijiao's avatar
Lijiao committed
20
### **NNI [v0.9](https://github.com/Microsoft/nni/releases) has been released! &nbsp;<a href="#nni-released-reminder"><img width="48" src="docs/img/release_icon.png"></a>**
QuanluZhang's avatar
QuanluZhang committed
21
<p align="center">
Lijiao's avatar
Lijiao committed
22
  <a href="#nni-has-been-released"><img src="docs/img/overview.svg" /></a>
QuanluZhang's avatar
QuanluZhang committed
23
</p>
24
<div>
QuanluZhang's avatar
QuanluZhang committed
25
26
<table>
  <tbody>
27
    <tr align="center" valign="bottom">
QuanluZhang's avatar
QuanluZhang committed
28
      <td>
29
        <b>Supported Frameworks</b>
30
        <img src="docs/img/bar.png"/>
QuanluZhang's avatar
QuanluZhang committed
31
32
      </td>
      <td>
33
        <b>Tuning Algorithms</b>
34
        <img src="docs/img/bar.png"/>
QuanluZhang's avatar
QuanluZhang committed
35
36
      </td>
      <td>
Gems's avatar
Gems committed
37
        <b>Training Services</b>
38
        <img src="docs/img/bar.png"/>
QuanluZhang's avatar
QuanluZhang committed
39
40
      </td>
    </tr>
41
    </tr>
QuanluZhang's avatar
QuanluZhang committed
42
43
    <tr valign="top">
      <td>
44
45
46
47
48
49
50
51
52
        <ul>
          <li>PyTorch</li>
          <li>TensorFlow</li>
          <li>Keras</li>
          <li>MXNet</li>
          <li>Caffe2</li>
          <li>CNTK (Python language)</li>
          <li>Chainer</li>
          <li>Theano</li>
QuanluZhang's avatar
QuanluZhang committed
53
54
        </ul>
      </td>
55
56
57
      <td align="left">
        <a href="docs/en_US/Tuner/BuiltinTuner.md">Tuner</a>
        <br />
QuanluZhang's avatar
QuanluZhang committed
58
        <ul>
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
          <b style="margin-left:-20px"><font size=4 color=#800000>General Tuner</font></b>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#Random"><font size=2.9>Random Search</font></a></li>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#Evolution"><font size=2.9>Naïve Evolution</font></a></li>      
          <b><font size=4 color=#800000 style="margin-left:-20px">Tuner for HPO</font></b>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#TPE"><font size=2.9>TPE</font></a></li>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#Anneal"><font size=2.9>Anneal</font></a></li>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#SMAC"><font size=2.9>SMAC</font></a></li>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#Batch"><font size=2.9>Batch</font></a></li>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#GridSearch"><font size=2.9>Grid Search</font></a></li>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#Hyperband"><font size=2.9>Hyperband</font></a></li>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#MetisTuner"><font size=2.9>Metis Tuner</font></a></li>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#BOHB"><font size=2.9>BOHB</font></a></li>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#GPTuner"><font size=2.9>GP Tuner</font></a></li>
          <b style="margin-left:-20px"><font size=4 color=#800000 style="margin-left:-20px">Tuner for NAS</font></b>
          <li><a href="docs/en_US/Tuner/BuiltinTuner.md#NetworkMorphism"><font size=2.9>Network Morphism</font></a></li>
          <li><a href="examples/tuners/enas_nni/README.md"><font size=2.9>ENAS</font></a></li>
Shufan Huang's avatar
Shufan Huang committed
75
        </ul>
76
          <a href="docs/en_US/Assessor/BuiltinAssessor.md">Assessor</a>
QuanluZhang's avatar
QuanluZhang committed
77
        <ul>
78
79
          <li><a href="docs/en_US/Assessor/BuiltinAssessor.md#Medianstop"><font size=2.9>Median Stop</font></a></li>
          <li><a href="docs/en_US/Assessor/BuiltinAssessor.md#Curvefitting"><font size=2.9>Curve Fitting</font></a></li>
QuanluZhang's avatar
QuanluZhang committed
80
81
82
83
        </ul>
      </td>
      <td>
      <ul>
84
85
86
87
88
89
90
        <li><a href="docs/en_US/TrainingService/LocalMode.md">Local Machine</a></li>
        <li><a href="docs/en_US/TrainingService/RemoteMachineMode.md">Remote Servers</a></li>
        <li><b>Kubernetes based services</b></li>
            <ul><li><a href="docs/en_US/TrainingService/PaiMode.md">OpenPAI</a></li>
            <li><a href="docs/en_US/TrainingService/KubeflowMode.md">Kubeflow</a></li>
            <li><a href="docs/en_US/TrainingService/FrameworkControllerMode.md">FrameworkController on K8S (AKS etc.)</a></li>
            </ul>
QuanluZhang's avatar
QuanluZhang committed
91
92
93
      </ul>
      </td>
    </tr>
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
    <tr align="center" valign="bottom">
      <td style="border-top:#FF0000 solid 0px;">
        <b>References</b>
        <img src="docs/img/bar.png"/>
      </td>
      <td style="border-top:#FF0000 solid 0px;">
        <b>References</b>
        <img src="docs/img/bar.png"/>
      </td>
      <td style="border-top:#FF0000 solid 0px;">
        <b>References</b>
        <img src="docs/img/bar.png"/>
      </td>
    </tr>
    <tr valign="top">
     <td style="border-top:#FF0000 solid 0px;">
      <ul>
        <li><a href="docs/en_US/sdk_reference.rst">Python API</a></li>
        <li><a href="docs/en_US/Tutorial/AnnotationSpec.md">NNI Annotation</a></li>
        <li><a href="docs/en_US/TrialExample/Trials.md#nni-python-annotation">Annotation tutorial</a></li>
      </ul>
      </td>
       <td style="border-top:#FF0000 solid 0px;">
      <ul>
        <li><a href="docs/en_US/tuners.rst">Try different tuners</a></li>
        <li><a href="docs/en_US/assessors.rst">Try different assessors</a></li>
        <li><a href="docs/en_US/Tuner/CustomizeTuner.md">Implement a customized tuner</a></li>
        <li><a href="docs/en_US/Tuner/CustomizeAdvisor.md">Implement a customized advisor</a></li>
        <li><a href="docs/en_US/Assessor/CustomizeAssessor.md">Implement a customized assessor </a></li>
        <li><a href="docs/en_US/CommunitySharings/HpoComparision.md">HPO Comparison</a></li>
        <li><a href="docs/en_US/CommunitySharings/NasComparision.md">NAS Comparison</a></li>
        <li><a href="docs/en_US/CommunitySharings/RecommendersSvd.md">Automatically tuning SVD on NNI</a></li>
      </ul>
      </td>
        <td style="border-top:#FF0000 solid 0px;">
      <ul>
        <li><a href="docs/en_US/TrainingService/HowToImplementTrainingService.md">Implement TrainingService in NNI</a></li>
        <li><a href="docs/en_US/TrainingService/LocalMode.md">Run an experiment on local</a></li>
        <li><a href="docs/en_US/TrainingService/KubeflowMode.md">Run an experiment on Kubeflow</a></li>
        <li><a href="docs/en_US/TrainingService/PaiMode.md">Run an experiment on OpenPAI?</a></li>
        <li><a href="docs/en_US/TrainingService/RemoteMachineMode.md">Run an experiment on multiple machines?</a></li>
      </ul>
      </td>     
QuanluZhang's avatar
QuanluZhang committed
137
138
  </tbody>
</table>
139
</div>
QuanluZhang's avatar
QuanluZhang committed
140

141
## **Who should consider using NNI**
Chi Song's avatar
Chi Song committed
142

Irene Tenison's avatar
Irene Tenison committed
143
144
145
146
* Those who want to try different AutoML algorithms in their training code (model) at their local machine.
* Those who want to run AutoML trial jobs in different environments to speed up search (e.g. remote servers and cloud).
* Researchers and data scientists who want to implement their own AutoML algorithms and compare it with other algorithms.
* ML Platform owners who want to support AutoML in their platform.
147

148
## Related Projects
Chi Song's avatar
Chi Song committed
149

150
Targeting at openness and advancing state-of-art technology, [Microsoft Research (MSR)](https://www.microsoft.com/en-us/research/group/systems-research-group-asia/) had also released few other open source projects.
Chi Song's avatar
Chi Song committed
151

152
153
154
* [OpenPAI](https://github.com/Microsoft/pai) : an open source platform that provides complete AI model training and resource management capabilities, it is easy to extend and supports on-premise, cloud and hybrid environments in various scale.
* [FrameworkController](https://github.com/Microsoft/frameworkcontroller) : an open source general-purpose Kubernetes Pod Controller that orchestrate all kinds of applications on Kubernetes by a single controller.
* [MMdnn](https://github.com/Microsoft/MMdnn) : A comprehensive, cross-framework solution to convert, visualize and diagnose deep neural network models. The "MM" in MMdnn stands for model management and "dnn" is an acronym for deep neural network.
Scarlett Li's avatar
Scarlett Li committed
155
156
* [SPTAG](https://github.com/Microsoft/SPTAG) : Space Partition Tree And Graph (SPTAG) is an open source library for large scale vector approximate nearest neighbor search scenario.

157
158
We encourage researchers and students leverage these projects to accelerate the AI development and research.

Scarlett Li's avatar
Scarlett Li committed
159
## **Install & Verify**
Chi Song's avatar
Chi Song committed
160

161
**Install through pip**
Chi Song's avatar
Chi Song committed
162

163
* We support Linux, MacOS and Windows(local, remote and pai mode) in current stage, Ubuntu 16.04 or higher, MacOS 10.14.1 along with Windows 10.1809 are tested and supported. Simply run the following `pip install` in an environment that has `python >= 3.5`.
Zejun Lin's avatar
Zejun Lin committed
164

165
Linux and MacOS
Chi Song's avatar
Chi Song committed
166

Zejun Lin's avatar
Zejun Lin committed
167
```bash
Chi Song's avatar
Chi Song committed
168
python3 -m pip install --upgrade nni
169
```
Chi Song's avatar
Chi Song committed
170

171
Windows
Chi Song's avatar
Chi Song committed
172

173
```bash
Chi Song's avatar
Chi Song committed
174
python -m pip install --upgrade nni
175
```
Chi Song's avatar
Chi Song committed
176

Zejun Lin's avatar
Zejun Lin committed
177
178
179
Note:

* `--user` can be added if you want to install NNI in your home directory, which does not require any special privileges.
180
* Currently NNI on Windows support local, remote and pai mode. Anaconda or Miniconda is highly recommended to install NNI on Windows.
181
* If there is any error like `Segmentation fault`, please refer to [FAQ](docs/en_US/Tutorial/FAQ.md)
Gems Guo's avatar
Gems Guo committed
182
183

**Install through source code**
Chi Song's avatar
Chi Song committed
184

185
* We support Linux (Ubuntu 16.04 or higher), MacOS (10.14.1) and Windows (10.1809) in our current stage.
186
187

Linux and MacOS
Chi Song's avatar
Chi Song committed
188

Gems Guo's avatar
Gems Guo committed
189
* Run the following commands in an environment that has `python >= 3.5`, `git` and `wget`.
Chi Song's avatar
Chi Song committed
190
191

```bash
Yan Ni's avatar
Yan Ni committed
192
    git clone -b v0.9 https://github.com/Microsoft/nni.git
Chi Song's avatar
Chi Song committed
193
194
    cd nni
    source install.sh
195
```
Chi Song's avatar
Chi Song committed
196

197
Windows
Chi Song's avatar
Chi Song committed
198
199
200

* Run the following commands in an environment that has `python >=3.5`, `git` and `PowerShell`

201
```bash
Yan Ni's avatar
Yan Ni committed
202
  git clone -b v0.9 https://github.com/Microsoft/nni.git
203
  cd nni
204
  powershell -ExecutionPolicy Bypass -file install.ps1
205
```
206

207
For the system requirements of NNI, please refer to [Install NNI](docs/en_US/Tutorial/Installation.md)
Chi Song's avatar
Chi Song committed
208

209
For NNI on Windows, please refer to [NNI on Windows](docs/en_US/Tutorial/NniOnWindows.md)
210

Chi Song's avatar
Chi Song committed
211
212
213
214
215
216
217
**Verify install**

The following example is an experiment built on TensorFlow. Make sure you have **TensorFlow installed** before running it.

* Download the examples via clone the source code.

```bash
Yan Ni's avatar
Yan Ni committed
218
    git clone -b v0.9 https://github.com/Microsoft/nni.git
Gems Guo's avatar
Gems Guo committed
219
```
Chi Song's avatar
Chi Song committed
220

221
Linux and MacOS
Chi Song's avatar
Chi Song committed
222
223
224

* Run the MNIST example.

225
```bash
Gems Guo's avatar
Gems Guo committed
226
    nnictl create --config nni/examples/trials/mnist/config.yml
227
```
Chi Song's avatar
Chi Song committed
228

229
Windows
Chi Song's avatar
Chi Song committed
230
231
232

* Run the MNIST example.

233
```bash
234
    nnictl create --config nni\examples\trials\mnist\config_windows.yml
235
```
Chi Song's avatar
Chi Song committed
236

237
* Wait for the message `INFO: Successfully started experiment!` in the command line. This message indicates that your experiment has been successfully started. You can explore the experiment using the `Web UI url`.
238

Chi Song's avatar
Chi Song committed
239
```text
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
INFO: Starting restful server...
INFO: Successfully started Restful server!
INFO: Setting local config...
INFO: Successfully set local config!
INFO: Starting experiment...
INFO: Successfully started experiment!
-----------------------------------------------------------------------
The experiment id is egchD4qy
The Web UI urls are: http://223.255.255.1:8080   http://127.0.0.1:8080
-----------------------------------------------------------------------

You can use these commands to get more information about the experiment
-----------------------------------------------------------------------
         commands                       description
1. nnictl experiment show        show the information of experiments
2. nnictl trial ls               list all of trial jobs
SparkSnail's avatar
SparkSnail committed
256
257
258
259
260
261
3. nnictl top                    monitor the status of running experiments
4. nnictl log stderr             show stderr log content
5. nnictl log stdout             show stdout log content
6. nnictl stop                   stop an experiment
7. nnictl trial kill             kill a trial job by id
8. nnictl --help                 get help information about nnictl
262
-----------------------------------------------------------------------
Scarlett Li's avatar
Scarlett Li committed
263
```
Scarlett Li's avatar
Scarlett Li committed
264

265
* Open the `Web UI url` in your browser, you can view detail information of the experiment and all the submitted trial jobs as shown below. [Here](docs/en_US/Tutorial/WebUI.md) are more Web UI pages.
266
267
268
269
270
271

<table style="border: none">
    <th><img src="./docs/img/webui_overview_page.png" alt="drawing" width="395"/></th>
    <th><img src="./docs/img/webui_trialdetail_page.png" alt="drawing" width="410"/></th>
</table>

Scarlett Li's avatar
Scarlett Li committed
272
## **Documentation**
273
274
Our primary documentation is at [here](https://nni.readthedocs.io/en/latest/Overview.html) and is generated from this repository.<br/>
Maybe you want to read:
Chi Song's avatar
Chi Song committed
275

Yan Ni's avatar
Yan Ni committed
276
* [NNI overview](docs/en_US/Overview.md)
277
278
279
280
281
* [Quick start](docs/en_US/Tutorial/QuickStart.md)
* [Contributing](docs/en_US/Tutorial/Contributing.md)
* [Examples](docs/en_US/examples.rst)
* [References](docs/en_US/reference.rst)
* [WebUI tutorial](docs/en_US/Tutorial/WebUI.md)
282

Scarlett Li's avatar
Scarlett Li committed
283
## **How to**
Chi Song's avatar
Chi Song committed
284

285
286
287
288
289
290
291
292
* [Install NNI](docs/en_US/Tutorial/Installation.md)
* [Use command line tool nnictl](docs/en_US/Tutorial/Nnictl.md)
* [Use NNIBoard](docs/en_US/Tutorial/WebUI.md)
* [How to define search space](docs/en_US/Tutorial/SearchSpaceSpec.md)
* [How to define a trial](docs/en_US/TrialExample/Trials.md)
* [How to choose tuner/search-algorithm](docs/en_US/Tuner/BuiltinTuner.md)
* [Config an experiment](docs/en_US/Tutorial/ExperimentConfig.md)
* [How to use annotation](docs/en_US/TrialExample/Trials.md#nni-python-annotation)
Chi Song's avatar
Chi Song committed
293

Scarlett Li's avatar
Scarlett Li committed
294
## **Tutorials**
Chi Song's avatar
Chi Song committed
295

296
* [Run an experiment on OpenPAI?](docs/en_US/PaiMode.md)
Yan Ni's avatar
Yan Ni committed
297
* [Run an experiment on Kubeflow?](docs/en_US/KubeflowMode.md)
298
299
* [Run an experiment on local (with multiple GPUs)?](docs/en_US/LocalMode.md)
* [Run an experiment on multiple machines?](docs/en_US/RemoteMachineMode.md)
Yan Ni's avatar
Yan Ni committed
300
301
* [Try different tuners](docs/en_US/tuners.rst)
* [Try different assessors](docs/en_US/assessors.rst)
302
* [Implement a customized tuner](docs/en_US/Tuner/CustomizeTuner.md)
303
* [Implement a customized assessor](docs/en_US/CustomizeAssessor.md)
Scarlett Li's avatar
Scarlett Li committed
304
* [Use Genetic Algorithm to find good model architectures for Reading Comprehension task](examples/trials/ga_squad/README.md)
Scarlett Li's avatar
Scarlett Li committed
305
306

## **Contribute**
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
This project welcomes contributions and there are many ways in which you can participate in the project, for example:
* Review [source code changes](https://github.com/microsoft/nni/pulls)
* Review the [documentation](https://github.com/microsoft/nni/tree/master/docs) and make pull requests for anything from typos to new content
* Find the issues tagged with ['good first issue'](https://github.com/Microsoft/nni/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) or ['help-wanted'](https://github.com/microsoft/nni/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22), these are simple and easy to start , we recommend new contributors to start with.

Before providing your hacks, there are a few simple guidelines that you need to follow:
* [How to debug](docs/en_US/Tutorial/HowToDebug.md)
* [Code Styles & Naming Conventions](docs/en_US/Tutorial/Contributing.md)
* How to Set up [NNI developer environment](docs/en_US/Tutorial/SetupNniDeveloperEnvironment.md)
* Review the [Contributing Instruction](docs/en_US/Tutorial/Contributing.md) and get familiar with the NNI Code Contribution Guideline

## **External Repositories**
Now we have some external usage examples run in NNI from our contributors. Thanks our lovely contributors. And welcome more and more people to join us!
* Run [ENAS](examples/tuners/enas_nni/README.md) in NNI
* Run [Neural Network Architecture Search](examples/trials/nas_cifar10/README.md) in NNI 

## **Feedback**
* Open [bug reports](https://github.com/microsoft/nni/issues/new/choose).<br/>
* Request a [new feature](https://github.com/microsoft/nni/issues/new/choose).
* Discuss on the NNI [Gitter](https://gitter.im/Microsoft/nni?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) in NNI
* Ask a question with NNI tags on [Stack Overflow](https://stackoverflow.com/questions/tagged/nni?sort=Newest&edited=true)or [file an issue](https://github.com/microsoft/nni/issues/new/choose)on GitHub.
* We are in construction of the instruction for [How to Debug](docs/en_US/Tutorial/HowToDebug.md), you are also welcome to contribute questions or suggestions on this area.
Chi Song's avatar
Chi Song committed
329

Microsoft Open Source's avatar
Microsoft Open Source committed
330

Chi Song's avatar
Chi Song committed
331
332
## **License**

333
The entire codebase is under [MIT license](LICENSE)