BuiltinTuner.md 26.6 KB
Newer Older
1
# Built-in Tuners for Hyperparameter Tuning
Yan Ni's avatar
Yan Ni committed
2

3
NNI provides state-of-the-art tuning algorithms as our built-in tuners and makes them easy to use. Below is the brief summary of NNI currently built-in tuners:
Yan Ni's avatar
Yan Ni committed
4

xuehui's avatar
xuehui committed
5
Note: Click the **Tuner's name** to get the Tuner's installation requirements, suggested scenario and using example. The link for a detailed description of the algorithm is at the end of the suggested scenario of each tuner. Here is an [article](../CommunitySharings/HpoComparision.md) about the comparison of different Tuners on several problems.
6

7
8
Currently we support the following algorithms:

Yan Ni's avatar
Yan Ni committed
9
10
|Tuner|Brief Introduction of Algorithm|
|---|---|
Yan Ni's avatar
Yan Ni committed
11
12
13
|[__TPE__](#TPE)|The Tree-structured Parzen Estimator (TPE) is a sequential model-based optimization (SMBO) approach. SMBO methods sequentially construct models to approximate the performance of hyperparameters based on historical measurements, and then subsequently choose new hyperparameters to test based on this model. [Reference Paper](https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf)|
|[__Random Search__](#Random)|In Random Search for Hyper-Parameter Optimization show that Random Search might be surprisingly simple and effective. We suggest that we could use Random Search as the baseline when we have no knowledge about the prior distribution of hyper-parameters. [Reference Paper](http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf)|
|[__Anneal__](#Anneal)|This simple annealing algorithm begins by sampling from the prior, but tends over time to sample from points closer and closer to the best ones observed. This algorithm is a simple variation on the random search that leverages smoothness in the response surface. The annealing rate is not adaptive.|
Chi Song's avatar
Chi Song committed
14
15
|[__Naïve Evolution__](#Evolution)|Naïve Evolution comes from Large-Scale Evolution of Image Classifiers. It randomly initializes a population-based on search space. For each generation, it chooses better ones and does some mutation (e.g., change a hyperparameter, add/remove one layer) on them to get the next generation. Naïve Evolution requires many trials to works, but it's very simple and easy to expand new features. [Reference paper](https://arxiv.org/pdf/1703.01041.pdf)|
|[__SMAC__](#SMAC)|SMAC is based on Sequential Model-Based Optimization (SMBO). It adapts the most prominent previously used model class (Gaussian stochastic process models) and introduces the model class of random forests to SMBO, in order to handle categorical parameters. The SMAC supported by NNI is a wrapper on the SMAC3 GitHub repo. Notice, SMAC need to be installed by `nnictl package` command. [Reference Paper,](https://www.cs.ubc.ca/~hutter/papers/10-TR-SMAC.pdf) [GitHub Repo](https://github.com/automl/SMAC3)|
Yan Ni's avatar
Yan Ni committed
16
|[__Batch tuner__](#Batch)|Batch tuner allows users to simply provide several configurations (i.e., choices of hyper-parameters) for their trial code. After finishing all the configurations, the experiment is done. Batch tuner only supports the type choice in search space spec.|
17
|[__Grid Search__](#GridSearch)|Grid Search performs an exhaustive searching through a manually specified subset of the hyperparameter space defined in the searchspace file. Note that the only acceptable types of search space are choice, quniform, randint. |
Shufan Huang's avatar
Shufan Huang committed
18
|[__Hyperband__](#Hyperband)|Hyperband tries to use the limited resource to explore as many configurations as possible, and finds out the promising ones to get the final result. The basic idea is generating many configurations and to run them for the small number of trial budget to find out promising one, then further training those promising ones to select several more promising one.[Reference Paper](https://arxiv.org/pdf/1603.06560.pdf)|
Yan Ni's avatar
Yan Ni committed
19
20
|[__Network Morphism__](#NetworkMorphism)|Network Morphism provides functions to automatically search for architecture of deep learning models. Every child network inherits the knowledge from its parent network and morphs into diverse types of networks, including changes of depth, width, and skip-connection. Next, it estimates the value of a child network using the historic architecture and metric pairs. Then it selects the most promising one to train. [Reference Paper](https://arxiv.org/abs/1806.10282)|
|[__Metis Tuner__](#MetisTuner)|Metis offers the following benefits when it comes to tuning parameters: While most tools only predict the optimal configuration, Metis gives you two outputs: (a) current prediction of optimal configuration, and (b) suggestion for the next trial. No more guesswork. While most tools assume training datasets do not have noisy data, Metis actually tells you if you need to re-sample a particular hyper-parameter. [Reference Paper](https://www.microsoft.com/en-us/research/publication/metis-robustly-tuning-tail-latencies-cloud-systems/)|
Chi Song's avatar
Chi Song committed
21
22
|[__BOHB__](#BOHB)|BOHB is a follow-up work of Hyperband. It targets the weakness of Hyperband that new configurations are generated randomly without leveraging finished trials. For the name BOHB, HB means Hyperband, BO means Bayesian Optimization. BOHB leverages finished trials by building multiple TPE models, a proportion of new configurations are generated through these models. [Reference Paper](https://arxiv.org/abs/1807.01774)|
|[__GP Tuner__](#GPTuner)|Gaussian Process Tuner is a sequential model-based optimization (SMBO) approach with Gaussian Process as the surrogate. [Reference Paper](https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf), [Github Repo](https://github.com/fmfn/BayesianOptimization)|
Chi Song's avatar
Chi Song committed
23
|[__PPO Tuner__](#PPOTuner)|PPO Tuner is a Reinforcement Learning tuner based on PPO algorithm. [Reference Paper](https://arxiv.org/abs/1707.06347)|
Yan Ni's avatar
Yan Ni committed
24

Chi Song's avatar
Chi Song committed
25
## Usage of Built-in Tuners
Yan Ni's avatar
Yan Ni committed
26

Chi Song's avatar
Chi Song committed
27
Use built-in tuner provided by NNI SDK requires to declare the  **builtinTunerName** and **classArgs** in `config.yml` file. In this part, we will introduce the detailed usage about the suggested scenarios, classArg requirements and example for each tuner.
Yan Ni's avatar
Yan Ni committed
28

Chi Song's avatar
Chi Song committed
29
Note: Please follow the format when you write your `config.yml` file. Some built-in tuner need to be installed by `nnictl package`, like SMAC.
Yan Ni's avatar
Yan Ni committed
30
31
32
33
34

<a name="TPE"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `TPE`

Chi Song's avatar
Chi Song committed
35
> Built-in Tuner Name: **TPE**
Yan Ni's avatar
Yan Ni committed
36
37
38

**Suggested scenario**

39
40
TPE, as a black-box optimization, can be used in various scenarios and shows good performance in general. Especially when you have limited computation resource and can only try a small number of trials. From a large amount of experiments, we could found that TPE is far better than Random Search. [Detailed Description](./HyperoptTuner.md)

Yan Ni's avatar
Yan Ni committed
41

42
**Requirement of classArgs**
Yan Ni's avatar
Yan Ni committed
43

44
* **optimize_mode** (*maximize or minimize, optional, default = maximize*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
Yan Ni's avatar
Yan Ni committed
45

Yan Ni's avatar
Yan Ni committed
46
Note: We have optimized the parallelism of TPE for large-scale trial-concurrency. For the principle of optimization or turn-on optimization, please refer to [TPE document](./HyperoptTuner.md).
xuehui's avatar
xuehui committed
47

Yan Ni's avatar
Yan Ni committed
48
49
**Usage example:**

Yan Ni's avatar
Yan Ni committed
50
```yaml
Yan Ni's avatar
Yan Ni committed
51
52
53
54
55
56
57
58
59
60
61
62
63
# config.yml
tuner:
  builtinTunerName: TPE
  classArgs:
    optimize_mode: maximize
```

<br>

<a name="Random"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `Random Search`

Chi Song's avatar
Chi Song committed
64
> Built-in Tuner Name: **Random**
Yan Ni's avatar
Yan Ni committed
65
66
67

**Suggested scenario**

68
Random search is suggested when each trial does not take too long (e.g., each trial can be completed very soon, or early stopped by assessor quickly), and you have enough computation resource. Or you want to uniformly explore the search space. Random Search could be considered as baseline of search algorithm. [Detailed Description](./HyperoptTuner.md)
Yan Ni's avatar
Yan Ni committed
69

70
**Requirement of classArgs**
Yan Ni's avatar
Yan Ni committed
71

72
* **optimize_mode** (*maximize or minimize, optional, default = maximize*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
Yan Ni's avatar
Yan Ni committed
73
74
75

**Usage example**

Yan Ni's avatar
Yan Ni committed
76
```yaml
Yan Ni's avatar
Yan Ni committed
77
78
79
80
81
82
83
84
85
86
87
# config.yml
tuner:
  builtinTunerName: Random
```

<br>

<a name="Anneal"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `Anneal`

Chi Song's avatar
Chi Song committed
88
> Built-in Tuner Name: **Anneal**
Yan Ni's avatar
Yan Ni committed
89
90
91

**Suggested scenario**

92
93
Anneal is suggested when each trial does not take too long, and you have enough computation resource(almost same with Random Search). Or the variables in search space could be sample from some prior distribution. [Detailed Description](./HyperoptTuner.md)

Yan Ni's avatar
Yan Ni committed
94

95
**Requirement of classArgs**
Yan Ni's avatar
Yan Ni committed
96

97
* **optimize_mode** (*maximize or minimize, optional, default = maximize*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
Yan Ni's avatar
Yan Ni committed
98
99
100

**Usage example**

Yan Ni's avatar
Yan Ni committed
101
```yaml
Yan Ni's avatar
Yan Ni committed
102
103
104
105
106
107
108
109
110
111
112
# config.yml
tuner:
  builtinTunerName: Anneal
  classArgs:
    optimize_mode: maximize
```

<br>

<a name="Evolution"></a>

Chi Song's avatar
Chi Song committed
113
![](https://placehold.it/15/1589F0/000000?text=+) `Naïve Evolution`
Yan Ni's avatar
Yan Ni committed
114

Chi Song's avatar
Chi Song committed
115
> Built-in Tuner Name: **Evolution**
Yan Ni's avatar
Yan Ni committed
116
117
118

**Suggested scenario**

119
120
Its requirement of computation resource is relatively high. Specifically, it requires large initial population to avoid falling into local optimum. If your trial is short or leverages assessor, this tuner is a good choice. And, it is more suggested when your trial code supports weight transfer, that is, the trial could inherit the converged weights from its parent(s). This can greatly speed up the training progress. [Detailed Description](./EvolutionTuner.md)

121
**Requirement of classArgs**
xuehui's avatar
xuehui committed
122
123
124

* **optimize_mode** (*maximize or minimize, optional, default = maximize*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.

125
* **population_size** (*int value (should > 0), optional, default = 20*) - the initial size of the population (trial num) in evolution tuner. Suggests `population_size` be much larger than `concurrency`, so users can get the most out of the algorithm (and at least `concurrency`, or the tuner will fail on their first generation of parameters).
Yan Ni's avatar
Yan Ni committed
126
127
128

**Usage example**

Yan Ni's avatar
Yan Ni committed
129
```yaml
Yan Ni's avatar
Yan Ni committed
130
131
132
133
134
# config.yml
tuner:
  builtinTunerName: Evolution
  classArgs:
    optimize_mode: maximize
xuehui's avatar
xuehui committed
135
    population_size: 100
Yan Ni's avatar
Yan Ni committed
136
137
138
139
140
141
142
143
```

<br>

<a name="SMAC"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `SMAC`

Chi Song's avatar
Chi Song committed
144
> Built-in Tuner Name: **SMAC**
Yan Ni's avatar
Yan Ni committed
145

146
**Please note that SMAC doesn't support running on Windows currently. The specific reason can be referred to this [GitHub issue](https://github.com/automl/SMAC3/issues/483).**
147

148
149
**Installation**

150
SMAC need to be installed by following command before first use. As a reminder, `swig` is required for SMAC: for Ubuntu `swig` can be installed with `apt`.
151
152
153
154
155

```bash
nnictl package install --name=SMAC
```

Yan Ni's avatar
Yan Ni committed
156
157
**Suggested scenario**

158
Similar to TPE, SMAC is also a black-box tuner which can be tried in various scenarios, and is suggested when computation resource is limited. It is optimized for discrete hyperparameters, thus, suggested when most of your hyperparameters are discrete. [Detailed Description](./SmacTuner.md)
Yan Ni's avatar
Yan Ni committed
159

160
**Requirement of classArgs**
Yan Ni's avatar
Yan Ni committed
161

162
* **optimize_mode** (*maximize or minimize, optional, default = maximize*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
163
* **config_dedup** (*True or False, optional, default = False*) - If True, the tuner will not generate a configuration that has been already generated. If False, a configuration may be generated twice, but it is rare for relatively large search space.
Yan Ni's avatar
Yan Ni committed
164
165
166

**Usage example**

Yan Ni's avatar
Yan Ni committed
167
```yaml
Yan Ni's avatar
Yan Ni committed
168
169
170
171
172
173
174
175
176
177
178
179
180
# config.yml
tuner:
  builtinTunerName: SMAC
  classArgs:
    optimize_mode: maximize
```

<br>

<a name="Batch"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `Batch Tuner`

Chi Song's avatar
Chi Song committed
181
> Built-in Tuner Name: BatchTuner
Yan Ni's avatar
Yan Ni committed
182
183
184

**Suggested scenario**

185
If the configurations you want to try have been decided, you can list them in searchspace file (using `choice`) and run them using batch tuner. [Detailed Description](./BatchTuner.md)
Yan Ni's avatar
Yan Ni committed
186
187
188

**Usage example**

Yan Ni's avatar
Yan Ni committed
189
```yaml
Yan Ni's avatar
Yan Ni committed
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
# config.yml
tuner:
  builtinTunerName: BatchTuner
```

<br>

Note that the search space that BatchTuner supported like:

```json
{
    "combine_params":
    {
        "_type" : "choice",
        "_value" : [{"optimizer": "Adam", "learning_rate": 0.00001},
                    {"optimizer": "Adam", "learning_rate": 0.0001},
                    {"optimizer": "Adam", "learning_rate": 0.001},
207
208
                    {"optimizer": "SGD", "learning_rate": 0.01},
                    {"optimizer": "SGD", "learning_rate": 0.005},
Yan Ni's avatar
Yan Ni committed
209
210
211
212
213
214
215
216
217
218
219
                    {"optimizer": "SGD", "learning_rate": 0.0002}]
    }
}
```

The search space file including the high-level key `combine_params`. The type of params in search space must be `choice` and the `values` including all the combined-params value.

<a name="GridSearch"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `Grid Search`

Chi Song's avatar
Chi Song committed
220
> Built-in Tuner Name: **Grid Search**
Yan Ni's avatar
Yan Ni committed
221
222
223

**Suggested scenario**

224
Note that the only acceptable types of search space are `choice`, `quniform`, `randint`. 
Yan Ni's avatar
Yan Ni committed
225

226
It is suggested when search space is small, it is feasible to exhaustively sweeping the whole search space. [Detailed Description](./GridsearchTuner.md)
Yan Ni's avatar
Yan Ni committed
227
228
229

**Usage example**

Yan Ni's avatar
Yan Ni committed
230
```yaml
Yan Ni's avatar
Yan Ni committed
231
232
233
234
235
236
237
238
239
240
241
# config.yml
tuner:
  builtinTunerName: GridSearch
```

<br>

<a name="Hyperband"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `Hyperband`

Chi Song's avatar
Chi Song committed
242
> Built-in Advisor Name: **Hyperband**
Yan Ni's avatar
Yan Ni committed
243
244
245

**Suggested scenario**

246
It is suggested when you have limited computation resource but have relatively large search space. It performs well in the scenario that intermediate result (e.g., accuracy) can reflect good or bad of final result (e.g., accuracy) to some extent. [Detailed Description](./HyperbandAdvisor.md)
Yan Ni's avatar
Yan Ni committed
247

248
**Requirement of classArgs**
Yan Ni's avatar
Yan Ni committed
249

250
* **optimize_mode** (*maximize or minimize, optional, default = maximize*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
Shufan Huang's avatar
Shufan Huang committed
251
* **R** (*int, optional, default = 60*) - the maximum budget given to a trial (could be the number of mini-batches or epochs) can be allocated to a trial. Each trial should use TRIAL_BUDGET to control how long it runs.
Yan Ni's avatar
Yan Ni committed
252
253
254
255
* **eta** (*int, optional, default = 3*) - `(eta-1)/eta` is the proportion of discarded trials

**Usage example**

Yan Ni's avatar
Yan Ni committed
256
```yaml
Yan Ni's avatar
Yan Ni committed
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
# config.yml
advisor:
  builtinAdvisorName: Hyperband
  classArgs:
    optimize_mode: maximize
    R: 60
    eta: 3
```

<br>

<a name="NetworkMorphism"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `Network Morphism`

Chi Song's avatar
Chi Song committed
272
> Built-in Tuner Name: **NetworkMorphism**
Yan Ni's avatar
Yan Ni committed
273
274
275

**Installation**

276
NetworkMorphism requires [PyTorch](https://pytorch.org/get-started/locally) and [Keras](https://keras.io/#installation), so users should install them first. The corresponding requirements file is [here](https://github.com/microsoft/nni/blob/master/examples/trials/network_morphism/requirements.txt).
Yan Ni's avatar
Yan Ni committed
277
278
279

**Suggested scenario**

280
It is suggested that you want to apply deep learning methods to your task (your own dataset) but you have no idea of how to choose or design a network. You modify the [example](https://github.com/Microsoft/nni/tree/master/examples/trials/network_morphism/cifar10/cifar10_keras.py) to fit your own dataset and your own data augmentation method. Also you can change the batch size, learning rate or optimizer. It is feasible for different tasks to find a good network architecture. Now this tuner only supports the computer vision domain. [Detailed Description](./NetworkmorphismTuner.md)
Yan Ni's avatar
Yan Ni committed
281

282
**Requirement of classArgs**
Yan Ni's avatar
Yan Ni committed
283

284
* **optimize_mode** (*maximize or minimize, optional, default = maximize*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
Yan Ni's avatar
Yan Ni committed
285
286
287
288
289
290
291
* **task** (*('cv'), optional, default = 'cv'*) - The domain of experiment, for now, this tuner only supports the computer vision(cv) domain.
* **input_width** (*int, optional, default = 32*) - input image width
* **input_channel** (*int, optional, default = 3*) - input image channel
* **n_output_node** (*int, optional, default = 10*) - number of classes

**Usage example**

Yan Ni's avatar
Yan Ni committed
292
```yaml
Yan Ni's avatar
Yan Ni committed
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
# config.yml
tuner:
  builtinTunerName: NetworkMorphism
    classArgs:
      optimize_mode: maximize
      task: cv
      input_width: 32
      input_channel: 3
      n_output_node: 10
```

<br>

<a name="MetisTuner"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `Metis Tuner`

Chi Song's avatar
Chi Song committed
310
> Built-in Tuner Name: **MetisTuner**
Yan Ni's avatar
Yan Ni committed
311

312
Note that the only acceptable types of search space are `quniform`, `uniform` and `randint` and numerical `choice`. Only numerical values are supported since the values will be used to evaluate the 'distance' between different points.
Chi Song's avatar
Chi Song committed
313

Yan Ni's avatar
Yan Ni committed
314
315
**Suggested scenario**

Chi Song's avatar
Chi Song committed
316
Similar to TPE and SMAC, Metis is a black-box tuner. If your system takes a long time to finish each trial, Metis is more favorable than other approaches such as random search. Furthermore, Metis provides guidance on the subsequent trial. Here is an [example](https://github.com/Microsoft/nni/tree/master/examples/trials/auto-gbdt/search_space_metis.json) about the use of Metis. User only need to send the final result like `accuracy` to tuner, by calling the NNI SDK. [Detailed Description](./MetisTuner.md)
Yan Ni's avatar
Yan Ni committed
317

318
**Requirement of classArgs**
Yan Ni's avatar
Yan Ni committed
319

320
* **optimize_mode** (*'maximize' or 'minimize', optional, default = 'maximize'*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
Yan Ni's avatar
Yan Ni committed
321
322
323

**Usage example**

Yan Ni's avatar
Yan Ni committed
324
```yaml
Yan Ni's avatar
Yan Ni committed
325
326
327
328
329
330
# config.yml
tuner:
  builtinTunerName: MetisTuner
  classArgs:
    optimize_mode: maximize
```
Shufan Huang's avatar
Shufan Huang committed
331
332
333
334
335
336
337

<br>

<a name="BOHB"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `BOHB Advisor`

Chi Song's avatar
Chi Song committed
338
> Built-in Tuner Name: **BOHB**
Shufan Huang's avatar
Shufan Huang committed
339
340
341
342
343
344
345
346
347
348
349

**Installation**

BOHB advisor requires [ConfigSpace](https://github.com/automl/ConfigSpace) package, ConfigSpace need to be installed by following command before first use.

```bash
nnictl package install --name=BOHB
```

**Suggested scenario**

Chi Song's avatar
Chi Song committed
350
Similar to Hyperband, it is suggested when you have limited computation resource but have relatively large search space. It performs well in the scenario that intermediate result (e.g., accuracy) can reflect good or bad of final result (e.g., accuracy) to some extent. In this case, it may converges to a better configuration due to Bayesian optimization usage. [Detailed Description](./BohbAdvisor.md)
Shufan Huang's avatar
Shufan Huang committed
351

352
**Requirement of classArgs**
Shufan Huang's avatar
Shufan Huang committed
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368

* **optimize_mode** (*maximize or minimize, optional, default = maximize*) - If 'maximize', tuners will target to maximize metrics. If 'minimize', tuner will target to minimize metrics.
* **min_budget** (*int, optional, default = 1*) - The smallest budget assign to a trial job, (budget could be the number of mini-batches or epochs). Needs to be positive.
* **max_budget** (*int, optional, default = 3*) - The largest budget assign to a trial job, (budget could be the number of mini-batches or epochs). Needs to be larger than min_budget.
* **eta** (*int, optional, default = 3*) - In each iteration, a complete run of sequential halving is executed. In it, after evaluating each configuration on the same subset size, only a fraction of 1/eta of them 'advances' to the next round. Must be greater or equal to 2.
* **min_points_in_model**(*int, optional, default = None*): number of observations to start building a KDE. Default 'None' means dim+1, when the number of completed trial in this budget is equal or larger than `max{dim+1, min_points_in_model}`, BOHB will start to build a KDE model of this budget, then use KDE model to guide the configuration selection. Need to be positive.(dim means the number of hyperparameters in search space)
* **top_n_percent**(*int, optional, default = 15*): percentage (between 1 and 99, default 15) of the observations that are considered good. Good points and bad points are used for building KDE models. For example, if you have 100 observed trials and top_n_percent is 15, then top 15 point will used for building good point models "l(x)", the remaining 85 point will used for building bad point models "g(x)".
* **num_samples**(*int, optional, default = 64*): number of samples to optimize EI (default 64). In this case, we will sample "num_samples"(default = 64) points, and compare the result of l(x)/g(x), then return one with the maximum l(x)/g(x) value as the next configuration if the optimize_mode is maximize. Otherwise, we return the smallest one.
* **random_fraction**(*float, optional, default = 0.33*): fraction of purely random configurations that are sampled from the prior without the model.
* **bandwidth_factor**(*float, optional, default = 3.0*): to encourage diversity, the points proposed to optimize EI, are sampled from a 'widened' KDE where the bandwidth is multiplied by this factor. Suggest to use default value if you are not familiar with KDE.
* **min_bandwidth**(*float, optional, default = 0.001*): to keep diversity, even when all (good) samples have the same value for one of the parameters, a minimum bandwidth (default: 1e-3) is used instead of zero. Suggest to use default value if you are not familiar with KDE.

*Please note that currently float type only support decimal representation, you have to use 0.333 instead of 1/3 and 0.001 instead of 1e-3.*

**Usage example**

369
```yaml
Shufan Huang's avatar
Shufan Huang committed
370
371
372
373
374
375
376
advisor:
  builtinAdvisorName: BOHB
  classArgs:
    optimize_mode: maximize
    min_budget: 1
    max_budget: 27
    eta: 3
377
```
Guoxin's avatar
Guoxin committed
378
379
380
381
382

<a name="GPTuner"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `GP Tuner`

Chi Song's avatar
Chi Song committed
383
> Built-in Tuner Name: **GPTuner**
Guoxin's avatar
Guoxin committed
384

385
Note that the only acceptable types of search space are `randint`, `uniform`, `quniform`,  `loguniform`, `qloguniform`, and numerical `choice`. Only numerical values are supported since the values will be used to evaluate the 'distance' between different points.
Guoxin's avatar
Guoxin committed
386
387
388

**Suggested scenario**

Chi Song's avatar
Chi Song committed
389
As a strategy in Sequential Model-based Global Optimization(SMBO) algorithm, GP Tuner uses a proxy optimization problem (finding the maximum of the acquisition function) that, albeit still a hard problem, is cheaper (in the computational sense) and common tools can be employed. Therefore GP Tuner is most adequate for situations where the function to be optimized is a very expensive endeavor. GP can be used when the computation resource is limited. While GP Tuner has a computational cost that grows at *O(N^3)* due to the requirement of inverting the Gram matrix, so it's not suitable when lots of trials are needed. [Detailed Description](./GPTuner.md)
Guoxin's avatar
Guoxin committed
390

391
**Requirement of classArgs**
Guoxin's avatar
Guoxin committed
392
393
394
395
396
397
398
399
400

* **optimize_mode** (*'maximize' or 'minimize', optional, default = 'maximize'*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
* **utility** (*'ei', 'ucb' or 'poi', optional, default = 'ei'*) - The kind of utility function(acquisition function). 'ei', 'ucb' and 'poi' corresponds to 'Expected Improvement', 'Upper Confidence Bound' and 'Probability of Improvement' respectively. 
* **kappa** (*float, optional, default = 5*) - Used by utility function 'ucb'. The bigger `kappa` is, the more the tuner will be exploratory.
* **xi** (*float, optional, default = 0*) - Used by utility function 'ei' and 'poi'. The bigger `xi` is, the more the tuner will be exploratory.
* **nu** (*float, optional, default = 2.5*) - Used to specify Matern kernel. The smaller nu, the less smooth the approximated function is.
* **alpha** (*float, optional, default = 1e-6*) - Used to specify Gaussian Process Regressor. Larger values correspond to increased noise level in the observations.
* **cold_start_num** (*int, optional, default = 10*) - Number of random exploration to perform before Gaussian Process. Random exploration can help by diversifying the exploration space.
* **selection_num_warm_up** (*int, optional, default = 1e5*) - Number of random points to evaluate for getting the point which maximizes the acquisition function.
Chi Song's avatar
Chi Song committed
401
* **selection_num_starting_points** (*int, optional, default = 250*) - Number of times to run L-BFGS-B from a random starting point after the warmup.
Guoxin's avatar
Guoxin committed
402
403
404
405
406
407
408
409
410

**Usage example**

```yaml
# config.yml
tuner:
  builtinTunerName: GPTuner
  classArgs:
    optimize_mode: maximize
Guoxin's avatar
Guoxin committed
411
412
413
    utility: 'ei'
    kappa: 5.0
    xi: 0.0
Guoxin's avatar
Guoxin committed
414
415
416
417
418
419
    nu: 2.5
    alpha: 1e-6
    cold_start_num: 10
    selection_num_warm_up: 100000
    selection_num_starting_points: 250
```
420
421
422
423
424
425
426
427
428
429
430

<a name="PPOTuner"></a>

![](https://placehold.it/15/1589F0/000000?text=+) `PPO Tuner`

> Built-in Tuner Name: **PPOTuner**

Note that the only acceptable type of search space is `mutable_layer`. `optional_input_size` can only be 0, 1, or [0, 1].

**Suggested scenario**

Chi Song's avatar
Chi Song committed
431
PPOTuner is a Reinforcement Learning tuner based on PPO algorithm. When you are using NNI NAS interface in your trial code to do neural architecture search, PPOTuner can be used. In general, Reinforcement Learning algorithm need more computing resource, though PPO algorithm is more efficient than others relatively. So it's recommended to use this tuner when there are large amount of computing resource. You could try it on very simple task, such as the [mnist-nas](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nas) example. [See details](./PPOTuner.md)
432
433
434
435

**Requirement of classArgs**

* **optimize_mode** (*'maximize' or 'minimize'*) - If 'maximize', the tuner will target to maximize metrics. If 'minimize', the tuner will target to minimize metrics.
Chi Song's avatar
Chi Song committed
436
* **trials_per_update** (*int, optional, default = 20*) - The number of trials to be used for one update. It must be divisible by minibatch_size. `trials_per_update` is recommended to be an exact multiple of `trialConcurrency` for better concurrency of trials.
437
* **epochs_per_update** (*int, optional, default = 4*) - The number of epochs for one update.
Chi Song's avatar
Chi Song committed
438
* **minibatch_size** (*int, optional, default = 4*) - Mini-batch size (i.e., number of trials for a mini-batch) for the update. Note that, trials_per_update must be divisible by minibatch_size.
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
* **ent_coef** (*float, optional, default = 0.0*) - Policy entropy coefficient in the optimization objective.
* **lr** (*float, optional, default = 3e-4*) - Learning rate of the model (lstm network), constant.
* **vf_coef** (*float, optional, default = 0.5*) - Value function loss coefficient in the optimization objective.
* **max_grad_norm** (*float, optional, default = 0.5*) - Gradient norm clipping coefficient.
* **gamma** (*float, optional, default = 0.99*) - Discounting factor.
* **lam** (*float, optional, default = 0.95*) - Advantage estimation discounting factor (lambda in the paper).
* **cliprange** (*float, optional, default = 0.2*) - Cliprange in the PPO algorithm, constant.

**Usage example**

```yaml
# config.yml
tuner:
  builtinTunerName: PPOTuner
  classArgs:
    optimize_mode: maximize
Chi Song's avatar
Chi Song committed
455
```
456
457
458
459
460
461
## **Reference and Feedback**
* To [report a bug](https://github.com/microsoft/nni/issues/new?template=bug-report.md) for this feature in GitHub;
* To [file a feature or improvement request](https://github.com/microsoft/nni/issues/new?template=enhancement.md) for this feature in GitHub;
* To know more about [Feature Engineering with NNI](https://github.com/microsoft/nni/blob/master/docs/en_US/FeatureEngineering/Overview.md);
* To know more about [NAS with NNI](https://github.com/microsoft/nni/blob/master/docs/en_US/NAS/Overview.md);
* To know more about [Model Compression with NNI](https://github.com/microsoft/nni/blob/master/docs/en_US/Compressor/Overview.md);