Release.md 25.4 KB
Newer Older
Yan Ni's avatar
Yan Ni committed
1
2
# ChangeLog

Yan Ni's avatar
Yan Ni committed
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
## Release 1.3 - 12/30/2019

### Major Features

#### Neural Architecture Search Algorithms Support
* [Single Path One Shot](https://github.com/microsoft/nni/tree/v1.3/examples/nas/spos/) algorithm and the example using it

#### Model Compression Algorithms Support
* [Knowledge Distillation](https://github.com/microsoft/nni/blob/v1.3/docs/en_US/TrialExample/KDExample.md) algorithm and the example using itExample
* Pruners
    * [L2Filter Pruner](https://github.com/microsoft/nni/blob/v1.3/docs/en_US/Compressor/Pruner.md#3-l2filter-pruner)
    * [ActivationAPoZRankFilterPruner](https://github.com/microsoft/nni/blob/v1.3/docs/en_US/Compressor/Pruner.md#1-activationapozrankfilterpruner)
    * [ActivationMeanRankFilterPruner](https://github.com/microsoft/nni/blob/v1.3/docs/en_US/Compressor/Pruner.md#2-activationmeanrankfilterpruner)
* [BNN Quantizer](https://github.com/microsoft/nni/blob/v1.3/docs/en_US/Compressor/Quantizer.md#bnn-quantizer)
#### Training Service
* NFS Support for PAI
    
    Instead of using HDFS as default storage, since OpenPAI v0.11, OpenPAI can have NFS or AzureBlob or other storage as default storage. In this release, NNI extended the support for this recent change made by OpenPAI, and could integrate with OpenPAI v0.11 or later version with various default storage.

* Kubeflow update adoption

    Adopted the Kubeflow 0.7's new supports for tf-operator.

### Engineering (code and build automation)
* Enforced [ESLint](https://eslint.org/) on static code analysis.

### Small changes & Bug Fixes
* correctly recognize builtin tuner and customized tuner
* logging in dispatcher base
* fix the bug where tuner/assessor's failure sometimes kills the experiment.
* Fix local system as remote machine [issue](https://github.com/microsoft/nni/issues/1852)
* de-duplicate trial configuration in smac tuner [ticket](https://github.com/microsoft/nni/issues/1364)

chicm-ms's avatar
chicm-ms committed
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
## Release 1.2 - 12/02/2019

### Major Features
* [Feature Engineering](https://github.com/microsoft/nni/blob/v1.2/docs/en_US/FeatureEngineering/Overview.md)
  - New feature engineering interface
  - Feature selection algorithms: [Gradient feature selector](https://github.com/microsoft/nni/blob/v1.2/docs/en_US/FeatureEngineering/GradientFeatureSelector.md) & [GBDT selector](https://github.com/microsoft/nni/blob/v1.2/docs/en_US/FeatureEngineering/GBDTSelector.md)
  - [Examples for feature engineering](https://github.com/microsoft/nni/tree/v1.2/examples/feature_engineering)
* Neural Architecture Search (NAS) on NNI
  - [New NAS interface](https://github.com/microsoft/nni/blob/v1.2/docs/en_US/NAS/NasInterface.md)
  - NAS algorithms: [ENAS](https://github.com/microsoft/nni/blob/v1.2/docs/en_US/NAS/Overview.md#enas), [DARTS](https://github.com/microsoft/nni/blob/v1.2/docs/en_US/NAS/Overview.md#darts), [P-DARTS](https://github.com/microsoft/nni/blob/v1.2/docs/en_US/NAS/Overview.md#p-darts) (in PyTorch)
  - NAS in classic mode (each trial runs independently)
* Model compression
  - [New model pruning algorithms](https://github.com/microsoft/nni/blob/v1.2/docs/en_US/Compressor/Overview.md): lottery ticket pruning approach, L1Filter pruner, Slim pruner, FPGM pruner
  - [New model quantization algorithms](https://github.com/microsoft/nni/blob/v1.2/docs/en_US/Compressor/Overview.md): QAT quantizer, DoReFa quantizer
  - Support the API for exporting compressed model.
* Training Service
  - Support OpenPAI token authentication
* Examples:
  - [An example to automatically tune rocksdb configuration with NNI](https://github.com/microsoft/nni/tree/v1.2/examples/trials/systems/rocksdb-fillrandom).
  - [A new MNIST trial example supports tensorflow 2.0](https://github.com/microsoft/nni/tree/v1.2/examples/trials/mnist-tfv2).
* Engineering Improvements
  - For remote training service,  trial jobs require no GPU are now scheduled with round-robin policy instead of random.
  - Pylint rules added to check pull requests, new pull requests need to comply with these [pylint rules](https://github.com/microsoft/nni/blob/v1.2/pylintrc).
* Web Portal & User Experience
  - Support user to add customized trial.
  - User can zoom out/in in detail graphs, except Hyper-parameter.
* Documentation
  - Improved NNI API documentation with more API docstring.

### Bug fix
  - Fix the table sort issue when failed trials haven't metrics. -Issue #1773
  - Maintain selected status(Maximal/Minimal) when the page switched. -PR#1710
  - Make hyper-parameters graph's default metric yAxis more accurate. -PR#1736
  - Fix GPU script permission issue. -Issue #1665

## Release 1.1 - 10/23/2019

### Major Features

* New tuner: [PPO Tuner](https://github.com/microsoft/nni/blob/v1.1/docs/en_US/Tuner/PPOTuner.md)
* [View stopped experiments](https://github.com/microsoft/nni/blob/v1.1/docs/en_US/Tutorial/Nnictl.md#view)
* Tuners can now use dedicated GPU resource (see `gpuIndices` in [tutorial](https://github.com/microsoft/nni/blob/v1.1/docs/en_US/Tutorial/ExperimentConfig.md) for details)
* Web UI improvements
  - Trials detail page can now list hyperparameters of each trial, as well as their start and end time (via "add column")
  - Viewing huge experiment is now less laggy
* More examples
  - [EfficientNet PyTorch example](https://github.com/ultmaster/EfficientNet-PyTorch)
  - [Cifar10 NAS example](https://github.com/microsoft/nni/blob/v1.1/examples/trials/nas_cifar10/README.md)
* [Model compression toolkit - Alpha release](https://github.com/microsoft/nni/blob/v1.1/docs/en_US/Compressor/Overview.md): We are glad to announce the alpha release for model compression toolkit on top of NNI, it's still in the experiment phase which might evolve based on usage feedback. We'd like to invite you to use, feedback and even contribute

### Fixed Bugs

* Multiphase job hangs when search space exhuasted (issue #1204)
* `nnictl` fails when log not available (issue #1548)

## Release 1.0 - 9/2/2019

### Major Features

* Tuners and Assessors
    - Support Auto-Feature generator & selection    -Issue#877  -PR #1387
         + Provide auto feature interface
         + Tuner based on beam search
Yan Ni's avatar
Yan Ni committed
99
         + [Add Pakdd example](https://github.com/microsoft/nni/tree/master/examples/trials/auto-feature-engineering)
chicm-ms's avatar
chicm-ms committed
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
    - Add a parallel algorithm to improve the performance of TPE with large concurrency.  -PR #1052
    - Support multiphase for hyperband    -PR #1257

* Training Service
     - Support private docker registry   -PR #755

 * Engineering Improvements
    - Python wrapper for rest api, support retrieve the values of the metrics in a programmatic way  PR #1318
    - New python API : get_experiment_id(), get_trial_id()  -PR #1353   -Issue #1331 & -Issue#1368
    - Optimized NAS Searchspace  -PR #1393
         + Unify NAS search space with _type -- "mutable_type"e
         + Update random search tuner
    - Set gpuNum as optional      -Issue #1365
    - Remove outputDir and dataDir configuration in PAI mode   -Issue #1342
    - When creating a trial in Kubeflow mode, codeDir will no longer be copied to logDir   -Issue #1224

* Web Portal & User Experience
    - Show the best metric curve during search progress in WebUI  -Issue #1218
    - Show the current number of parameters list in multiphase experiment   -Issue1210  -PR #1348
    - Add "Intermediate count" option in AddColumn.      -Issue #1210
    - Support search parameters value in WebUI     -Issue #1208
    - Enable automatic scaling of axes for metric value  in default metric graph   -Issue #1360
    - Add a detailed documentation link to the nnictl command in the command prompt    -Issue #1260
    - UX improvement for showing Error log   -Issue #1173

* Documentation
    - Update the docs structure  -Issue #1231
Yan Ni's avatar
Yan Ni committed
127
    - [Multi phase document improvement](AdvancedFeature/MultiPhase.md)   -Issue #1233  -PR #1242
chicm-ms's avatar
chicm-ms committed
128
         + Add configuration example
Yan Ni's avatar
Yan Ni committed
129
    - [WebUI description improvement](Tutorial/WebUI.md)  -PR #1419
chicm-ms's avatar
chicm-ms committed
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146


### Bug fix
* (Bug fix)Fix the broken links in 0.9 release  -Issue #1236
* (Bug fix)Script for auto-complete
* (Bug fix)Fix pipeline issue that it only check exit code of last command in a script.  -PR #1417
* (Bug fix)quniform fors tuners    -Issue #1377
* (Bug fix)'quniform' has different meaning beween GridSearch and other tuner.   -Issue #1335
* (Bug fix)"nnictl experiment list" give the status of a "RUNNING" experiment as "INITIALIZED" -PR #1388
* (Bug fix)SMAC cannot be installed if nni is installed in dev mode    -Issue #1376
* (Bug fix)The filter button of the intermediate result cannot be clicked   -Issue #1263
* (Bug fix)API "/api/v1/nni/trial-jobs/xxx" doesn't show a trial's all parameters in multiphase experiment    -Issue #1258
* (Bug fix)Succeeded trial doesn't have final result but webui show ×××(FINAL)  -Issue #1207
* (Bug fix)IT for nnictl stop -Issue #1298
* (Bug fix)fix security warning
* (Bug fix)Hyper-parameter page broken  -Issue #1332
* (Bug fix)Run flake8 tests to find Python syntax errors and undefined names -PR #1217
147

Yan Ni's avatar
Yan Ni committed
148
149
150
151
152
## Release 0.9 - 7/1/2019

### Major Features
* General NAS programming interface
    * Add `enas-mode`  and `oneshot-mode` for NAS interface: [PR #1201](https://github.com/microsoft/nni/pull/1201#issue-291094510)
153
* [Gaussian Process Tuner with Matern kernel](Tuner/GPTuner.md) 
Yan Ni's avatar
Yan Ni committed
154
155
156
157
158
159

* Multiphase experiment supports
    * Added new training service support for multiphase experiment: PAI mode supports multiphase experiment since v0.9.
    * Added multiphase capability for the following builtin tuners: 
        * TPE, Random Search, Anneal, Naïve Evolution, SMAC, Network Morphism, Metis Tuner.
    
160
    For details, please refer to [Write a tuner that leverages multi-phase](AdvancedFeature/MultiPhase.md)
Yan Ni's avatar
Yan Ni committed
161
162

* Web Portal
163
164
165
166
    * Enable trial comparation in Web Portal. For details, refer to [View trials status](Tutorial/WebUI.md)
    * Allow users to adjust rendering interval of Web Portal. For details, refer to [View Summary Page](Tutorial/WebUI.md)
    * show intermediate results more friendly. For details, refer to [View trials status](Tutorial/WebUI.md)
* [Commandline Interface](Tutorial/Nnictl.md)
Yan Ni's avatar
Yan Ni committed
167
168
169
170
171
172
173
174
175
176
    * `nnictl experiment delete`: delete one or all experiments, it includes log, result, environment information and cache. It uses to delete useless experiment result, or save disk space.
    * `nnictl platform clean`: It uses to clean up disk on a target platform. The provided YAML file includes the information of target platform, and it follows the same schema as the NNI configuration file.
### Bug fix and other changes
* Tuner Installation Improvements: add [sklearn](https://scikit-learn.org/stable/) to nni dependencies.
* (Bug Fix) Failed to connect to PAI http code - [Issue #1076](https://github.com/microsoft/nni/issues/1076)
* (Bug Fix) Validate file name for PAI platform - [Issue #1164](https://github.com/microsoft/nni/issues/1164)
* (Bug Fix) Update GMM evaluation in Metis Tuner
* (Bug Fix) Negative time number rendering in Web Portal - [Issue #1182](https://github.com/microsoft/nni/issues/1182), [Issue #1185](https://github.com/microsoft/nni/issues/1185)
* (Bug Fix) Hyper-parameter not shown correctly in WebUI when there is only one hyper parameter - [Issue #1192](https://github.com/microsoft/nni/issues/1192)

177
178
179
180
181
182
183
184
185
186
187
188
## Release 0.8 - 6/4/2019

### Major Features

* Support NNI on Windows for OpenPAI/Remote mode
  * NNI running on windows for remote mode
  * NNI running on windows for OpenPAI mode
* Advanced features for using GPU
  * Run multiple trial jobs on the same GPU for local and remote mode
  * Run trial jobs on the GPU running non-NNI jobs
* Kubeflow v1beta2 operator
  * Support Kubeflow TFJob/PyTorchJob v1beta2
189
* [General NAS programming interface](https://github.com/microsoft/nni/blob/v0.8/docs/en_US/GeneralNasInterfaces.md)
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
  * Provide NAS programming interface for users to easily express their neural architecture search space through NNI annotation
  * Provide a new command `nnictl trial codegen` for debugging the NAS code
  * Tutorial of NAS programming interface, example of NAS on MNIST, customized random tuner for NAS
* Support resume tuner/advisor's state for experiment resume
* For experiment resume, tuner/advisor will be resumed by replaying finished trial data
* Web Portal
  * Improve the design of copying trial's parameters
  * Support 'randint' type in hyper-parameter graph
  * Use should ComponentUpdate to avoid unnecessary render

### Bug fix and other changes

* Bug fix that `nnictl update` has inconsistent command styles
* Support import data for SMAC tuner
* Bug fix that experiment state transition from ERROR back to RUNNING
* Fix bug of table entries
* Nested search space refinement
* Refine 'randint' type and support lower bound
208
209
210
* [Comparison of different hyper-parameter tuning algorithm](CommunitySharings/HpoComparision.md)
* [Comparison of NAS algorithm](CommunitySharings/NasComparision.md)
* [NNI practice on Recommenders](CommunitySharings/RecommendersSvd.md)
Xiaohui Xue's avatar
Xiaohui Xue committed
211

chicm-ms's avatar
chicm-ms committed
212
213
214
## Release 0.7 - 4/29/2018

### Major Features
Chi Song's avatar
Chi Song committed
215

216
* [Support NNI on Windows](Tutorial/NniOnWindows.md)
Chi Song's avatar
Chi Song committed
217
  * NNI running on windows for local mode
218
* [New advisor: BOHB](Tuner/BohbAdvisor.md)
Chi Song's avatar
Chi Song committed
219
  * Support a new advisor BOHB, which is a robust and efficient hyperparameter tuning algorithm, combines the advantages of Bayesian optimization and Hyperband
220
* [Support import and export experiment data through nnictl](Tutorial/Nnictl.md#experiment)
Chi Song's avatar
Chi Song committed
221
222
  * Generate analysis results report after the experiment execution
  * Support import data to tuner and advisor for tuning
223
* [Designated gpu devices for NNI trial jobs](Tutorial/ExperimentConfig.md#localConfig)
Chi Song's avatar
Chi Song committed
224
  * Specify GPU devices for NNI trial jobs by gpuIndices configuration, if gpuIndices is set in experiment configuration file, only the specified GPU devices are used for NNI trial jobs.
chicm-ms's avatar
chicm-ms committed
225
* Web Portal enhancement
Chi Song's avatar
Chi Song committed
226
227
228
229
  * Decimal format of metrics other than default on the Web UI
  * Hints in WebUI about Multi-phase
  * Enable copy/paste for hyperparameters as python dict
  * Enable early stopped trials data for tuners.
chicm-ms's avatar
chicm-ms committed
230
* NNICTL provide better error message
Chi Song's avatar
Chi Song committed
231
  * nnictl provide more meaningful error message for YAML file format error
chicm-ms's avatar
chicm-ms committed
232
233

### Bug fix
Chi Song's avatar
Chi Song committed
234

chicm-ms's avatar
chicm-ms committed
235
* Unable to kill all python threads after nnictl stop in async dispatcher mode
Chi Song's avatar
Chi Song committed
236
* nnictl --version does not work with make dev-install
237
* All trail jobs status stays on 'waiting' for long time on OpenPAI platform
chicm-ms's avatar
chicm-ms committed
238
239

## Release 0.6 - 4/2/2019
Chi Song's avatar
Chi Song committed
240

chicm-ms's avatar
chicm-ms committed
241
### Major Features
Chi Song's avatar
Chi Song committed
242

243
* [Version checking](TrainingService/PaiMode.md)
Chi Song's avatar
Chi Song committed
244
  * check whether the version is consistent between nniManager and trialKeeper
245
* [Report final metrics for early stop job](https://github.com/microsoft/nni/issues/776)
Chi Song's avatar
Chi Song committed
246
  * If includeIntermediateResults is true, the last intermediate result of the trial that is early stopped by assessor is sent to tuner as final result. The default value of includeIntermediateResults is false.
247
* [Separate Tuner/Assessor](https://github.com/microsoft/nni/issues/841)
Chi Song's avatar
Chi Song committed
248
  * Adds two pipes to separate message receiving channels for tuner and assessor.
chicm-ms's avatar
chicm-ms committed
249
250
251
252
* Make log collection feature configurable
* Add intermediate result graph for all trials

### Bug fix
Chi Song's avatar
Chi Song committed
253

254
* [Add shmMB config key for OpenPAI](https://github.com/microsoft/nni/issues/842)
chicm-ms's avatar
chicm-ms committed
255
256
257
258
259
260
* Fix the bug that doesn't show any result if metrics is dict
* Fix the number calculation issue for float types in hyperband
* Fix a bug in the search space conversion in SMAC tuner
* Fix the WebUI issue when parsing experiment.json with illegal format
* Fix cold start issue in Metis Tuner

chicm-ms's avatar
chicm-ms committed
261
## Release 0.5.2 - 3/4/2019
Chi Song's avatar
Chi Song committed
262

chicm-ms's avatar
chicm-ms committed
263
### Improvements
Chi Song's avatar
Chi Song committed
264

chicm-ms's avatar
chicm-ms committed
265
266
267
268
* Curve fitting assessor performance improvement.

### Documentation
* Chinese version document: https://nni.readthedocs.io/zh/latest/
269
270
* Debuggability/serviceability document: https://nni.readthedocs.io/en/latest/Tutorial/HowToDebug.html
* Tuner assessor reference: https://nni.readthedocs.io/en/latest/sdk_reference.html
chicm-ms's avatar
chicm-ms committed
271
272
273
274
275
276

### Bug Fixes and Other Changes
* Fix a race condition bug that does not store trial job cancel status correctly.
* Fix search space parsing error when using SMAC tuner.
* Fix cifar10 example broken pipe issue.
* Add unit test cases for nnimanager and local training service.
277
278
* Add integration test azure pipelines for remote machine, OpenPAI and kubeflow training services.
* Support Pylon in OpenPAI webhdfs client.
chicm-ms's avatar
chicm-ms committed
279

280
281
## Release 0.5.1 - 1/31/2018
### Improvements
282
283
* Making [log directory](https://github.com/microsoft/nni/blob/v0.5.1/docs/ExperimentConfig.md) configurable
* Support [different levels of logs](https://github.com/microsoft/nni/blob/v0.5.1/docs/ExperimentConfig.md), making it easier for debugging
284
285
286
287
288
289

### Documentation
* Reorganized documentation & New Homepage Released: https://nni.readthedocs.io/en/latest/

### Bug Fixes and Other Changes
* Fix the bug of installation in python virtualenv, and refactor the installation logic
290
* Fix the bug of HDFS access failure on OpenPAI mode after OpenPAI is upgraded.
291
292
* Fix the bug that sometimes in-place flushed stdout makes experiment crash

Yan Ni's avatar
Yan Ni committed
293
## Release 0.5.0 - 01/14/2019
294

Yan Ni's avatar
Yan Ni committed
295
### Major Features
296

Yan Ni's avatar
Yan Ni committed
297
#### New tuner and assessor supports
Yan Ni's avatar
Yan Ni committed
298

299
* Support [Metis tuner](Tuner/MetisTuner.md) as a new NNI tuner. Metis algorithm has been proofed to be well performed for **online** hyper-parameter tuning.
300
* Support [ENAS customized tuner](https://github.com/countif/enas_nni), a tuner contributed by github community user, is an algorithm for neural network search, it could learn neural network architecture via reinforcement learning and serve a better performance than NAS.
301
* Support [Curve fitting assessor](Assessor/CurvefittingAssessor.md) for early stop policy using learning curve extrapolation.
302
* Advanced Support of [Weight Sharing](https://github.com/microsoft/nni/blob/v0.5/docs/AdvancedNAS.md): Enable weight sharing for NAS tuners, currently through NFS.
xuehui's avatar
xuehui committed
303

Yan Ni's avatar
Yan Ni committed
304
#### Training Service Enhancement
305

306
* [FrameworkController Training service](TrainingService/FrameworkControllerMode.md): Support run experiments using frameworkcontroller on kubernetes
307
308
309
  * FrameworkController is a Controller on kubernetes that is general enough to run (distributed) jobs with various machine learning frameworks, such as tensorflow, pytorch, MXNet.
  * NNI provides unified and simple specification for job definition.
  * MNIST example for how to use FrameworkController.
xuehui's avatar
xuehui committed
310

Yan Ni's avatar
Yan Ni committed
311
#### User Experience improvements
312

313
* A better trial logging support for NNI experiments in OpenPAI, Kubeflow and FrameworkController mode:
314
315
316
  * An improved logging architecture to send stdout/stderr of trials to NNI manager via Http post. NNI manager will store trial's stdout/stderr messages in local log file.
  * Show the link for trial log file on WebUI.
* Support to show final result's all key-value pairs.
xuehui's avatar
xuehui committed
317

Yan Ni's avatar
Yan Ni committed
318
## Release 0.4.1 - 12/14/2018
319

Yan Ni's avatar
Yan Ni committed
320
### Major Features
321

Yan Ni's avatar
Yan Ni committed
322
#### New tuner supports
323

324
* Support [network morphism](Tuner/NetworkmorphismTuner.md) as a new tuner
xuehui's avatar
xuehui committed
325

Yan Ni's avatar
Yan Ni committed
326
#### Training Service improvements
327

328
* Migrate [Kubeflow training service](TrainingService/KubeflowMode.md)'s dependency from kubectl CLI to [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/) client
329
330
331
* [Pytorch-operator](https://github.com/kubeflow/pytorch-operator) support for Kubeflow training service
* Improvement on local code files uploading to OpenPAI HDFS
* Fixed OpenPAI integration WebUI bug: WebUI doesn't show latest trial job status, which is caused by OpenPAI token expiration
xuehui's avatar
xuehui committed
332

Yan Ni's avatar
Yan Ni committed
333
#### NNICTL improvements
334
335

* Show version information both in nnictl and WebUI. You can run **nnictl -v** to show your current installed NNI version
xuehui's avatar
xuehui committed
336

Yan Ni's avatar
Yan Ni committed
337
#### WebUI improvements
338
339
340
341
342
343
344

* Enable modify concurrency number during experiment
* Add feedback link to NNI github 'create issue' page
* Enable customize top 10 trials regarding to metric numbers (largest or smallest)
* Enable download logs for dispatcher & nnimanager
* Enable automatic scaling of axes for metric number
* Update annotation to support displaying real choice in searchspace
xuehui's avatar
xuehui committed
345

Yan Ni's avatar
Yan Ni committed
346
### New examples
347

348
349
* [FashionMnist](https://github.com/microsoft/nni/tree/master/examples/trials/network_morphism), work together with network morphism tuner
* [Distributed MNIST example](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-distributed-pytorch) written in PyTorch
350

Yan Ni's avatar
Yan Ni committed
351
## Release 0.4 - 12/6/2018
352

Yan Ni's avatar
Yan Ni committed
353
### Major Features
354

355
* [Kubeflow Training service](TrainingService/KubeflowMode.md)
356
  * Support tf-operator
357
  * [Distributed trial example](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-distributed/dist_mnist.py) on Kubeflow
358
359
* [Grid search tuner](Tuner/GridsearchTuner.md)
* [Hyperband tuner](Tuner/HyperbandAdvisor.md)
360
361
362
363
364
365
366
367
368
369
* Support launch NNI experiment on MAC
* WebUI
  * UI support for hyperband tuner
  * Remove tensorboard button
  * Show experiment error message
  * Show line numbers in search space and trial profile
  * Support search a specific trial by trial number
  * Show trial's hdfsLogPath
  * Download experiment parameters

Yan Ni's avatar
Yan Ni committed
370
### Others
371
372

* Asynchronous dispatcher
373
374
* Docker file update, add pytorch library
* Refactor 'nnictl stop' process, send SIGTERM to nni manager process, rather than calling stop Rest API.
375
* OpenPAI training service bug fix
376
  * Support NNI Manager IP configuration(nniManagerIp) in OpenPAI cluster config file, to fix the issue that user’s machine has no eth0 device
377
  * File number in codeDir is capped to 1000 now, to avoid user mistakenly fill root dir for codeDir
378
  * Don’t print useless ‘metrics is empty’ log in OpenPAI job’s stdout. Only print useful message once new metrics are recorded, to reduce confusion when user checks OpenPAI trial’s output for debugging purpose
379
  * Add timestamp at the beginning of each log entry in trial keeper.
380

Yan Ni's avatar
Yan Ni committed
381
## Release 0.3.0 - 11/2/2018
382

Yan Ni's avatar
Yan Ni committed
383
### NNICTL new features and updates
384

385
386
* Support running multiple experiments simultaneously.

Chi Song's avatar
Chi Song committed
387
  Before v0.3, NNI only supports running single experiment once a time. After this release, users are able to run multiple experiments simultaneously. Each experiment will require a unique port, the 1st experiment will be set to the default port as previous versions. You can specify a unique port for the rest experiments as below:
388
389
390
391

  ```bash
  nnictl create --port 8081 --config <config file path>
  ```
chicm-ms's avatar
chicm-ms committed
392

393
* Support updating max trial number.
394
  use `nnictl update --help` to learn more. Or refer to [NNICTL Spec](Tutorial/Nnictl.md) for the fully usage of NNICTL.
chicm-ms's avatar
chicm-ms committed
395

Yan Ni's avatar
Yan Ni committed
396
### API new features and updates
397

398
* <span style="color:red">**breaking change**</span>: nn.get_parameters() is refactored to nni.get_next_parameter. All examples of prior releases can not run on v0.3, please clone nni repo to get new examples. If you had applied NNI to your own codes, please update the API accordingly.
chicm-ms's avatar
chicm-ms committed
399

400
* New API **nni.get_sequence_id()**.
401
402
403
  Each trial job is allocated a unique sequence number, which can be retrieved by nni.get_sequence_id() API.

  ```bash
404
  git clone -b v0.3 https://github.com/microsoft/nni.git
405
406
407
  ```

* **nni.report_final_result(result)** API supports more data types for result parameter.
408

409
410
411
412
  It can be of following types:
  * int
  * float
  * A python dict containing 'default' key, the value of 'default' key should be of type int or float. The dict can contain any other key value pairs.
chicm-ms's avatar
chicm-ms committed
413

Yan Ni's avatar
Yan Ni committed
414
### New tuner support
415

416
* **Batch Tuner** which iterates all parameter combination, can be used to submit batch trial jobs.
chicm-ms's avatar
chicm-ms committed
417

Yan Ni's avatar
Yan Ni committed
418
### New examples
419

420
* A NNI Docker image for public usage:
421
422
423
424
425

  ```bash
  docker pull msranni/nni:latest
  ```

426
427
* New trial example: [NNI Sklearn Example](https://github.com/microsoft/nni/tree/master/examples/trials/sklearn)
* New competition example: [Kaggle Competition TGS Salt Example](https://github.com/microsoft/nni/tree/master/examples/trials/kaggle-tgs-salt)
428

Yan Ni's avatar
Yan Ni committed
429
### Others
430

431
* UI refactoring, refer to [WebUI doc](Tutorial/WebUI.md) for how to work with the new UI.
432
* Continuous Integration: NNI had switched to Azure pipelines
chicm-ms's avatar
chicm-ms committed
433

Yan Ni's avatar
Yan Ni committed
434
## Release 0.2.0 - 9/29/2018
435

Yan Ni's avatar
Yan Ni committed
436
### Major Features
437

438
* Support [OpenPAI](https://github.com/microsoft/pai) Training Platform (See [here](TrainingService/PaiMode.md) for instructions about how to submit NNI job in pai mode)
439
440
  * Support training services on pai mode. NNI trials will be scheduled to run on OpenPAI cluster
  * NNI trial's output (including logs and model file) will be copied to OpenPAI HDFS for further debugging and checking
441
* Support [SMAC](https://www.cs.ubc.ca/~hutter/papers/10-TR-SMAC.pdf) tuner (See [here](Tuner/SmacTuner.md) for instructions about how to use SMAC tuner)
442
443
444
445
446
  * [SMAC](https://www.cs.ubc.ca/~hutter/papers/10-TR-SMAC.pdf) is based on Sequential Model-Based Optimization (SMBO). It adapts the most prominent previously used model class (Gaussian stochastic process models) and introduces the model class of random forests to SMBO to handle categorical parameters. The SMAC supported by NNI is a wrapper on [SMAC3](https://github.com/automl/SMAC3)
* Support NNI installation on [conda](https://conda.io/docs/index.html) and python virtual environment
* Others
  * Update ga squad example and related documentation
  * WebUI UX small enhancement and bug fix
fishyds's avatar
fishyds committed
447
448


Yan Ni's avatar
Yan Ni committed
449
## Release 0.1.0 - 9/10/2018 (initial release)
450
451
452

Initial release of Neural Network Intelligence (NNI).

Yan Ni's avatar
Yan Ni committed
453
### Major Features
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469

* Installation and Deployment
  * Support pip install and source codes install
  * Support training services on local mode(including Multi-GPU mode) as well as multi-machines mode
* Tuners, Assessors and Trial
  * Support AutoML algorithms including:  hyperopt_tpe, hyperopt_annealing, hyperopt_random, and evolution_tuner
  * Support assessor(early stop) algorithms including: medianstop algorithm
  * Provide Python API for user defined tuners and assessors
  * Provide Python API for user to wrap trial code as NNI deployable codes
* Experiments
  * Provide a command line toolkit 'nnictl' for experiments management
  * Provide a WebUI for viewing experiments details and managing experiments
* Continuous Integration
  * Support CI by providing out-of-box integration with [travis-ci](https://github.com/travis-ci) on ubuntu
* Others
  * Support simple GPU job scheduling
470