Skip to content

GitLab

Unverified Commit ccff6733 authored by Chi Song's avatar Chi Song Committed by GitHub
Browse files

Doc fix: rewording and typos (#685)

parent 4539b4d3
Hide whitespace changes
Inline Side-by-side
Showing with 508 additions and 502 deletions
docs/Builtin_Assessors.md
View file @ ccff6733
...@@ -4,10 +4,8 @@ NNI provides state-of-the-art tuning algorithm in our builtin-assessors and make ...@@ -4,10 +4,8 @@ NNI provides state-of-the-art tuning algorithm in our builtin-assessors and make
|Assessor|Brief Introduction of Algorithm| |Assessor|Brief Introduction of Algorithm|
|---|---| |---|---|
|**Medianstop**<br>[(Usage)](#MedianStop)|Medianstop is a simple early stopping rule mentioned in the [paper](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/46180.pdf). It stops a pending trial X at step S if the trial’s best objective value by step S is strictly worse than the median value of the running averages of all completed trials’ objectives reported up to step S.| |**Medianstop** [(Usage)](#MedianStop)|Medianstop is a simple early stopping rule mentioned in the [paper](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/46180.pdf). It stops a pending trial X at step S if the trial’s best objective value by step S is strictly worse than the median value of the running averages of all completed trials’ objectives reported up to step S.|
|[Curvefitting](https://github.com/Microsoft/nni/blob/master/src/sdk/pynni/nni/curvefitting_assessor/README.md)<br>[(Usage)](#Curvefitting)|Curve Fitting Assessor is a LPA(learning, predicting, assessing) algorithm. It stops a pending trial X at step S if the prediction of final epoch's performance worse than the best final performance in the trial history. In this algorithm, we use 12 curves to fit the accuracy curve| |[Curvefitting](https://github.com/Microsoft/nni/blob/master/src/sdk/pynni/nni/curvefitting_assessor/README.md) [(Usage)](#Curvefitting)|Curve Fitting Assessor is a LPA(learning, predicting, assessing) algorithm. It stops a pending trial X at step S if the prediction of final epoch's performance worse than the best final performance in the trial history. In this algorithm, we use 12 curves to fit the accuracy curve|
<br>
## Usage of Builtin Assessors ## Usage of Builtin Assessors
......
docs/Builtin_Tuner.md
View file @ ccff6733
...@@ -4,16 +4,16 @@ NNI provides state-of-the-art tuning algorithm as our builtin-tuners and makes t ...@@ -4,16 +4,16 @@ NNI provides state-of-the-art tuning algorithm as our builtin-tuners and makes t
|Tuner|Brief Introduction of Algorithm| |Tuner|Brief Introduction of Algorithm|
|---|---| |---|---|
|**TPE**<br>[(Usage)](#TPE)|The Tree-structured Parzen Estimator (TPE) is a sequential model-based optimization (SMBO) approach. SMBO methods sequentially construct models to approximate the performance of hyperparameters based on historical measurements, and then subsequently choose new hyperparameters to test based on this model.| |**TPE** [(Usage)](#TPE)|The Tree-structured Parzen Estimator (TPE) is a sequential model-based optimization (SMBO) approach. SMBO methods sequentially construct models to approximate the performance of hyperparameters based on historical measurements, and then subsequently choose new hyperparameters to test based on this model.|
|**Random Search**<br>[(Usage)](#Random)|In Random Search for Hyper-Parameter Optimization show that Random Search might be surprisingly simple and effective. We suggest that we could use Random Search as the baseline when we have no knowledge about the prior distribution of hyper-parameters.| |**Random Search** [(Usage)](#Random)|In Random Search for Hyper-Parameter Optimization show that Random Search might be surprisingly simple and effective. We suggest that we could use Random Search as the baseline when we have no knowledge about the prior distribution of hyper-parameters.|
|**Anneal**<br>[(Usage)](#Anneal)|This simple annealing algorithm begins by sampling from the prior, but tends over time to sample from points closer and closer to the best ones observed. This algorithm is a simple variation on the random search that leverages smoothness in the response surface. The annealing rate is not adaptive.| |**Anneal** [(Usage)](#Anneal)|This simple annealing algorithm begins by sampling from the prior, but tends over time to sample from points closer and closer to the best ones observed. This algorithm is a simple variation on the random search that leverages smoothness in the response surface. The annealing rate is not adaptive.|
|**Naive Evolution**<br>[(Usage)](#Evolution)|Naive Evolution comes from Large-Scale Evolution of Image Classifiers. It randomly initializes a population-based on search space. For each generation, it chooses better ones and does some mutation (e.g., change a hyperparameter, add/remove one layer) on them to get the next generation. Naive Evolution requires many trials to works, but it's very simple and easy to expand new features.| |**Naive Evolution** [(Usage)](#Evolution)|Naive Evolution comes from Large-Scale Evolution of Image Classifiers. It randomly initializes a population-based on search space. For each generation, it chooses better ones and does some mutation (e.g., change a hyperparameter, add/remove one layer) on them to get the next generation. Naive Evolution requires many trials to works, but it's very simple and easy to expand new features.|
|**SMAC**<br>[(Usage)](#SMAC)|SMAC is based on Sequential Model-Based Optimization (SMBO). It adapts the most prominent previously used model class (Gaussian stochastic process models) and introduces the model class of random forests to SMBO, in order to handle categorical parameters. The SMAC supported by nni is a wrapper on the SMAC3 Github repo.| |**SMAC** [(Usage)](#SMAC)|SMAC is based on Sequential Model-Based Optimization (SMBO). It adapts the most prominent previously used model class (Gaussian stochastic process models) and introduces the model class of random forests to SMBO, in order to handle categorical parameters. The SMAC supported by nni is a wrapper on the SMAC3 Github repo.|
|**Batch tuner**<br>[(Usage)](#Batch)|Batch tuner allows users to simply provide several configurations (i.e., choices of hyper-parameters) for their trial code. After finishing all the configurations, the experiment is done. Batch tuner only supports the type choice in search space spec.| |**Batch tuner** [(Usage)](#Batch)|Batch tuner allows users to simply provide several configurations (i.e., choices of hyper-parameters) for their trial code. After finishing all the configurations, the experiment is done. Batch tuner only supports the type choice in search space spec.|
|**Grid Search**<br>[(Usage)](#GridSearch)|Grid Search performs an exhaustive searching through a manually specified subset of the hyperparameter space defined in the searchspace file. Note that the only acceptable types of search space are choice, quniform, qloguniform. The number q in quniform and qloguniform has special meaning (different from the spec in search space spec). It means the number of values that will be sampled evenly from the range low and high.| |**Grid Search** [(Usage)](#GridSearch)|Grid Search performs an exhaustive searching through a manually specified subset of the hyperparameter space defined in the searchspace file. Note that the only acceptable types of search space are choice, quniform, qloguniform. The number q in quniform and qloguniform has special meaning (different from the spec in search space spec). It means the number of values that will be sampled evenly from the range low and high.|
|[Hyperband](https://github.com/Microsoft/nni/tree/master/src/sdk/pynni/nni/hyperband_advisor)<br>[(Usage)](#Hyperband)|Hyperband tries to use the limited resource to explore as many configurations as possible, and finds out the promising ones to get the final result. The basic idea is generating many configurations and to run them for the small number of STEPs to find out promising one, then further training those promising ones to select several more promising one.| |[Hyperband](https://github.com/Microsoft/nni/tree/master/src/sdk/pynni/nni/hyperband_advisor) [(Usage)](#Hyperband)|Hyperband tries to use the limited resource to explore as many configurations as possible, and finds out the promising ones to get the final result. The basic idea is generating many configurations and to run them for the small number of STEPs to find out promising one, then further training those promising ones to select several more promising one.|
|[Network Morphism](https://github.com/Microsoft/nni/blob/master/src/sdk/pynni/nni/networkmorphism_tuner/README.md)<br>[(Usage)](#NetworkMorphism)|Network Morphism provides functions to automatically search for architecture of deep learning models. Every child network inherits the knowledge from its parent network and morphs into diverse types of networks, including changes of depth, width, and skip-connection. Next, it estimates the value of a child network using the historic architecture and metric pairs. Then it selects the most promising one to train.| |[Network Morphism](https://github.com/Microsoft/nni/blob/master/src/sdk/pynni/nni/networkmorphism_tuner/README.md) [(Usage)](#NetworkMorphism)|Network Morphism provides functions to automatically search for architecture of deep learning models. Every child network inherits the knowledge from its parent network and morphs into diverse types of networks, including changes of depth, width, and skip-connection. Next, it estimates the value of a child network using the historic architecture and metric pairs. Then it selects the most promising one to train.|
|**Metis Tuner**<br>[(Usage)](#MetisTuner)|Metis offers the following benefits when it comes to tuning parameters: While most tools only predict the optimal configuration, Metis gives you two outputs: (a) current prediction of optimal configuration, and (b) suggestion for the next trial. No more guesswork. While most tools assume training datasets do not have noisy data, Metis actually tells you if you need to re-sample a particular hyper-parameter.| |**Metis Tuner** [(Usage)](#MetisTuner)|Metis offers the following benefits when it comes to tuning parameters: While most tools only predict the optimal configuration, Metis gives you two outputs: (a) current prediction of optimal configuration, and (b) suggestion for the next trial. No more guesswork. While most tools assume training datasets do not have noisy data, Metis actually tells you if you need to re-sample a particular hyper-parameter.|
<br> <br>
......
docs/Customize_Assessor.md
View file @ ccff6733
# Customize Assessor # Customize Assessor
NNI also support building an assessor by yourself to adjust your tuning demand. NNI supports to build an assessor by yourself for tuning demand.
If you want to implement a customized Assessor, there are three things for you to do: If you want to implement a customized Assessor, there are three things to do:
1) Inherit an assessor of a base Assessor class 1. Inherit the base Assessor class
2) Implement assess_trial function 1. Implement assess_trial function
3) Configure your customized Assessor in experiment YAML config file 1. Configure your customized Assessor in experiment YAML config file
**1. Inherit an assessor of a base Assessor class** **1. Inherit the base Assessor class**
```python ```python
from nni.assessor import Assessor from nni.assessor import Assessor
...@@ -19,6 +19,7 @@ class CustomizedAssessor(Assessor): ...@@ -19,6 +19,7 @@ class CustomizedAssessor(Assessor):
``` ```
**2. Implement assess trial function** **2. Implement assess trial function**
```python ```python
from nni.assessor import Assessor, AssessResult from nni.assessor import Assessor, AssessResult
...@@ -41,7 +42,6 @@ class CustomizedAssessor(Assessor): ...@@ -41,7 +42,6 @@ class CustomizedAssessor(Assessor):
NNI needs to locate your customized Assessor class and instantiate the class, so you need to specify the location of the customized Assessor class and pass literal values as parameters to the \_\_init__ constructor. NNI needs to locate your customized Assessor class and instantiate the class, so you need to specify the location of the customized Assessor class and pass literal values as parameters to the \_\_init__ constructor.
```yaml ```yaml
assessor: assessor:
codeDir: /home/abc/myassessor codeDir: /home/abc/myassessor
classFileName: my_customized_assessor.py classFileName: my_customized_assessor.py
...@@ -50,7 +50,6 @@ assessor: ...@@ -50,7 +50,6 @@ assessor:
# can be specified in this optional classArgs field, for example # can be specified in this optional classArgs field, for example
classArgs: classArgs:
arg1: value1 arg1: value1
``` ```
Please noted in **2**. The object `trial_history` are exact the object that Trial send to Assessor by using SDK `report_intermediate_result` function. Please noted in **2**. The object `trial_history` are exact the object that Trial send to Assessor by using SDK `report_intermediate_result` function.
......
docs/Customize_Tuner.md
View file @ ccff6733
...@@ -2,17 +2,17 @@ ...@@ -2,17 +2,17 @@
## Customize Tuner ## Customize Tuner
NNI provides state-of-the-art tuning algorithm in our builtin-tuners. We also support building a tuner by yourself to adjust your tuning demand. NNI provides state-of-the-art tuning algorithm in builtin-tuners. NNI supports to build a tuner by yourself for tuning demand.
If you want to implement and use your own tuning algorithm, you can implement a customized Tuner, there are three things for you to do: If you want to implement your own tuning algorithm, you can implement a customized Tuner, there are three things to do:
1) Inherit a tuner of a base Tuner class 1. Inherit the base Tuner class
2) Implement receive_trial_result and generate_parameter function 1. Implement receive_trial_result and generate_parameter function
3) Configure your customized tuner in experiment YAML config file 1. Configure your customized tuner in experiment YAML config file
Here is an example: Here is an example:
**1. Inherit a tuner of a base Tuner class** **1. Inherit the base Tuner class**
```python ```python
from nni.tuner import Tuner from nni.tuner import Tuner
...@@ -33,10 +33,10 @@ class CustomizedTuner(Tuner): ...@@ -33,10 +33,10 @@ class CustomizedTuner(Tuner):
def receive_trial_result(self, parameter_id, parameters, value): def receive_trial_result(self, parameter_id, parameters, value):
''' '''
Record an observation of the objective function and Train Receive trial's final result.
parameter_id: int parameter_id: int
parameters: object created by 'generate_parameters()' parameters: object created by 'generate_parameters()'
value: final metrics of the trial, including default matrix value: final metrics of the trial, including default metric
''' '''
# your code implements here. # your code implements here.
... ...
...@@ -59,7 +59,6 @@ For example: ...@@ -59,7 +59,6 @@ For example:
If the you implement the `generate_parameters` like this: If the you implement the `generate_parameters` like this:
```python ```python
def generate_parameters(self, parameter_id): def generate_parameters(self, parameter_id):
''' '''
Returns a set of trial (hyper-)parameters, as a serializable object Returns a set of trial (hyper-)parameters, as a serializable object
...@@ -72,21 +71,17 @@ def generate_parameters(self, parameter_id): ...@@ -72,21 +71,17 @@ def generate_parameters(self, parameter_id):
It means your Tuner will always generate parameters `{"dropout": 0.3, "learning_rate": 0.4}`. Then Trial will receive `{"dropout": 0.3, "learning_rate": 0.4}` by calling API `nni.get_next_parameter()`. Once the trial ends with a result (normally some kind of metrics), it can send the result to Tuner by calling API `nni.report_final_result()`, for example `nni.report_final_result(0.93)`. Then your Tuner's `receive_trial_result` function will receied the result like： It means your Tuner will always generate parameters `{"dropout": 0.3, "learning_rate": 0.4}`. Then Trial will receive `{"dropout": 0.3, "learning_rate": 0.4}` by calling API `nni.get_next_parameter()`. Once the trial ends with a result (normally some kind of metrics), it can send the result to Tuner by calling API `nni.report_final_result()`, for example `nni.report_final_result(0.93)`. Then your Tuner's `receive_trial_result` function will receied the result like：
``` ```python
parameter_id = 82347 parameter_id = 82347
parameters = {"dropout": 0.3, "learning_rate": 0.4} parameters = {"dropout": 0.3, "learning_rate": 0.4}
value = 0.93 value = 0.93
``` ```
**Note that** if you want to access a file (e.g., `data.txt`) in the directory of your own tuner, you cannot use `open('data.txt', 'r')`. Instead, you should use the following: **Note that** if you want to access a file (e.g., `data.txt`) in the directory of your own tuner, you cannot use `open('data.txt', 'r')`. Instead, you should use the following:
``` ```python
_pwd = os.path.dirname(__file__) _pwd = os.path.dirname(__file__)
_fd = open(os.path.join(_pwd, 'data.txt'), 'r') _fd = open(os.path.join(_pwd, 'data.txt'), 'r')
``` ```
This is because your tuner is not executed in the directory of your tuner (i.e., `pwd` is not the directory of your own tuner). This is because your tuner is not executed in the directory of your tuner (i.e., `pwd` is not the directory of your own tuner).
...@@ -96,7 +91,6 @@ This is because your tuner is not executed in the directory of your tuner (i.e., ...@@ -96,7 +91,6 @@ This is because your tuner is not executed in the directory of your tuner (i.e.,
NNI needs to locate your customized tuner class and instantiate the class, so you need to specify the location of the customized tuner class and pass literal values as parameters to the \_\_init__ constructor. NNI needs to locate your customized tuner class and instantiate the class, so you need to specify the location of the customized tuner class and pass literal values as parameters to the \_\_init__ constructor.
```yaml ```yaml
tuner: tuner:
codeDir: /home/abc/mytuner codeDir: /home/abc/mytuner
classFileName: my_customized_tuner.py classFileName: my_customized_tuner.py
......
docs/ExperimentConfig.md
View file @ ccff6733
...@@ -4,184 +4,186 @@ A config file is needed when create an experiment, the path of the config file i ...@@ -4,184 +4,186 @@ A config file is needed when create an experiment, the path of the config file i
The config file is written in YAML format, and need to be written correctly. The config file is written in YAML format, and need to be written correctly.
This document describes the rule to write config file, and will provide some examples and templates. This document describes the rule to write config file, and will provide some examples and templates.
- [Template](#Template) (the templates of an config file) * [Template](#Template) (the templates of an config file)
- [Configuration spec](#Configuration) (the configuration specification of every attribute in config file) * [Configuration spec](#Configuration) (the configuration specification of every attribute in config file)
- [Examples](#Examples) (the examples of config file) * [Examples](#Examples) (the examples of config file)
<a name="Template"></a> <a name="Template"></a>
## Template ## Template
* __light weight(without Annotation and Assessor)__
``` * __light weight(without Annotation and Assessor)__
authorName:
experimentName: ```yaml
trialConcurrency: authorName:
maxExecDuration: experimentName:
maxTrialNum: trialConcurrency:
maxExecDuration:
maxTrialNum:
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: trainingServicePlatform:
searchSpacePath: searchSpacePath:
#choice: true, false #choice: true, false
useAnnotation: useAnnotation:
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: builtinTunerName:
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: optimize_mode:
gpuNum: gpuNum:
trial: trial:
command: command:
codeDir: codeDir:
gpuNum: gpuNum:
#machineList can be empty if the platform is local #machineList can be empty if the platform is local
machineList: machineList:
- ip: - ip:
port: port:
username: username:
passwd: passwd:
``` ```
* __Use Assessor__ * __Use Assessor__
``` ```yaml
authorName: authorName:
experimentName: experimentName:
trialConcurrency: trialConcurrency:
maxExecDuration: maxExecDuration:
maxTrialNum: maxTrialNum:
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: trainingServicePlatform:
searchSpacePath: searchSpacePath:
#choice: true, false #choice: true, false
useAnnotation: useAnnotation:
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: builtinTunerName:
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: optimize_mode:
gpuNum: gpuNum:
assessor: assessor:
#choice: Medianstop #choice: Medianstop
builtinAssessorName: builtinAssessorName:
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: optimize_mode:
gpuNum: gpuNum:
trial: trial:
command: command:
codeDir: codeDir:
gpuNum: gpuNum:
#machineList can be empty if the platform is local #machineList can be empty if the platform is local
machineList: machineList:
- ip: - ip:
port: port:
username: username:
passwd: passwd:
``` ```
* __Use Annotation__ * __Use Annotation__
``` ```yaml
authorName: authorName:
experimentName: experimentName:
trialConcurrency: trialConcurrency:
maxExecDuration: maxExecDuration:
maxTrialNum: maxTrialNum:
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: trainingServicePlatform:
#choice: true, false #choice: true, false
useAnnotation: useAnnotation:
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: builtinTunerName:
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: optimize_mode:
gpuNum: gpuNum:
assessor: assessor:
#choice: Medianstop #choice: Medianstop
builtinAssessorName: builtinAssessorName:
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: optimize_mode:
gpuNum: gpuNum:
trial: trial:
command: command:
codeDir: codeDir:
gpuNum: gpuNum:
#machineList can be empty if the platform is local #machineList can be empty if the platform is local
machineList: machineList:
- ip: - ip:
port: port:
username: username:
passwd: passwd:
``` ```
<a name="Configuration"></a> <a name="Configuration"></a>
## Configuration spec ## Configuration spec
* __authorName__ * __authorName__
* Description * Description
__authorName__ is the name of the author who create the experiment. __authorName__ is the name of the author who create the experiment.
TBD: add default value TBD: add default value
* __experimentName__ * __experimentName__
* Description * Description
__experimentName__ is the name of the experiment created. __experimentName__ is the name of the experiment created.
TBD: add default value TBD: add default value
* __trialConcurrency__ * __trialConcurrency__
* Description * Description
__trialConcurrency__ specifies the max num of trial jobs run simultaneously. __trialConcurrency__ specifies the max num of trial jobs run simultaneously.
Note: if trialGpuNum is bigger than the free gpu numbers, and the trial jobs running simultaneously can not reach trialConcurrency number, some trial jobs will be put into a queue to wait for gpu allocation. Note: if trialGpuNum is bigger than the free gpu numbers, and the trial jobs running simultaneously can not reach trialConcurrency number, some trial jobs will be put into a queue to wait for gpu allocation.
* __maxExecDuration__ * __maxExecDuration__
* Description * Description
__maxExecDuration__ specifies the max duration time of an experiment.The unit of the time is {__s__, __m__, __h__, __d__}, which means {_seconds_, _minutes_, _hours_, _days_}.
Note: The maxExecDuration spec set the time of an experiment, not a trial job. If the experiment reach the max duration time, the experiment will not stop, but could not submit new trial jobs any more. __maxExecDuration__ specifies the max duration time of an experiment.The unit of the time is {__s__, __m__, __h__, __d__}, which means {_seconds_, _minutes_, _hours_, _days_}.
Note: The maxExecDuration spec set the time of an experiment, not a trial job. If the experiment reach the max duration time, the experiment will not stop, but could not submit new trial jobs any more.
* __maxTrialNum__ * __maxTrialNum__
* Description * Description
__maxTrialNum__ specifies the max number of trial jobs created by NNI, including succeeded and failed jobs. __maxTrialNum__ specifies the max number of trial jobs created by NNI, including succeeded and failed jobs.
* __trainingServicePlatform__ * __trainingServicePlatform__
* Description * Description
__trainingServicePlatform__ specifies the platform to run the experiment, including {__local__, __remote__, __pai__, __kubeflow__}. __trainingServicePlatform__ specifies the platform to run the experiment, including {__local__, __remote__, __pai__, __kubeflow__}.
* __local__ run an experiment on local ubuntu machine. * __local__ run an experiment on local ubuntu machine.
* __remote__ submit trial jobs to remote ubuntu machines, and __machineList__ field should be filed in order to set up SSH connection to remote machine. * __remote__ submit trial jobs to remote ubuntu machines, and __machineList__ field should be filed in order to set up SSH connection to remote machine.
* __pai__ submit trial jobs to [OpenPai](https://github.com/Microsoft/pai) of Microsoft. For more details of pai configuration, please reference [PAIMOdeDoc](./PAIMode.md) * __pai__ submit trial jobs to [OpenPai](https://github.com/Microsoft/pai) of Microsoft. For more details of pai configuration, please reference [PAIMOdeDoc](./PAIMode.md)
* __kubeflow__ submit trial jobs to [kubeflow](https://www.kubeflow.org/docs/about/kubeflow/), NNI support kubeflow based on normal kubernetes and [azure kubernetes](https://azure.microsoft.com/en-us/services/kubernetes-service/). * __kubeflow__ submit trial jobs to [kubeflow](https://www.kubeflow.org/docs/about/kubeflow/), NNI support kubeflow based on normal kubernetes and [azure kubernetes](https://azure.microsoft.com/en-us/services/kubernetes-service/).
* __searchSpacePath__ * __searchSpacePath__
* Description * Description
__searchSpacePath__ specifies the path of search space file, which should be a valid path in the local linux machine. __searchSpacePath__ specifies the path of search space file, which should be a valid path in the local linux machine.
Note: if set useAnnotation=True, the searchSpacePath field should be removed. Note: if set useAnnotation=True, the searchSpacePath field should be removed.
* __useAnnotation__ * __useAnnotation__
* Description * Description
__useAnnotation__ use annotation to analysis trial code and generate search space. __useAnnotation__ use annotation to analysis trial code and generate search space.
Note: if set useAnnotation=True, the searchSpacePath field should be removed. Note: if set useAnnotation=True, the searchSpacePath field should be removed.
* __nniManagerIp__ * __nniManagerIp__
* Description * Description
__nniManagerIp__ set the IP address of the machine on which NNI manager process runs. This field is optional, and if it's not set, eth0 device IP will be used instead. __nniManagerIp__ set the IP address of the machine on which NNI manager process runs. This field is optional, and if it's not set, eth0 device IP will be used instead.
Note: run ifconfig on NNI manager's machine to check if eth0 device exists. If not, we recommend to set nnimanagerIp explicitly. Note: run ifconfig on NNI manager's machine to check if eth0 device exists. If not, we recommend to set nnimanagerIp explicitly.
...@@ -196,95 +198,109 @@ machineList: ...@@ -196,95 +198,109 @@ machineList:
__logLevel__ sets log level for the experiment, available log levels are: `trace, debug, info, warning, error, fatal`. The default value is `info`. __logLevel__ sets log level for the experiment, available log levels are: `trace, debug, info, warning, error, fatal`. The default value is `info`.
* __tuner__ * __tuner__
* Description * Description
__tuner__ specifies the tuner algorithm in the experiment, there are two kinds of ways to set tuner. One way is to use tuner provided by NNI sdk, need to set __builtinTunerName__ and __classArgs__. Another way is to use users' own tuner file, and need to set __codeDirectory__, __classFileName__, __className__ and __classArgs__. __tuner__ specifies the tuner algorithm in the experiment, there are two kinds of ways to set tuner. One way is to use tuner provided by NNI sdk, need to set __builtinTunerName__ and __classArgs__. Another way is to use users' own tuner file, and need to set __codeDirectory__, __classFileName__, __className__ and __classArgs__.
* __builtinTunerName__ and __classArgs__ * __builtinTunerName__ and __classArgs__
* __builtinTunerName__ * __builtinTunerName__
__builtinTunerName__ specifies the name of system tuner, NNI sdk provides four kinds of tuner, including {__TPE__, __Random__, __Anneal__, __Evolution__, __BatchTuner__, __GridSearch__} __builtinTunerName__ specifies the name of system tuner, NNI sdk provides four kinds of tuner, including {__TPE__, __Random__, __Anneal__, __Evolution__, __BatchTuner__, __GridSearch__}
* __classArgs__ * __classArgs__
__classArgs__ specifies the arguments of tuner algorithm. If the __builtinTunerName__ is in {__TPE__, __Random__, __Anneal__, __Evolution__}, user should set __optimize_mode__. __classArgs__ specifies the arguments of tuner algorithm. If the __builtinTunerName__ is in {__TPE__, __Random__, __Anneal__, __Evolution__}, user should set __optimize_mode__.
* __codeDir__, __classFileName__, __className__ and __classArgs__ * __codeDir__, __classFileName__, __className__ and __classArgs__
* __codeDir__ * __codeDir__
__codeDir__ specifies the directory of tuner code. __codeDir__ specifies the directory of tuner code.
* __classFileName__ * __classFileName__
__classFileName__ specifies the name of tuner file. __classFileName__ specifies the name of tuner file.
* __className__ * __className__
__className__ specifies the name of tuner class. __className__ specifies the name of tuner class.
* __classArgs__ * __classArgs__
__classArgs__ specifies the arguments of tuner algorithm. __classArgs__ specifies the arguments of tuner algorithm.
* __gpuNum__ * __gpuNum__
__gpuNum__ specifies the gpu number to run the tuner process. The value of this field should be a positive number. __gpuNum__ specifies the gpu number to run the tuner process. The value of this field should be a positive number.
Note: users could only specify one way to set tuner, for example, set {tunerName, optimizationMode} or {tunerCommand, tunerCwd}, and could not set them both. Note: users could only specify one way to set tuner, for example, set {tunerName, optimizationMode} or {tunerCommand, tunerCwd}, and could not set them both.
* __assessor__ * __assessor__
* Description * Description
__assessor__ specifies the assessor algorithm to run an experiment, there are two kinds of ways to set assessor. One way is to use assessor provided by NNI sdk, users need to set __builtinAssessorName__ and __classArgs__. Another way is to use users' own assessor file, and need to set __codeDirectory__, __classFileName__, __className__ and __classArgs__. __assessor__ specifies the assessor algorithm to run an experiment, there are two kinds of ways to set assessor. One way is to use assessor provided by NNI sdk, users need to set __builtinAssessorName__ and __classArgs__. Another way is to use users' own assessor file, and need to set __codeDirectory__, __classFileName__, __className__ and __classArgs__.
* __builtinAssessorName__ and __classArgs__ * __builtinAssessorName__ and __classArgs__
* __builtinAssessorName__ * __builtinAssessorName__
__builtinAssessorName__ specifies the name of system assessor, NNI sdk provides one kind of assessor {__Medianstop__} __builtinAssessorName__ specifies the name of system assessor, NNI sdk provides one kind of assessor {__Medianstop__}
* __classArgs__ * __classArgs__
__classArgs__ specifies the arguments of assessor algorithm __classArgs__ specifies the arguments of assessor algorithm
* __codeDir__, __classFileName__, __className__ and __classArgs__ * __codeDir__, __classFileName__, __className__ and __classArgs__
* __codeDir__ * __codeDir__
__codeDir__ specifies the directory of assessor code. __codeDir__ specifies the directory of assessor code.
* __classFileName__ * __classFileName__
__classFileName__ specifies the name of assessor file. __classFileName__ specifies the name of assessor file.
* __className__ * __className__
__className__ specifies the name of assessor class. __className__ specifies the name of assessor class.
* __classArgs__ * __classArgs__
__classArgs__ specifies the arguments of assessor algorithm. __classArgs__ specifies the arguments of assessor algorithm.
* __gpuNum__ * __gpuNum__
__gpuNum__ specifies the gpu number to run the assessor process. The value of this field should be a positive number.
Note: users' could only specify one way to set assessor, for example,set {assessorName, optimizationMode} or {assessorCommand, assessorCwd}, and users could not set them both.If users do not want to use assessor, assessor fileld should leave to empty. __gpuNum__ specifies the gpu number to run the assessor process. The value of this field should be a positive number.
Note: users' could only specify one way to set assessor, for example,set {assessorName, optimizationMode} or {assessorCommand, assessorCwd}, and users could not set them both.If users do not want to use assessor, assessor fileld should leave to empty.
* __trial(local, remote)__ * __trial(local, remote)__
* __command__ * __command__
__command__ specifies the command to run trial process. __command__ specifies the command to run trial process.
* __codeDir__ * __codeDir__
__codeDir__ specifies the directory of your own trial file. __codeDir__ specifies the directory of your own trial file.
* __gpuNum__ * __gpuNum__
__gpuNum__ specifies the num of gpu to run the trial process. Default value is 0. __gpuNum__ specifies the num of gpu to run the trial process. Default value is 0.
* __trial(pai)__ * __trial(pai)__
* __command__ * __command__
__command__ specifies the command to run trial process. __command__ specifies the command to run trial process.
* __codeDir__ * __codeDir__
__codeDir__ specifies the directory of the own trial file. __codeDir__ specifies the directory of the own trial file.
* __gpuNum__ * __gpuNum__
__gpuNum__ specifies the num of gpu to run the trial process. Default value is 0. __gpuNum__ specifies the num of gpu to run the trial process. Default value is 0.
* __cpuNum__ * __cpuNum__
__cpuNum__ is the cpu number of cpu to be used in pai container. __cpuNum__ is the cpu number of cpu to be used in pai container.
* __memoryMB__ * __memoryMB__
__memoryMB__ set the momory size to be used in pai's container. __memoryMB__ set the momory size to be used in pai's container.
* __image__ * __image__
__image__ set the image to be used in pai. __image__ set the image to be used in pai.
...@@ -292,417 +308,416 @@ machineList: ...@@ -292,417 +308,416 @@ machineList:
* __dataDir__ * __dataDir__
__dataDir__ is the data directory in hdfs to be used. __dataDir__ is the data directory in hdfs to be used.
* __outputDir__ * __outputDir__
__outputDir__ is the output directory in hdfs to be used in pai, the stdout and stderr files are stored in the directory after job finished.
__outputDir__ is the output directory in hdfs to be used in pai, the stdout and stderr files are stored in the directory after job finished.
* __trial(kubeflow)__ * __trial(kubeflow)__
* __codeDir__ * __codeDir__
__codeDir__ is the local directory where the code files in. __codeDir__ is the local directory where the code files in.
* __ps(optional)__ * __ps(optional)__
__ps__ is the configuration for kubeflow's tensorflow-operator. __ps__ is the configuration for kubeflow's tensorflow-operator.
* __replicas__ * __replicas__
__replicas__ is the replica number of __ps__ role. __replicas__ is the replica number of __ps__ role.
* __command__ * __command__
__command__ is the run script in __ps__'s container. __command__ is the run script in __ps__'s container.
* __gpuNum__ * __gpuNum__
__gpuNum__ set the gpu number to be used in __ps__ container. __gpuNum__ set the gpu number to be used in __ps__ container.
* __cpuNum__ * __cpuNum__
__cpuNum__ set the cpu number to be used in __ps__ container. __cpuNum__ set the cpu number to be used in __ps__ container.
* __memoryMB__ * __memoryMB__
__memoryMB__ set the memory size of the container. __memoryMB__ set the memory size of the container.
* __image__ * __image__
__image__ set the image to be used in __ps__. __image__ set the image to be used in __ps__.
* __worker__ * __worker__
__worker__ is the configuration for kubeflow's tensorflow-operator. __worker__ is the configuration for kubeflow's tensorflow-operator.
* __replicas__ * __replicas__
__replicas__ is the replica number of __worker__ role. __replicas__ is the replica number of __worker__ role.
* __command__ * __command__
__command__ is the run script in __worker__'s container. __command__ is the run script in __worker__'s container.
* __gpuNum__ * __gpuNum__
__gpuNum__ set the gpu number to be used in __worker__ container. __gpuNum__ set the gpu number to be used in __worker__ container.
* __cpuNum__ * __cpuNum__
__cpuNum__ set the cpu number to be used in __worker__ container. __cpuNum__ set the cpu number to be used in __worker__ container.
* __memoryMB__ * __memoryMB__
__memoryMB__ set the memory size of the container. __memoryMB__ set the memory size of the container.
* __image__ * __image__
__image__ set the image to be used in __worker__. __image__ set the image to be used in __worker__.
* __machineList__
__machineList__ should be set if __trainingServicePlatform__ is set to remote, or it should be empty.
* __machineList__
__machineList__ should be set if users set __trainingServicePlatform__=remote, or it could be empty.
* __ip__ * __ip__
__ip__ is the ip address of remote machine. __ip__ is the ip address of remote machine.
* __port__ * __port__
__port__ is the ssh port to be used to connect machine. __port__ is the ssh port to be used to connect machine.
Note: if users set port empty, the default value will be 22. Note: if users set port empty, the default value will be 22.
* __username__ * __username__
__username__ is the account of remote machine. __username__ is the account of remote machine.
* __passwd__ * __passwd__
__passwd__ specifies the password of the account. __passwd__ specifies the password of the account.
* __sshKeyPath__ * __sshKeyPath__
If users use ssh key to login remote machine, could set __sshKeyPath__ in config file. __sshKeyPath__ is the path of ssh key file, which should be valid. If users use ssh key to login remote machine, could set __sshKeyPath__ in config file. __sshKeyPath__ is the path of ssh key file, which should be valid.
Note: if users set passwd and sshKeyPath simultaneously, NNI will try passwd. Note: if users set passwd and sshKeyPath simultaneously, NNI will try passwd.
* __passphrase__ * __passphrase__
__passphrase__ is used to protect ssh key, which could be empty if users don't have passphrase. __passphrase__ is used to protect ssh key, which could be empty if users don't have passphrase.
* __kubeflowConfig__: * __kubeflowConfig__:
* __operator__ * __operator__
__operator__ specify the kubeflow's operator to be used, NNI support __tf-operator__ in current version. __operator__ specify the kubeflow's operator to be used, NNI support __tf-operator__ in current version.
* __storage__ * __storage__
__storage__ specify the storage type of kubeflow, including {__nfs__, __azureStorage__}. This field is optional, and the default value is __nfs__. If the config use azureStorage, this field must be completed. __storage__ specify the storage type of kubeflow, including {__nfs__, __azureStorage__}. This field is optional, and the default value is __nfs__. If the config use azureStorage, this field must be completed.
* __nfs__ * __nfs__
__server__ is the host of nfs server __server__ is the host of nfs server
__path__ is the mounted path of nfs __path__ is the mounted path of nfs
* __keyVault__ * __keyVault__
If users want to use azure kubernetes service, they should set keyVault to storage the private key of your azure storage account. Refer: https://docs.microsoft.com/en-us/azure/key-vault/key-vault-manage-with-cli2 If users want to use azure kubernetes service, they should set keyVault to storage the private key of your azure storage account. Refer: https://docs.microsoft.com/en-us/azure/key-vault/key-vault-manage-with-cli2
* __vaultName__ * __vaultName__
__vaultName__ is the value of ```--vault-name``` used in az command. __vaultName__ is the value of `--vault-name` used in az command.
* __name__ * __name__
__name__ is the value of ```--name``` used in az command. __name__ is the value of `--name` used in az command.
* __azureStorage__ * __azureStorage__
If users use azure kubernetes service, they should set azure storage account to store code files. If users use azure kubernetes service, they should set azure storage account to store code files.
* __accountName__ * __accountName__
__accountName__ is the name of azure storage account. __accountName__ is the name of azure storage account.
* __azureShare__ * __azureShare__
__azureShare__ is the share of the azure file storage. __azureShare__ is the share of the azure file storage.
* __paiConfig__ * __paiConfig__
* __userName__ * __userName__
__userName__ is the user name of your pai account. __userName__ is the user name of your pai account.
* __password__ * __password__
__password__ is the password of the pai account. __password__ is the password of the pai account.
* __host__ * __host__
__host__ is the host of pai. __host__ is the host of pai.
<a name="Examples"></a> <a name="Examples"></a>
## Examples ## Examples
* __local mode__ * __local mode__
If users want to run trial jobs in local machine, and use annotation to generate search space, could use the following config: If users want to run trial jobs in local machine, and use annotation to generate search space, could use the following config:
``` ```yaml
authorName: test authorName: test
experimentName: test_experiment experimentName: test_experiment
trialConcurrency: 3 trialConcurrency: 3
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 10 maxTrialNum: 10
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: local trainingServicePlatform: local
#choice: true, false #choice: true, false
useAnnotation: true useAnnotation: true
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
trial: trial:
command: python3 mnist.py command: python3 mnist.py
codeDir: /nni/mnist codeDir: /nni/mnist
gpuNum: 0 gpuNum: 0
``` ```
Could add assessor configuration in config file if set assessor. You can add assessor configuration.
``` ```yaml
authorName: test authorName: test
experimentName: test_experiment experimentName: test_experiment
trialConcurrency: 3 trialConcurrency: 3
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 10 maxTrialNum: 10
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: local trainingServicePlatform: local
searchSpacePath: /nni/search_space.json searchSpacePath: /nni/search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
assessor: assessor:
#choice: Medianstop #choice: Medianstop
builtinAssessorName: Medianstop builtinAssessorName: Medianstop
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
trial: trial:
command: python3 mnist.py command: python3 mnist.py
codeDir: /nni/mnist codeDir: /nni/mnist
gpuNum: 0 gpuNum: 0
``` ```
Or you could specify your own tuner and assessor file as following: Or you could specify your own tuner and assessor file as following,
``` ```yaml
authorName: test authorName: test
experimentName: test_experiment experimentName: test_experiment
trialConcurrency: 3 trialConcurrency: 3
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 10 maxTrialNum: 10
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: local trainingServicePlatform: local
searchSpacePath: /nni/search_space.json searchSpacePath: /nni/search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
tuner: tuner:
codeDir: /nni/tuner codeDir: /nni/tuner
classFileName: mytuner.py classFileName: mytuner.py
className: MyTuner className: MyTuner
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
assessor: assessor:
codeDir: /nni/assessor codeDir: /nni/assessor
classFileName: myassessor.py classFileName: myassessor.py
className: MyAssessor className: MyAssessor
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
trial: trial:
command: python3 mnist.py command: python3 mnist.py
codeDir: /nni/mnist codeDir: /nni/mnist
gpuNum: 0 gpuNum: 0
``` ```
* __remote mode__ * __remote mode__
If run trial jobs in remote machine, users could specify the remote mahcine information as fllowing format: If run trial jobs in remote machine, users could specify the remote mahcine information as fllowing format:
``` ```yaml
authorName: test authorName: test
experimentName: test_experiment experimentName: test_experiment
trialConcurrency: 3 trialConcurrency: 3
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 10 maxTrialNum: 10
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: remote trainingServicePlatform: remote
searchSpacePath: /nni/search_space.json searchSpacePath: /nni/search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
trial: trial:
command: python3 mnist.py command: python3 mnist.py
codeDir: /nni/mnist codeDir: /nni/mnist
gpuNum: 0 gpuNum: 0
#machineList can be empty if the platform is local #machineList can be empty if the platform is local
machineList: machineList:
- ip: 10.10.10.10 - ip: 10.10.10.10
port: 22 port: 22
username: test username: test
passwd: test passwd: test
- ip: 10.10.10.11 - ip: 10.10.10.11
port: 22 port: 22
username: test username: test
passwd: test passwd: test
- ip: 10.10.10.12 - ip: 10.10.10.12
port: 22 port: 22
username: test username: test
sshKeyPath: /nni/sshkey sshKeyPath: /nni/sshkey
passphrase: qwert passphrase: qwert
``` ```
* __pai mode__ * __pai mode__
``` ```yaml
authorName: test authorName: test
experimentName: nni_test1 experimentName: nni_test1
trialConcurrency: 1 trialConcurrency: 1
maxExecDuration:500h maxExecDuration:500h
maxTrialNum: 1 maxTrialNum: 1
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: pai trainingServicePlatform: pai
searchSpacePath: search_space.json searchSpacePath: search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution, BatchTuner #choice: TPE, Random, Anneal, Evolution, BatchTuner
#SMAC (SMAC should be installed through nnictl) #SMAC (SMAC should be installed through nnictl)
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
trial: trial:
command: python3 main.py command: python3 main.py
codeDir: . codeDir: .
gpuNum: 4 gpuNum: 4
cpuNum: 2 cpuNum: 2
memoryMB: 10000 memoryMB: 10000
#The docker image to run NNI job on pai #The docker image to run NNI job on pai
image: msranni/nni:latest
#The hdfs directory to store data on pai, format 'hdfs://host:port/directory'
dataDir: hdfs://10.11.12.13:9000/test
#The hdfs directory to store output data generated by NNI, format 'hdfs://host:port/directory'
outputDir: hdfs://10.11.12.13:9000/test
paiConfig:
#The username to login pai
userName: test
#The password to login pai
passWord: test
#The host of restful server of pai
host: 10.10.10.10
```
* __kubeflow mode__
kubeflow use nfs as storage.
```
authorName: default
experimentName: example_mni
trialConcurrency: 1
maxExecDuration: 1h
maxTrialNum: 1
#choice: local, remote, pai, kubeflow
trainingServicePlatform: kubeflow
searchSpacePath: search_space.json
#choice: true, false
useAnnotation: false
tuner:
#choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE
classArgs:
#choice: maximize, minimize
optimize_mode: maximize
trial:
codeDir: .
worker:
replicas: 1
command: python3 mnist.py
gpuNum: 0
cpuNum: 1
memoryMB: 8192
image: msranni/nni:latest image: msranni/nni:latest
kubeflowConfig: #The hdfs directory to store data on pai, format 'hdfs://host:port/directory'
operator: tf-operator dataDir: hdfs://10.11.12.13:9000/test
nfs: #The hdfs directory to store output data generated by NNI, format 'hdfs://host:port/directory'
server: 10.10.10.10 outputDir: hdfs://10.11.12.13:9000/test
path: /var/nfs/general paiConfig:
``` #The username to login pai
userName: test
#The password to login pai
passWord: test
#The host of restful server of pai
host: 10.10.10.10
```
kubeflow use azure storage * __kubeflow mode__
``` kubeflow with nfs storage.
authorName: default
experimentName: example_mni ```yaml
trialConcurrency: 1 authorName: default
maxExecDuration: 1h experimentName: example_mni
maxTrialNum: 1 trialConcurrency: 1
#choice: local, remote, pai, kubeflow maxExecDuration: 1h
trainingServicePlatform: kubeflow maxTrialNum: 1
searchSpacePath: search_space.json #choice: local, remote, pai, kubeflow
#choice: true, false trainingServicePlatform: kubeflow
useAnnotation: false searchSpacePath: search_space.json
#nniManagerIp: 10.10.10.10 #choice: true, false
tuner: useAnnotation: false
#choice: TPE, Random, Anneal, Evolution tuner:
builtinTunerName: TPE #choice: TPE, Random, Anneal, Evolution
classArgs: builtinTunerName: TPE
#choice: maximize, minimize classArgs:
optimize_mode: maximize #choice: maximize, minimize
assessor: optimize_mode: maximize
builtinAssessorName: Medianstop trial:
classArgs: codeDir: .
optimize_mode: maximize worker:
gpuNum: 0 replicas: 1
trial: command: python3 mnist.py
codeDir: . gpuNum: 0
worker: cpuNum: 1
replicas: 1 memoryMB: 8192
command: python3 mnist.py image: msranni/nni:latest
kubeflowConfig:
operator: tf-operator
nfs:
server: 10.10.10.10
path: /var/nfs/general
```
kubeflow with azure storage
```yaml
authorName: default
experimentName: example_mni
trialConcurrency: 1
maxExecDuration: 1h
maxTrialNum: 1
#choice: local, remote, pai, kubeflow
trainingServicePlatform: kubeflow
searchSpacePath: search_space.json
#choice: true, false
useAnnotation: false
#nniManagerIp: 10.10.10.10
tuner:
#choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE
classArgs:
#choice: maximize, minimize
optimize_mode: maximize
assessor:
builtinAssessorName: Medianstop
classArgs:
optimize_mode: maximize
gpuNum: 0 gpuNum: 0
cpuNum: 1 trial:
memoryMB: 4096 codeDir: .
image: msranni/nni:latest worker:
kubeflowConfig: replicas: 1
operator: tf-operator command: python3 mnist.py
keyVault: gpuNum: 0
vaultName: Contoso-Vault cpuNum: 1
name: AzureStorageAccountKey memoryMB: 4096
azureStorage: image: msranni/nni:latest
accountName: storage kubeflowConfig:
azureShare: share01 operator: tf-operator
``` keyVault:
vaultName: Contoso-Vault
name: AzureStorageAccountKey
azureStorage:
accountName: storage
azureShare: share01
```
docs/cifar10_examples.md
View file @ ccff6733
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
### **Goals** ### **Goals**
As we all know, the choice of model optimizer is directly affects the performance of the final matrix. The goal of this tutorial is to **tune a better performace optimizer** to train a relatively small convolutional neural network (CNN) for recognizing images. As we all know, the choice of model optimizer is directly affects the performance of the final metrics. The goal of this tutorial is to **tune a better performace optimizer** to train a relatively small convolutional neural network (CNN) for recognizing images.
In this example, we have selected the following common deep learning optimizer: In this example, we have selected the following common deep learning optimizer:
...@@ -65,12 +65,12 @@ code directory: [examples/trials/cifar10_pytorch/config_pai.yml][7] ...@@ -65,12 +65,12 @@ code directory: [examples/trials/cifar10_pytorch/config_pai.yml][7]
*The complete examples we have implemented: [examples/trials/cifar10_pytorch/][1]* *The complete examples we have implemented: [examples/trials/cifar10_pytorch/][1]*
#### Lauch the experiment #### Launch the experiment
We are ready for the experiment, let's now **run the config.yml file from your command line to start the experiment**. We are ready for the experiment, let's now **run the config.yml file from your command line to start the experiment**.
```bash ```bash
nnictl create --config nni/examples/trials/cifar10_pytorch/config.yml nnictl create --config nni/examples/trials/cifar10_pytorch/config.yml
``` ```
[1]: https://github.com/Microsoft/nni/tree/master/examples/trials/cifar10_pytorch [1]: https://github.com/Microsoft/nni/tree/master/examples/trials/cifar10_pytorch
......
docs/sdk_reference.rst
View file @ ccff6733
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
Python API Reference Python API Reference
########################### ###########################
API for trial code Trial
------------------------ ------------------------
.. autofunction:: nni.get_next_parameter .. autofunction:: nni.get_next_parameter
.. autofunction:: nni.get_current_parameter .. autofunction:: nni.get_current_parameter
...@@ -11,7 +11,7 @@ API for trial code ...@@ -11,7 +11,7 @@ API for trial code
.. autofunction:: nni.get_sequence_id .. autofunction:: nni.get_sequence_id
API for tuners Tuner
------------------------ ------------------------
.. autoclass:: nni.tuner.Tuner .. autoclass:: nni.tuner.Tuner
:members: :members:
...@@ -28,7 +28,7 @@ API for tuners ...@@ -28,7 +28,7 @@ API for tuners
.. autoclass:: nni.smac_tuner.smac_tuner.SMACTuner .. autoclass:: nni.smac_tuner.smac_tuner.SMACTuner
:members: :members:
API for assessors Assessor
------------------------ ------------------------
.. autoclass:: nni.assessor.Assessor .. autoclass:: nni.assessor.Assessor
:members: :members:
...@@ -40,6 +40,6 @@ API for assessors ...@@ -40,6 +40,6 @@ API for assessors
:members: :members:
API for Advisors Advisor
------------------------ ------------------------
.. autoclass:: nni.hyperband_advisor.hyperband_advisor.Hyperband .. autoclass:: nni.hyperband_advisor.hyperband_advisor.Hyperband
\ No newline at end of file
docs/training_services.rst
View file @ ccff6733
...@@ -4,6 +4,6 @@ Introduction to NNI Training Services ...@@ -4,6 +4,6 @@ Introduction to NNI Training Services
.. toctree:: .. toctree::
Local<tutorial_1_CR_exp_local_api> Local<tutorial_1_CR_exp_local_api>
Remote<RemoteMachineMode> Remote<RemoteMachineMode>
PAI<PAIMode> OpenPAI<PAIMode>
Kubeflow<KubeflowMode> Kubeflow<KubeflowMode>
FrameworkController Mode<FrameworkControllerMode> FrameworkController<FrameworkControllerMode>
\ No newline at end of file \ No newline at end of file
docs/tuners.rst
View file @ ccff6733
...@@ -4,7 +4,7 @@ Tuners ...@@ -4,7 +4,7 @@ Tuners
NNI provides an easy way to adopt an approach to set up parameter tuning algorithms, we call them **Tuner**. NNI provides an easy way to adopt an approach to set up parameter tuning algorithms, we call them **Tuner**.
Tuner receives the result from `Trial` as a matrix to evaluate the performance of a specific parameters/architecture configures. And tuner sends next hyper-parameter or architecture configure to Trial. Tuner receives metrics from `Trial` to evaluate the performance of a specific parameters/architecture configures. And tuner sends next hyper-parameter or architecture configure to Trial.
In NNI, we support two approaches to set the tuner: first is directly use builtin tuner provided by nni sdk, second is customize a tuner file by yourself. We also have Advisor that combines the functinality of Tuner & Assessor. In NNI, we support two approaches to set the tuner: first is directly use builtin tuner provided by nni sdk, second is customize a tuner file by yourself. We also have Advisor that combines the functinality of Tuner & Assessor.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment