Skip to content

GitLab

Unverified Commit ccff6733 authored by Chi Song's avatar Chi Song Committed by GitHub
Browse files

Doc fix: rewording and typos (#685)

parent 4539b4d3
Show whitespace changes
Inline Side-by-side
Showing with 508 additions and 502 deletions
docs/Builtin_Assessors.md
View file @ ccff6733
...@@ -4,10 +4,8 @@ NNI provides state-of-the-art tuning algorithm in our builtin-assessors and make ...@@ -4,10 +4,8 @@ NNI provides state-of-the-art tuning algorithm in our builtin-assessors and make
|Assessor|Brief Introduction of Algorithm| |Assessor|Brief Introduction of Algorithm|
|---|---| |---|---|
|**Medianstop**<br>[(Usage)](#MedianStop)|Medianstop is a simple early stopping rule mentioned in the [paper](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/46180.pdf). It stops a pending trial X at step S if the trial’s best objective value by step S is strictly worse than the median value of the running averages of all completed trials’ objectives reported up to step S.| |**Medianstop** [(Usage)](#MedianStop)|Medianstop is a simple early stopping rule mentioned in the [paper](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/46180.pdf). It stops a pending trial X at step S if the trial’s best objective value by step S is strictly worse than the median value of the running averages of all completed trials’ objectives reported up to step S.|
|[Curvefitting](https://github.com/Microsoft/nni/blob/master/src/sdk/pynni/nni/curvefitting_assessor/README.md)<br>[(Usage)](#Curvefitting)|Curve Fitting Assessor is a LPA(learning, predicting, assessing) algorithm. It stops a pending trial X at step S if the prediction of final epoch's performance worse than the best final performance in the trial history. In this algorithm, we use 12 curves to fit the accuracy curve| |[Curvefitting](https://github.com/Microsoft/nni/blob/master/src/sdk/pynni/nni/curvefitting_assessor/README.md) [(Usage)](#Curvefitting)|Curve Fitting Assessor is a LPA(learning, predicting, assessing) algorithm. It stops a pending trial X at step S if the prediction of final epoch's performance worse than the best final performance in the trial history. In this algorithm, we use 12 curves to fit the accuracy curve|
<br>
## Usage of Builtin Assessors ## Usage of Builtin Assessors
......
docs/Builtin_Tuner.md
View file @ ccff6733
...@@ -4,16 +4,16 @@ NNI provides state-of-the-art tuning algorithm as our builtin-tuners and makes t ...@@ -4,16 +4,16 @@ NNI provides state-of-the-art tuning algorithm as our builtin-tuners and makes t
|Tuner|Brief Introduction of Algorithm| |Tuner|Brief Introduction of Algorithm|
|---|---| |---|---|
|**TPE**<br>[(Usage)](#TPE)|The Tree-structured Parzen Estimator (TPE) is a sequential model-based optimization (SMBO) approach. SMBO methods sequentially construct models to approximate the performance of hyperparameters based on historical measurements, and then subsequently choose new hyperparameters to test based on this model.| |**TPE** [(Usage)](#TPE)|The Tree-structured Parzen Estimator (TPE) is a sequential model-based optimization (SMBO) approach. SMBO methods sequentially construct models to approximate the performance of hyperparameters based on historical measurements, and then subsequently choose new hyperparameters to test based on this model.|
|**Random Search**<br>[(Usage)](#Random)|In Random Search for Hyper-Parameter Optimization show that Random Search might be surprisingly simple and effective. We suggest that we could use Random Search as the baseline when we have no knowledge about the prior distribution of hyper-parameters.| |**Random Search** [(Usage)](#Random)|In Random Search for Hyper-Parameter Optimization show that Random Search might be surprisingly simple and effective. We suggest that we could use Random Search as the baseline when we have no knowledge about the prior distribution of hyper-parameters.|
|**Anneal**<br>[(Usage)](#Anneal)|This simple annealing algorithm begins by sampling from the prior, but tends over time to sample from points closer and closer to the best ones observed. This algorithm is a simple variation on the random search that leverages smoothness in the response surface. The annealing rate is not adaptive.| |**Anneal** [(Usage)](#Anneal)|This simple annealing algorithm begins by sampling from the prior, but tends over time to sample from points closer and closer to the best ones observed. This algorithm is a simple variation on the random search that leverages smoothness in the response surface. The annealing rate is not adaptive.|
|**Naive Evolution**<br>[(Usage)](#Evolution)|Naive Evolution comes from Large-Scale Evolution of Image Classifiers. It randomly initializes a population-based on search space. For each generation, it chooses better ones and does some mutation (e.g., change a hyperparameter, add/remove one layer) on them to get the next generation. Naive Evolution requires many trials to works, but it's very simple and easy to expand new features.| |**Naive Evolution** [(Usage)](#Evolution)|Naive Evolution comes from Large-Scale Evolution of Image Classifiers. It randomly initializes a population-based on search space. For each generation, it chooses better ones and does some mutation (e.g., change a hyperparameter, add/remove one layer) on them to get the next generation. Naive Evolution requires many trials to works, but it's very simple and easy to expand new features.|
|**SMAC**<br>[(Usage)](#SMAC)|SMAC is based on Sequential Model-Based Optimization (SMBO). It adapts the most prominent previously used model class (Gaussian stochastic process models) and introduces the model class of random forests to SMBO, in order to handle categorical parameters. The SMAC supported by nni is a wrapper on the SMAC3 Github repo.| |**SMAC** [(Usage)](#SMAC)|SMAC is based on Sequential Model-Based Optimization (SMBO). It adapts the most prominent previously used model class (Gaussian stochastic process models) and introduces the model class of random forests to SMBO, in order to handle categorical parameters. The SMAC supported by nni is a wrapper on the SMAC3 Github repo.|
|**Batch tuner**<br>[(Usage)](#Batch)|Batch tuner allows users to simply provide several configurations (i.e., choices of hyper-parameters) for their trial code. After finishing all the configurations, the experiment is done. Batch tuner only supports the type choice in search space spec.| |**Batch tuner** [(Usage)](#Batch)|Batch tuner allows users to simply provide several configurations (i.e., choices of hyper-parameters) for their trial code. After finishing all the configurations, the experiment is done. Batch tuner only supports the type choice in search space spec.|
|**Grid Search**<br>[(Usage)](#GridSearch)|Grid Search performs an exhaustive searching through a manually specified subset of the hyperparameter space defined in the searchspace file. Note that the only acceptable types of search space are choice, quniform, qloguniform. The number q in quniform and qloguniform has special meaning (different from the spec in search space spec). It means the number of values that will be sampled evenly from the range low and high.| |**Grid Search** [(Usage)](#GridSearch)|Grid Search performs an exhaustive searching through a manually specified subset of the hyperparameter space defined in the searchspace file. Note that the only acceptable types of search space are choice, quniform, qloguniform. The number q in quniform and qloguniform has special meaning (different from the spec in search space spec). It means the number of values that will be sampled evenly from the range low and high.|
|[Hyperband](https://github.com/Microsoft/nni/tree/master/src/sdk/pynni/nni/hyperband_advisor)<br>[(Usage)](#Hyperband)|Hyperband tries to use the limited resource to explore as many configurations as possible, and finds out the promising ones to get the final result. The basic idea is generating many configurations and to run them for the small number of STEPs to find out promising one, then further training those promising ones to select several more promising one.| |[Hyperband](https://github.com/Microsoft/nni/tree/master/src/sdk/pynni/nni/hyperband_advisor) [(Usage)](#Hyperband)|Hyperband tries to use the limited resource to explore as many configurations as possible, and finds out the promising ones to get the final result. The basic idea is generating many configurations and to run them for the small number of STEPs to find out promising one, then further training those promising ones to select several more promising one.|
|[Network Morphism](https://github.com/Microsoft/nni/blob/master/src/sdk/pynni/nni/networkmorphism_tuner/README.md)<br>[(Usage)](#NetworkMorphism)|Network Morphism provides functions to automatically search for architecture of deep learning models. Every child network inherits the knowledge from its parent network and morphs into diverse types of networks, including changes of depth, width, and skip-connection. Next, it estimates the value of a child network using the historic architecture and metric pairs. Then it selects the most promising one to train.| |[Network Morphism](https://github.com/Microsoft/nni/blob/master/src/sdk/pynni/nni/networkmorphism_tuner/README.md) [(Usage)](#NetworkMorphism)|Network Morphism provides functions to automatically search for architecture of deep learning models. Every child network inherits the knowledge from its parent network and morphs into diverse types of networks, including changes of depth, width, and skip-connection. Next, it estimates the value of a child network using the historic architecture and metric pairs. Then it selects the most promising one to train.|
|**Metis Tuner**<br>[(Usage)](#MetisTuner)|Metis offers the following benefits when it comes to tuning parameters: While most tools only predict the optimal configuration, Metis gives you two outputs: (a) current prediction of optimal configuration, and (b) suggestion for the next trial. No more guesswork. While most tools assume training datasets do not have noisy data, Metis actually tells you if you need to re-sample a particular hyper-parameter.| |**Metis Tuner** [(Usage)](#MetisTuner)|Metis offers the following benefits when it comes to tuning parameters: While most tools only predict the optimal configuration, Metis gives you two outputs: (a) current prediction of optimal configuration, and (b) suggestion for the next trial. No more guesswork. While most tools assume training datasets do not have noisy data, Metis actually tells you if you need to re-sample a particular hyper-parameter.|
<br> <br>
......
docs/Customize_Assessor.md
View file @ ccff6733
# Customize Assessor # Customize Assessor
NNI also support building an assessor by yourself to adjust your tuning demand. NNI supports to build an assessor by yourself for tuning demand.
If you want to implement a customized Assessor, there are three things for you to do: If you want to implement a customized Assessor, there are three things to do:
1) Inherit an assessor of a base Assessor class 1. Inherit the base Assessor class
2) Implement assess_trial function 1. Implement assess_trial function
3) Configure your customized Assessor in experiment YAML config file 1. Configure your customized Assessor in experiment YAML config file
**1. Inherit an assessor of a base Assessor class** **1. Inherit the base Assessor class**
```python ```python
from nni.assessor import Assessor from nni.assessor import Assessor
...@@ -19,6 +19,7 @@ class CustomizedAssessor(Assessor): ...@@ -19,6 +19,7 @@ class CustomizedAssessor(Assessor):
``` ```
**2. Implement assess trial function** **2. Implement assess trial function**
```python ```python
from nni.assessor import Assessor, AssessResult from nni.assessor import Assessor, AssessResult
...@@ -41,7 +42,6 @@ class CustomizedAssessor(Assessor): ...@@ -41,7 +42,6 @@ class CustomizedAssessor(Assessor):
NNI needs to locate your customized Assessor class and instantiate the class, so you need to specify the location of the customized Assessor class and pass literal values as parameters to the \_\_init__ constructor. NNI needs to locate your customized Assessor class and instantiate the class, so you need to specify the location of the customized Assessor class and pass literal values as parameters to the \_\_init__ constructor.
```yaml ```yaml
assessor: assessor:
codeDir: /home/abc/myassessor codeDir: /home/abc/myassessor
classFileName: my_customized_assessor.py classFileName: my_customized_assessor.py
...@@ -50,7 +50,6 @@ assessor: ...@@ -50,7 +50,6 @@ assessor:
# can be specified in this optional classArgs field, for example # can be specified in this optional classArgs field, for example
classArgs: classArgs:
arg1: value1 arg1: value1
``` ```
Please noted in **2**. The object `trial_history` are exact the object that Trial send to Assessor by using SDK `report_intermediate_result` function. Please noted in **2**. The object `trial_history` are exact the object that Trial send to Assessor by using SDK `report_intermediate_result` function.
......
docs/Customize_Tuner.md
View file @ ccff6733
...@@ -2,17 +2,17 @@ ...@@ -2,17 +2,17 @@
## Customize Tuner ## Customize Tuner
NNI provides state-of-the-art tuning algorithm in our builtin-tuners. We also support building a tuner by yourself to adjust your tuning demand. NNI provides state-of-the-art tuning algorithm in builtin-tuners. NNI supports to build a tuner by yourself for tuning demand.
If you want to implement and use your own tuning algorithm, you can implement a customized Tuner, there are three things for you to do: If you want to implement your own tuning algorithm, you can implement a customized Tuner, there are three things to do:
1) Inherit a tuner of a base Tuner class 1. Inherit the base Tuner class
2) Implement receive_trial_result and generate_parameter function 1. Implement receive_trial_result and generate_parameter function
3) Configure your customized tuner in experiment YAML config file 1. Configure your customized tuner in experiment YAML config file
Here is an example: Here is an example:
**1. Inherit a tuner of a base Tuner class** **1. Inherit the base Tuner class**
```python ```python
from nni.tuner import Tuner from nni.tuner import Tuner
...@@ -33,10 +33,10 @@ class CustomizedTuner(Tuner): ...@@ -33,10 +33,10 @@ class CustomizedTuner(Tuner):
def receive_trial_result(self, parameter_id, parameters, value): def receive_trial_result(self, parameter_id, parameters, value):
''' '''
Record an observation of the objective function and Train Receive trial's final result.
parameter_id: int parameter_id: int
parameters: object created by 'generate_parameters()' parameters: object created by 'generate_parameters()'
value: final metrics of the trial, including default matrix value: final metrics of the trial, including default metric
''' '''
# your code implements here. # your code implements here.
... ...
...@@ -59,7 +59,6 @@ For example: ...@@ -59,7 +59,6 @@ For example:
If the you implement the `generate_parameters` like this: If the you implement the `generate_parameters` like this:
```python ```python
def generate_parameters(self, parameter_id): def generate_parameters(self, parameter_id):
''' '''
Returns a set of trial (hyper-)parameters, as a serializable object Returns a set of trial (hyper-)parameters, as a serializable object
...@@ -72,21 +71,17 @@ def generate_parameters(self, parameter_id): ...@@ -72,21 +71,17 @@ def generate_parameters(self, parameter_id):
It means your Tuner will always generate parameters `{"dropout": 0.3, "learning_rate": 0.4}`. Then Trial will receive `{"dropout": 0.3, "learning_rate": 0.4}` by calling API `nni.get_next_parameter()`. Once the trial ends with a result (normally some kind of metrics), it can send the result to Tuner by calling API `nni.report_final_result()`, for example `nni.report_final_result(0.93)`. Then your Tuner's `receive_trial_result` function will receied the result like： It means your Tuner will always generate parameters `{"dropout": 0.3, "learning_rate": 0.4}`. Then Trial will receive `{"dropout": 0.3, "learning_rate": 0.4}` by calling API `nni.get_next_parameter()`. Once the trial ends with a result (normally some kind of metrics), it can send the result to Tuner by calling API `nni.report_final_result()`, for example `nni.report_final_result(0.93)`. Then your Tuner's `receive_trial_result` function will receied the result like：
``` ```python
parameter_id = 82347 parameter_id = 82347
parameters = {"dropout": 0.3, "learning_rate": 0.4} parameters = {"dropout": 0.3, "learning_rate": 0.4}
value = 0.93 value = 0.93
``` ```
**Note that** if you want to access a file (e.g., `data.txt`) in the directory of your own tuner, you cannot use `open('data.txt', 'r')`. Instead, you should use the following: **Note that** if you want to access a file (e.g., `data.txt`) in the directory of your own tuner, you cannot use `open('data.txt', 'r')`. Instead, you should use the following:
``` ```python
_pwd = os.path.dirname(__file__) _pwd = os.path.dirname(__file__)
_fd = open(os.path.join(_pwd, 'data.txt'), 'r') _fd = open(os.path.join(_pwd, 'data.txt'), 'r')
``` ```
This is because your tuner is not executed in the directory of your tuner (i.e., `pwd` is not the directory of your own tuner). This is because your tuner is not executed in the directory of your tuner (i.e., `pwd` is not the directory of your own tuner).
...@@ -96,7 +91,6 @@ This is because your tuner is not executed in the directory of your tuner (i.e., ...@@ -96,7 +91,6 @@ This is because your tuner is not executed in the directory of your tuner (i.e.,
NNI needs to locate your customized tuner class and instantiate the class, so you need to specify the location of the customized tuner class and pass literal values as parameters to the \_\_init__ constructor. NNI needs to locate your customized tuner class and instantiate the class, so you need to specify the location of the customized tuner class and pass literal values as parameters to the \_\_init__ constructor.
```yaml ```yaml
tuner: tuner:
codeDir: /home/abc/mytuner codeDir: /home/abc/mytuner
classFileName: my_customized_tuner.py classFileName: my_customized_tuner.py
......
docs/ExperimentConfig.md
View file @ ccff6733
...@@ -4,15 +4,16 @@ A config file is needed when create an experiment, the path of the config file i ...@@ -4,15 +4,16 @@ A config file is needed when create an experiment, the path of the config file i
The config file is written in YAML format, and need to be written correctly. The config file is written in YAML format, and need to be written correctly.
This document describes the rule to write config file, and will provide some examples and templates. This document describes the rule to write config file, and will provide some examples and templates.
- [Template](#Template) (the templates of an config file) * [Template](#Template) (the templates of an config file)
- [Configuration spec](#Configuration) (the configuration specification of every attribute in config file) * [Configuration spec](#Configuration) (the configuration specification of every attribute in config file)
- [Examples](#Examples) (the examples of config file) * [Examples](#Examples) (the examples of config file)
<a name="Template"></a> <a name="Template"></a>
## Template ## Template
* __light weight(without Annotation and Assessor)__ * __light weight(without Annotation and Assessor)__
``` ```yaml
authorName: authorName:
experimentName: experimentName:
trialConcurrency: trialConcurrency:
...@@ -44,7 +45,7 @@ machineList: ...@@ -44,7 +45,7 @@ machineList:
* __Use Assessor__ * __Use Assessor__
``` ```yaml
authorName: authorName:
experimentName: experimentName:
trialConcurrency: trialConcurrency:
...@@ -83,7 +84,7 @@ machineList: ...@@ -83,7 +84,7 @@ machineList:
* __Use Annotation__ * __Use Annotation__
``` ```yaml
authorName: authorName:
experimentName: experimentName:
trialConcurrency: trialConcurrency:
...@@ -118,8 +119,10 @@ machineList: ...@@ -118,8 +119,10 @@ machineList:
username: username:
passwd: passwd:
``` ```
<a name="Configuration"></a> <a name="Configuration"></a>
## Configuration spec ## Configuration spec
* __authorName__ * __authorName__
* Description * Description
...@@ -158,10 +161,8 @@ machineList: ...@@ -158,10 +161,8 @@ machineList:
* __local__ run an experiment on local ubuntu machine. * __local__ run an experiment on local ubuntu machine.
* __remote__ submit trial jobs to remote ubuntu machines, and __machineList__ field should be filed in order to set up SSH connection to remote machine. * __remote__ submit trial jobs to remote ubuntu machines, and __machineList__ field should be filed in order to set up SSH connection to remote machine.
* __pai__ submit trial jobs to [OpenPai](https://github.com/Microsoft/pai) of Microsoft. For more details of pai configuration, please reference [PAIMOdeDoc](./PAIMode.md) * __pai__ submit trial jobs to [OpenPai](https://github.com/Microsoft/pai) of Microsoft. For more details of pai configuration, please reference [PAIMOdeDoc](./PAIMode.md)
* __kubeflow__ submit trial jobs to [kubeflow](https://www.kubeflow.org/docs/about/kubeflow/), NNI support kubeflow based on normal kubernetes and [azure kubernetes](https://azure.microsoft.com/en-us/services/kubernetes-service/). * __kubeflow__ submit trial jobs to [kubeflow](https://www.kubeflow.org/docs/about/kubeflow/), NNI support kubeflow based on normal kubernetes and [azure kubernetes](https://azure.microsoft.com/en-us/services/kubernetes-service/).
...@@ -172,6 +173,7 @@ machineList: ...@@ -172,6 +173,7 @@ machineList:
__searchSpacePath__ specifies the path of search space file, which should be a valid path in the local linux machine. __searchSpacePath__ specifies the path of search space file, which should be a valid path in the local linux machine.
Note: if set useAnnotation=True, the searchSpacePath field should be removed. Note: if set useAnnotation=True, the searchSpacePath field should be removed.
* __useAnnotation__ * __useAnnotation__
* Description * Description
...@@ -196,7 +198,6 @@ machineList: ...@@ -196,7 +198,6 @@ machineList:
__logLevel__ sets log level for the experiment, available log levels are: `trace, debug, info, warning, error, fatal`. The default value is `info`. __logLevel__ sets log level for the experiment, available log levels are: `trace, debug, info, warning, error, fatal`. The default value is `info`.
* __tuner__ * __tuner__
* Description * Description
...@@ -239,48 +240,63 @@ machineList: ...@@ -239,48 +240,63 @@ machineList:
* __classArgs__ * __classArgs__
__classArgs__ specifies the arguments of assessor algorithm __classArgs__ specifies the arguments of assessor algorithm
* __codeDir__, __classFileName__, __className__ and __classArgs__ * __codeDir__, __classFileName__, __className__ and __classArgs__
* __codeDir__ * __codeDir__
__codeDir__ specifies the directory of assessor code. __codeDir__ specifies the directory of assessor code.
* __classFileName__ * __classFileName__
__classFileName__ specifies the name of assessor file. __classFileName__ specifies the name of assessor file.
* __className__ * __className__
__className__ specifies the name of assessor class. __className__ specifies the name of assessor class.
* __classArgs__ * __classArgs__
__classArgs__ specifies the arguments of assessor algorithm. __classArgs__ specifies the arguments of assessor algorithm.
* __gpuNum__ * __gpuNum__
__gpuNum__ specifies the gpu number to run the assessor process. The value of this field should be a positive number. __gpuNum__ specifies the gpu number to run the assessor process. The value of this field should be a positive number.
Note: users' could only specify one way to set assessor, for example,set {assessorName, optimizationMode} or {assessorCommand, assessorCwd}, and users could not set them both.If users do not want to use assessor, assessor fileld should leave to empty. Note: users' could only specify one way to set assessor, for example,set {assessorName, optimizationMode} or {assessorCommand, assessorCwd}, and users could not set them both.If users do not want to use assessor, assessor fileld should leave to empty.
* __trial(local, remote)__ * __trial(local, remote)__
* __command__ * __command__
__command__ specifies the command to run trial process. __command__ specifies the command to run trial process.
* __codeDir__ * __codeDir__
__codeDir__ specifies the directory of your own trial file. __codeDir__ specifies the directory of your own trial file.
* __gpuNum__ * __gpuNum__
__gpuNum__ specifies the num of gpu to run the trial process. Default value is 0. __gpuNum__ specifies the num of gpu to run the trial process. Default value is 0.
* __trial(pai)__ * __trial(pai)__
* __command__ * __command__
__command__ specifies the command to run trial process. __command__ specifies the command to run trial process.
* __codeDir__ * __codeDir__
__codeDir__ specifies the directory of the own trial file. __codeDir__ specifies the directory of the own trial file.
* __gpuNum__ * __gpuNum__
__gpuNum__ specifies the num of gpu to run the trial process. Default value is 0. __gpuNum__ specifies the num of gpu to run the trial process. Default value is 0.
* __cpuNum__ * __cpuNum__
__cpuNum__ is the cpu number of cpu to be used in pai container. __cpuNum__ is the cpu number of cpu to be used in pai container.
* __memoryMB__ * __memoryMB__
__memoryMB__ set the momory size to be used in pai's container. __memoryMB__ set the momory size to be used in pai's container.
...@@ -297,8 +313,6 @@ machineList: ...@@ -297,8 +313,6 @@ machineList:
__outputDir__ is the output directory in hdfs to be used in pai, the stdout and stderr files are stored in the directory after job finished. __outputDir__ is the output directory in hdfs to be used in pai, the stdout and stderr files are stored in the directory after job finished.
* __trial(kubeflow)__ * __trial(kubeflow)__
* __codeDir__ * __codeDir__
...@@ -308,6 +322,7 @@ machineList: ...@@ -308,6 +322,7 @@ machineList:
* __ps(optional)__ * __ps(optional)__
__ps__ is the configuration for kubeflow's tensorflow-operator. __ps__ is the configuration for kubeflow's tensorflow-operator.
* __replicas__ * __replicas__
__replicas__ is the replica number of __ps__ role. __replicas__ is the replica number of __ps__ role.
...@@ -335,6 +350,7 @@ machineList: ...@@ -335,6 +350,7 @@ machineList:
* __worker__ * __worker__
__worker__ is the configuration for kubeflow's tensorflow-operator. __worker__ is the configuration for kubeflow's tensorflow-operator.
* __replicas__ * __replicas__
__replicas__ is the replica number of __worker__ role. __replicas__ is the replica number of __worker__ role.
...@@ -359,14 +375,14 @@ machineList: ...@@ -359,14 +375,14 @@ machineList:
__image__ set the image to be used in __worker__. __image__ set the image to be used in __worker__.
* __machineList__ * __machineList__
__machineList__ should be set if users set __trainingServicePlatform__=remote, or it could be empty. __machineList__ should be set if __trainingServicePlatform__ is set to remote, or it should be empty.
* __ip__ * __ip__
__ip__ is the ip address of remote machine. __ip__ is the ip address of remote machine.
* __port__ * __port__
__port__ is the ssh port to be used to connect machine. __port__ is the ssh port to be used to connect machine.
...@@ -411,11 +427,11 @@ machineList: ...@@ -411,11 +427,11 @@ machineList:
* __vaultName__ * __vaultName__
__vaultName__ is the value of ```--vault-name``` used in az command. __vaultName__ is the value of `--vault-name` used in az command.
* __name__ * __name__
__name__ is the value of ```--name``` used in az command. __name__ is the value of `--name` used in az command.
* __azureStorage__ * __azureStorage__
...@@ -443,84 +459,83 @@ machineList: ...@@ -443,84 +459,83 @@ machineList:
__host__ is the host of pai. __host__ is the host of pai.
<a name="Examples"></a> <a name="Examples"></a>
## Examples ## Examples
* __local mode__ * __local mode__
If users want to run trial jobs in local machine, and use annotation to generate search space, could use the following config: If users want to run trial jobs in local machine, and use annotation to generate search space, could use the following config:
``` ```yaml
authorName: test authorName: test
experimentName: test_experiment experimentName: test_experiment
trialConcurrency: 3 trialConcurrency: 3
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 10 maxTrialNum: 10
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: local trainingServicePlatform: local
#choice: true, false #choice: true, false
useAnnotation: true useAnnotation: true
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
trial: trial:
command: python3 mnist.py command: python3 mnist.py
codeDir: /nni/mnist codeDir: /nni/mnist
gpuNum: 0 gpuNum: 0
``` ```
Could add assessor configuration in config file if set assessor. You can add assessor configuration.
``` ```yaml
authorName: test authorName: test
experimentName: test_experiment experimentName: test_experiment
trialConcurrency: 3 trialConcurrency: 3
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 10 maxTrialNum: 10
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: local trainingServicePlatform: local
searchSpacePath: /nni/search_space.json searchSpacePath: /nni/search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
assessor: assessor:
#choice: Medianstop #choice: Medianstop
builtinAssessorName: Medianstop builtinAssessorName: Medianstop
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
trial: trial:
command: python3 mnist.py command: python3 mnist.py
codeDir: /nni/mnist codeDir: /nni/mnist
gpuNum: 0 gpuNum: 0
``` ```
Or you could specify your own tuner and assessor file as following: Or you could specify your own tuner and assessor file as following,
``` ```yaml
authorName: test authorName: test
experimentName: test_experiment experimentName: test_experiment
trialConcurrency: 3 trialConcurrency: 3
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 10 maxTrialNum: 10
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: local trainingServicePlatform: local
searchSpacePath: /nni/search_space.json searchSpacePath: /nni/search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
tuner: tuner:
codeDir: /nni/tuner codeDir: /nni/tuner
classFileName: mytuner.py classFileName: mytuner.py
className: MyTuner className: MyTuner
...@@ -528,7 +543,7 @@ tuner: ...@@ -528,7 +543,7 @@ tuner:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
assessor: assessor:
codeDir: /nni/assessor codeDir: /nni/assessor
classFileName: myassessor.py classFileName: myassessor.py
className: MyAssessor className: MyAssessor
...@@ -536,40 +551,40 @@ assessor: ...@@ -536,40 +551,40 @@ assessor:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
trial: trial:
command: python3 mnist.py command: python3 mnist.py
codeDir: /nni/mnist codeDir: /nni/mnist
gpuNum: 0 gpuNum: 0
``` ```
* __remote mode__ * __remote mode__
If run trial jobs in remote machine, users could specify the remote mahcine information as fllowing format: If run trial jobs in remote machine, users could specify the remote mahcine information as fllowing format:
``` ```yaml
authorName: test authorName: test
experimentName: test_experiment experimentName: test_experiment
trialConcurrency: 3 trialConcurrency: 3
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 10 maxTrialNum: 10
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: remote trainingServicePlatform: remote
searchSpacePath: /nni/search_space.json searchSpacePath: /nni/search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
trial: trial:
command: python3 mnist.py command: python3 mnist.py
codeDir: /nni/mnist codeDir: /nni/mnist
gpuNum: 0 gpuNum: 0
#machineList can be empty if the platform is local #machineList can be empty if the platform is local
machineList: machineList:
- ip: 10.10.10.10 - ip: 10.10.10.10
port: 22 port: 22
username: test username: test
...@@ -583,29 +598,29 @@ machineList: ...@@ -583,29 +598,29 @@ machineList:
username: test username: test
sshKeyPath: /nni/sshkey sshKeyPath: /nni/sshkey
passphrase: qwert passphrase: qwert
``` ```
* __pai mode__ * __pai mode__
``` ```yaml
authorName: test authorName: test
experimentName: nni_test1 experimentName: nni_test1
trialConcurrency: 1 trialConcurrency: 1
maxExecDuration:500h maxExecDuration:500h
maxTrialNum: 1 maxTrialNum: 1
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: pai trainingServicePlatform: pai
searchSpacePath: search_space.json searchSpacePath: search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution, BatchTuner #choice: TPE, Random, Anneal, Evolution, BatchTuner
#SMAC (SMAC should be installed through nnictl) #SMAC (SMAC should be installed through nnictl)
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
trial: trial:
command: python3 main.py command: python3 main.py
codeDir: . codeDir: .
gpuNum: 4 gpuNum: 4
...@@ -617,37 +632,37 @@ trial: ...@@ -617,37 +632,37 @@ trial:
dataDir: hdfs://10.11.12.13:9000/test dataDir: hdfs://10.11.12.13:9000/test
#The hdfs directory to store output data generated by NNI, format 'hdfs://host:port/directory' #The hdfs directory to store output data generated by NNI, format 'hdfs://host:port/directory'
outputDir: hdfs://10.11.12.13:9000/test outputDir: hdfs://10.11.12.13:9000/test
paiConfig: paiConfig:
#The username to login pai #The username to login pai
userName: test userName: test
#The password to login pai #The password to login pai
passWord: test passWord: test
#The host of restful server of pai #The host of restful server of pai
host: 10.10.10.10 host: 10.10.10.10
``` ```
* __kubeflow mode__ * __kubeflow mode__
kubeflow use nfs as storage. kubeflow with nfs storage.
``` ```yaml
authorName: default authorName: default
experimentName: example_mni experimentName: example_mni
trialConcurrency: 1 trialConcurrency: 1
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 1 maxTrialNum: 1
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: kubeflow trainingServicePlatform: kubeflow
searchSpacePath: search_space.json searchSpacePath: search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
trial: trial:
codeDir: . codeDir: .
worker: worker:
replicas: 1 replicas: 1
...@@ -656,39 +671,39 @@ trial: ...@@ -656,39 +671,39 @@ trial:
cpuNum: 1 cpuNum: 1
memoryMB: 8192 memoryMB: 8192
image: msranni/nni:latest image: msranni/nni:latest
kubeflowConfig: kubeflowConfig:
operator: tf-operator operator: tf-operator
nfs: nfs:
server: 10.10.10.10 server: 10.10.10.10
path: /var/nfs/general path: /var/nfs/general
``` ```
kubeflow use azure storage kubeflow with azure storage
``` ```yaml
authorName: default authorName: default
experimentName: example_mni experimentName: example_mni
trialConcurrency: 1 trialConcurrency: 1
maxExecDuration: 1h maxExecDuration: 1h
maxTrialNum: 1 maxTrialNum: 1
#choice: local, remote, pai, kubeflow #choice: local, remote, pai, kubeflow
trainingServicePlatform: kubeflow trainingServicePlatform: kubeflow
searchSpacePath: search_space.json searchSpacePath: search_space.json
#choice: true, false #choice: true, false
useAnnotation: false useAnnotation: false
#nniManagerIp: 10.10.10.10 #nniManagerIp: 10.10.10.10
tuner: tuner:
#choice: TPE, Random, Anneal, Evolution #choice: TPE, Random, Anneal, Evolution
builtinTunerName: TPE builtinTunerName: TPE
classArgs: classArgs:
#choice: maximize, minimize #choice: maximize, minimize
optimize_mode: maximize optimize_mode: maximize
assessor: assessor:
builtinAssessorName: Medianstop builtinAssessorName: Medianstop
classArgs: classArgs:
optimize_mode: maximize optimize_mode: maximize
gpuNum: 0 gpuNum: 0
trial: trial:
codeDir: . codeDir: .
worker: worker:
replicas: 1 replicas: 1
...@@ -697,7 +712,7 @@ trial: ...@@ -697,7 +712,7 @@ trial:
cpuNum: 1 cpuNum: 1
memoryMB: 4096 memoryMB: 4096
image: msranni/nni:latest image: msranni/nni:latest
kubeflowConfig: kubeflowConfig:
operator: tf-operator operator: tf-operator
keyVault: keyVault:
vaultName: Contoso-Vault vaultName: Contoso-Vault
...@@ -705,4 +720,4 @@ kubeflowConfig: ...@@ -705,4 +720,4 @@ kubeflowConfig:
azureStorage: azureStorage:
accountName: storage accountName: storage
azureShare: share01 azureShare: share01
``` ```
docs/cifar10_examples.md
View file @ ccff6733
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
### **Goals** ### **Goals**
As we all know, the choice of model optimizer is directly affects the performance of the final matrix. The goal of this tutorial is to **tune a better performace optimizer** to train a relatively small convolutional neural network (CNN) for recognizing images. As we all know, the choice of model optimizer is directly affects the performance of the final metrics. The goal of this tutorial is to **tune a better performace optimizer** to train a relatively small convolutional neural network (CNN) for recognizing images.
In this example, we have selected the following common deep learning optimizer: In this example, we have selected the following common deep learning optimizer:
...@@ -65,12 +65,12 @@ code directory: [examples/trials/cifar10_pytorch/config_pai.yml][7] ...@@ -65,12 +65,12 @@ code directory: [examples/trials/cifar10_pytorch/config_pai.yml][7]
*The complete examples we have implemented: [examples/trials/cifar10_pytorch/][1]* *The complete examples we have implemented: [examples/trials/cifar10_pytorch/][1]*
#### Lauch the experiment #### Launch the experiment
We are ready for the experiment, let's now **run the config.yml file from your command line to start the experiment**. We are ready for the experiment, let's now **run the config.yml file from your command line to start the experiment**.
```bash ```bash
nnictl create --config nni/examples/trials/cifar10_pytorch/config.yml nnictl create --config nni/examples/trials/cifar10_pytorch/config.yml
``` ```
[1]: https://github.com/Microsoft/nni/tree/master/examples/trials/cifar10_pytorch [1]: https://github.com/Microsoft/nni/tree/master/examples/trials/cifar10_pytorch
......
docs/sdk_reference.rst
View file @ ccff6733
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
Python API Reference Python API Reference
########################### ###########################
API for trial code Trial
------------------------ ------------------------
.. autofunction:: nni.get_next_parameter .. autofunction:: nni.get_next_parameter
.. autofunction:: nni.get_current_parameter .. autofunction:: nni.get_current_parameter
...@@ -11,7 +11,7 @@ API for trial code ...@@ -11,7 +11,7 @@ API for trial code
.. autofunction:: nni.get_sequence_id .. autofunction:: nni.get_sequence_id
API for tuners Tuner
------------------------ ------------------------
.. autoclass:: nni.tuner.Tuner .. autoclass:: nni.tuner.Tuner
:members: :members:
...@@ -28,7 +28,7 @@ API for tuners ...@@ -28,7 +28,7 @@ API for tuners
.. autoclass:: nni.smac_tuner.smac_tuner.SMACTuner .. autoclass:: nni.smac_tuner.smac_tuner.SMACTuner
:members: :members:
API for assessors Assessor
------------------------ ------------------------
.. autoclass:: nni.assessor.Assessor .. autoclass:: nni.assessor.Assessor
:members: :members:
...@@ -40,6 +40,6 @@ API for assessors ...@@ -40,6 +40,6 @@ API for assessors
:members: :members:
API for Advisors Advisor
------------------------ ------------------------
.. autoclass:: nni.hyperband_advisor.hyperband_advisor.Hyperband .. autoclass:: nni.hyperband_advisor.hyperband_advisor.Hyperband
\ No newline at end of file
docs/training_services.rst
View file @ ccff6733
...@@ -4,6 +4,6 @@ Introduction to NNI Training Services ...@@ -4,6 +4,6 @@ Introduction to NNI Training Services
.. toctree:: .. toctree::
Local<tutorial_1_CR_exp_local_api> Local<tutorial_1_CR_exp_local_api>
Remote<RemoteMachineMode> Remote<RemoteMachineMode>
PAI<PAIMode> OpenPAI<PAIMode>
Kubeflow<KubeflowMode> Kubeflow<KubeflowMode>
FrameworkController Mode<FrameworkControllerMode> FrameworkController<FrameworkControllerMode>
\ No newline at end of file \ No newline at end of file
docs/tuners.rst
View file @ ccff6733
...@@ -4,7 +4,7 @@ Tuners ...@@ -4,7 +4,7 @@ Tuners
NNI provides an easy way to adopt an approach to set up parameter tuning algorithms, we call them **Tuner**. NNI provides an easy way to adopt an approach to set up parameter tuning algorithms, we call them **Tuner**.
Tuner receives the result from `Trial` as a matrix to evaluate the performance of a specific parameters/architecture configures. And tuner sends next hyper-parameter or architecture configure to Trial. Tuner receives metrics from `Trial` to evaluate the performance of a specific parameters/architecture configures. And tuner sends next hyper-parameter or architecture configure to Trial.
In NNI, we support two approaches to set the tuner: first is directly use builtin tuner provided by nni sdk, second is customize a tuner file by yourself. We also have Advisor that combines the functinality of Tuner & Assessor. In NNI, we support two approaches to set the tuner: first is directly use builtin tuner provided by nni sdk, second is customize a tuner file by yourself. We also have Advisor that combines the functinality of Tuner & Assessor.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment