Commit 4afe1670 authored by Guoxin's avatar Guoxin Committed by QuanluZhang
Browse files

re-organize links & Fix link err (#1125)

re-organise links for detailed descriptions of the tuners and accessors; fix link err in HpoComparision.md. delete #section from cross-file links to make links work in both readthedocs and github docs
parent d6763095
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
NNI provides state-of-the-art tuning algorithm in our builtin-assessors and makes them easy to use. Below is the brief overview of NNI current builtin Assessors: NNI provides state-of-the-art tuning algorithm in our builtin-assessors and makes them easy to use. Below is the brief overview of NNI current builtin Assessors:
Note: Click the **Assessor's name** to get a detailed description of the algorithm, click the corresponding **Usage** to get the Assessor's installation requirements, suggested scenario and using example. Note: Click the **Assessor's name** to get the Assessor's installation requirements, suggested scenario and using example. The link for a detailed description of the algorithm is at the end of the suggested scenario of each Assessor.
Currently we support the following Assessors: Currently we support the following Assessors:
...@@ -25,7 +25,7 @@ Note: Please follow the format when you write your `config.yml` file. ...@@ -25,7 +25,7 @@ Note: Please follow the format when you write your `config.yml` file.
**Suggested scenario** **Suggested scenario**
It is applicable in a wide range of performance curves, thus, can be used in various scenarios to speed up the tuning progress. It is applicable in a wide range of performance curves, thus, can be used in various scenarios to speed up the tuning progress. [Detailed Description](./MedianstopAssessor.md)
**Requirement of classArg** **Requirement of classArg**
...@@ -53,7 +53,7 @@ assessor: ...@@ -53,7 +53,7 @@ assessor:
**Suggested scenario** **Suggested scenario**
It is applicable in a wide range of performance curves, thus, can be used in various scenarios to speed up the tuning progress. Even better, it's able to handle and assess curves with similar performance. It is applicable in a wide range of performance curves, thus, can be used in various scenarios to speed up the tuning progress. Even better, it's able to handle and assess curves with similar performance. [Detailed Description](./CurvefittingAssessor.md)
**Requirement of classArg** **Requirement of classArg**
......
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
NNI provides state-of-the-art tuning algorithm as our builtin-tuners and makes them easy to use. Below is the brief summary of NNI currently built-in Tuners: NNI provides state-of-the-art tuning algorithm as our builtin-tuners and makes them easy to use. Below is the brief summary of NNI currently built-in Tuners:
Note: Click the **Tuner's name** to get a detailed description of the algorithm, click the corresponding **Usage** to get the Tuner's installation requirements, suggested scenario and using example. Here is an [article](./CommunitySharings/HPOComparison.md) about the comparison of different Tuners on several problems. Note: Click the **Tuner's name** to get the Tuner's installation requirements, suggested scenario and using example. The link for a detailed description of the algorithm is at the end of the suggested scenario of each tuner. Here is an [article](./CommunitySharings/HpoComparision.md) about the comparison of different Tuners on several problems.
Currently we support the following algorithms: Currently we support the following algorithms:
...@@ -36,7 +36,8 @@ Note: Please follow the format when you write your `config.yml` file. Some built ...@@ -36,7 +36,8 @@ Note: Please follow the format when you write your `config.yml` file. Some built
**Suggested scenario** **Suggested scenario**
TPE, as a black-box optimization, can be used in various scenarios and shows good performance in general. Especially when you have limited computation resource and can only try a small number of trials. From a large amount of experiments, we could found that TPE is far better than Random Search. TPE, as a black-box optimization, can be used in various scenarios and shows good performance in general. Especially when you have limited computation resource and can only try a small number of trials. From a large amount of experiments, we could found that TPE is far better than Random Search. [Detailed Description](./HyperoptTuner.md)
**Requirement of classArg** **Requirement of classArg**
...@@ -62,7 +63,7 @@ tuner: ...@@ -62,7 +63,7 @@ tuner:
**Suggested scenario** **Suggested scenario**
Random search is suggested when each trial does not take too long (e.g., each trial can be completed very soon, or early stopped by assessor quickly), and you have enough computation resource. Or you want to uniformly explore the search space. Random Search could be considered as baseline of search algorithm. Random search is suggested when each trial does not take too long (e.g., each trial can be completed very soon, or early stopped by assessor quickly), and you have enough computation resource. Or you want to uniformly explore the search space. Random Search could be considered as baseline of search algorithm. [Detailed Description](./HyperoptTuner.md)
**Requirement of classArg:** **Requirement of classArg:**
...@@ -86,7 +87,8 @@ tuner: ...@@ -86,7 +87,8 @@ tuner:
**Suggested scenario** **Suggested scenario**
Anneal is suggested when each trial does not take too long, and you have enough computation resource(almost same with Random Search). Or the variables in search space could be sample from some prior distribution. Anneal is suggested when each trial does not take too long, and you have enough computation resource(almost same with Random Search). Or the variables in search space could be sample from some prior distribution. [Detailed Description](./HyperoptTuner.md)
**Requirement of classArg** **Requirement of classArg**
...@@ -112,7 +114,8 @@ tuner: ...@@ -112,7 +114,8 @@ tuner:
**Suggested scenario** **Suggested scenario**
Its requirement of computation resource is relatively high. Specifically, it requires large initial population to avoid falling into local optimum. If your trial is short or leverages assessor, this tuner is a good choice. And, it is more suggested when your trial code supports weight transfer, that is, the trial could inherit the converged weights from its parent(s). This can greatly speed up the training progress. Its requirement of computation resource is relatively high. Specifically, it requires large initial population to avoid falling into local optimum. If your trial is short or leverages assessor, this tuner is a good choice. And, it is more suggested when your trial code supports weight transfer, that is, the trial could inherit the converged weights from its parent(s). This can greatly speed up the training progress. [Detailed Description](./EvolutionTuner.md)
**Usage example** **Usage example**
...@@ -144,7 +147,7 @@ nnictl package install --name=SMAC ...@@ -144,7 +147,7 @@ nnictl package install --name=SMAC
**Suggested scenario** **Suggested scenario**
Similar to TPE, SMAC is also a black-box tuner which can be tried in various scenarios, and is suggested when computation resource is limited. It is optimized for discrete hyperparameters, thus, suggested when most of your hyperparameters are discrete. Similar to TPE, SMAC is also a black-box tuner which can be tried in various scenarios, and is suggested when computation resource is limited. It is optimized for discrete hyperparameters, thus, suggested when most of your hyperparameters are discrete. [Detailed Description](./SmacTuner.md)
**Requirement of classArg** **Requirement of classArg**
...@@ -170,7 +173,7 @@ tuner: ...@@ -170,7 +173,7 @@ tuner:
**Suggested scenario** **Suggested scenario**
If the configurations you want to try have been decided, you can list them in searchspace file (using `choice`) and run them using batch tuner. If the configurations you want to try have been decided, you can list them in searchspace file (using `choice`) and run them using batch tuner. [Detailed Description](./BatchTuner.md)
**Usage example** **Usage example**
...@@ -211,7 +214,7 @@ The search space file including the high-level key `combine_params`. The type of ...@@ -211,7 +214,7 @@ The search space file including the high-level key `combine_params`. The type of
Note that the only acceptable types of search space are `choice`, `quniform`, `qloguniform`. **The number `q` in `quniform` and `qloguniform` has special meaning (different from the spec in [search space spec](./SearchSpaceSpec.md)). It means the number of values that will be sampled evenly from the range `low` and `high`.** Note that the only acceptable types of search space are `choice`, `quniform`, `qloguniform`. **The number `q` in `quniform` and `qloguniform` has special meaning (different from the spec in [search space spec](./SearchSpaceSpec.md)). It means the number of values that will be sampled evenly from the range `low` and `high`.**
It is suggested when search space is small, it is feasible to exhaustively sweeping the whole search space. It is suggested when search space is small, it is feasible to exhaustively sweeping the whole search space. [Detailed Description](./GridsearchTuner.md)
**Usage example** **Usage example**
...@@ -231,7 +234,7 @@ tuner: ...@@ -231,7 +234,7 @@ tuner:
**Suggested scenario** **Suggested scenario**
It is suggested when you have limited computation resource but have relatively large search space. It performs well in the scenario that intermediate result (e.g., accuracy) can reflect good or bad of final result (e.g., accuracy) to some extent. It is suggested when you have limited computation resource but have relatively large search space. It performs well in the scenario that intermediate result (e.g., accuracy) can reflect good or bad of final result (e.g., accuracy) to some extent. [Detailed Description](./HyperbandAdvisor.md)
**Requirement of classArg** **Requirement of classArg**
...@@ -265,7 +268,7 @@ NetworkMorphism requires [pyTorch](https://pytorch.org/get-started/locally), so ...@@ -265,7 +268,7 @@ NetworkMorphism requires [pyTorch](https://pytorch.org/get-started/locally), so
**Suggested scenario** **Suggested scenario**
It is suggested that you want to apply deep learning methods to your task (your own dataset) but you have no idea of how to choose or design a network. You modify the [example](https://github.com/Microsoft/nni/tree/master/examples/trials/network_morphism/cifar10/cifar10_keras.py) to fit your own dataset and your own data augmentation method. Also you can change the batch size, learning rate or optimizer. It is feasible for different tasks to find a good network architecture. Now this tuner only supports the computer vision domain. It is suggested that you want to apply deep learning methods to your task (your own dataset) but you have no idea of how to choose or design a network. You modify the [example](https://github.com/Microsoft/nni/tree/master/examples/trials/network_morphism/cifar10/cifar10_keras.py) to fit your own dataset and your own data augmentation method. Also you can change the batch size, learning rate or optimizer. It is feasible for different tasks to find a good network architecture. Now this tuner only supports the computer vision domain. [Detailed Description](./NetworkmorphismTuner.md)
**Requirement of classArg** **Requirement of classArg**
...@@ -305,7 +308,7 @@ Metis Tuner requires [sklearn](https://scikit-learn.org/), so users should insta ...@@ -305,7 +308,7 @@ Metis Tuner requires [sklearn](https://scikit-learn.org/), so users should insta
**Suggested scenario** **Suggested scenario**
Similar to TPE and SMAC, Metis is a black-box tuner. If your system takes a long time to finish each trial, Metis is more favorable than other approaches such as random search. Furthermore, Metis provides guidance on the subsequent trial. Here is an [example](https://github.com/Microsoft/nni/tree/master/examples/trials/auto-gbdt/search_space_metis.json) about the use of Metis. User only need to send the final result like `accuracy` to tuner, by calling the nni SDK. Similar to TPE and SMAC, Metis is a black-box tuner. If your system takes a long time to finish each trial, Metis is more favorable than other approaches such as random search. Furthermore, Metis provides guidance on the subsequent trial. Here is an [example](https://github.com/Microsoft/nni/tree/master/examples/trials/auto-gbdt/search_space_metis.json) about the use of Metis. User only need to send the final result like `accuracy` to tuner, by calling the nni SDK. [Detailed Description](./MetisTuner.md)
**Requirement of classArg** **Requirement of classArg**
...@@ -339,7 +342,7 @@ nnictl package install --name=BOHB ...@@ -339,7 +342,7 @@ nnictl package install --name=BOHB
**Suggested scenario** **Suggested scenario**
Similar to Hyperband, it is suggested when you have limited computation resource but have relatively large search space. It performs well in the scenario that intermediate result (e.g., accuracy) can reflect good or bad of final result (e.g., accuracy) to some extent. In this case, it may converges to a better configuration due to bayesian optimization usage. Similar to Hyperband, it is suggested when you have limited computation resource but have relatively large search space. It performs well in the scenario that intermediate result (e.g., accuracy) can reflect good or bad of final result (e.g., accuracy) to some extent. In this case, it may converges to a better configuration due to bayesian optimization usage. [Detailed Description](./BohbAdvisor.md)
**Requirement of classArg** **Requirement of classArg**
......
...@@ -5,17 +5,17 @@ Comparison of Hyperparameter Optimization algorithms on several problems. ...@@ -5,17 +5,17 @@ Comparison of Hyperparameter Optimization algorithms on several problems.
Hyperparameter Optimization algorithms are list below: Hyperparameter Optimization algorithms are list below:
- [Random Search](../Builtin_Tuner.md#Random) - [Random Search](../BuiltinTuner.md)
- [Grid Search](../Builtin_Tuner.md#Random) - [Grid Search](../BuiltinTuner.md)
- [Evolution](../Builtin_Tuner.md#Evolution) - [Evolution](../BuiltinTuner.md)
- [Anneal](../Builtin_Tuner.md#Anneal) - [Anneal](../BuiltinTuner.md)
- [Metis](../Builtin_Tuner.md#MetisTuner) - [Metis](../BuiltinTuner.md)
- [TPE](../Builtin_Tuner.md#TPE) - [TPE](../BuiltinTuner.md)
- [SMAC](../Builtin_Tuner.md#SMAC) - [SMAC](../BuiltinTuner.md)
- [HyperBand](../Builtin_Tuner.md#Hyperband) - [HyperBand](../BuiltinTuner.md)
- [BOHB](../Builtin_Tuner.md#BOHB) - [BOHB](../BuiltinTuner.md)
All algorithms run in NNI local environment All algorithms run in NNI local environment.
Machine Environment: Machine Environment:
......
...@@ -4,7 +4,7 @@ Currently we support local, remote and pai mode on Windows. Windows 10.1809 is w ...@@ -4,7 +4,7 @@ Currently we support local, remote and pai mode on Windows. Windows 10.1809 is w
## **Installation on Windows** ## **Installation on Windows**
please refer to [Installation](Installation.md#installation-on-windows) for more details. please refer to [Installation](Installation.md) for more details.
When these things are done, use the **config_windows.yml** configuration to start an experiment for validation. When these things are done, use the **config_windows.yml** configuration to start an experiment for validation.
......
...@@ -4,6 +4,6 @@ Builtin-Assessors ...@@ -4,6 +4,6 @@ Builtin-Assessors
.. toctree:: .. toctree::
:maxdepth: 1 :maxdepth: 1
Overview<BuiltinAssessors> Overview<BuiltinAssessor>
Medianstop<MedianstopAssessor> Medianstop<MedianstopAssessor>
Curvefitting<CurvefittingAssessor> Curvefitting<CurvefittingAssessor>
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment