Skip to content

GitLab

Unverified Commit c4fd7151 authored by Junwei Sun's avatar Junwei Sun Committed by GitHub
Browse files

Update docs for aml and nasbenchmark, re-submit from #2647 (#2651)

parent e4a6427b
Hide whitespace changes
Inline Side-by-side
Showing with 94 additions and 55 deletions
docs/en_US/NAS/Benchmarks.md
View file @ c4fd7151
......@@ -7,6 +7,9 @@
Example Usages <BenchmarksExample>
```
## Introduction
To imporve the reproducibility of NAS algorithms as well as reducing computing resource requirements, researchers proposed a series of NAS benchmarks such as [NAS-Bench-101](https://arxiv.org/abs/1902.09635), [NAS-Bench-201](https://arxiv.org/abs/2001.00326), [NDS](https://arxiv.org/abs/1905.13214), etc. NNI provides a query interface for users to acquire these benchmarks. Within just a few lines of code, researcher are able to evaluate their NAS algorithms easily and fairly by utilizing these benchmarks.
## Prerequisites
* Please prepare a folder to household all the benchmark databases. By default, it can be found at `${HOME}/.nni/nasbenchmark`. You can place it anywhere you like, and specify it in `NASBENCHMARK_DIR` before importing NNI.
......@@ -14,33 +17,15 @@
## Data Preparation
To avoid storage and legal issues, we do not provide any prepared databases. We strongly recommend users to use docker to run the generation scripts, to ease the burden of installing multiple dependencies. Please follow the following steps.
**Step 1.** Clone NNI repo. Replace `${NNI_VERSION}` with a released version name or branch name, e.g., `v1.6`.
```bash
git clone -b ${NNI_VERSION} https://github.com/microsoft/nni
```
**Step 2.** Run docker.
For NAS-Bench-101,
To avoid storage and legality issues, we do not provide any prepared databases. Please follow the following steps.
```bash
docker run -v ${HOME}/.nni/nasbenchmark:/outputs -v /path/to/your/nni:/nni tensorflow/tensorflow:1.15.2-py3 /bin/bash /nni/examples/nas/benchmarks/nasbench101.sh
1. Clone NNI to your machine and enter `examples/nas/benchmarks` directory.
```
For NAS-Bench-201,
```bash
docker run -v ${HOME}/.nni/nasbenchmark:/outputs -v /path/to/your/nni:/nni ufoym/deepo:pytorch-cpu /bin/bash /nni/examples/nas/benchmarks/nasbench201.sh
```
For NDS,
```bash
docker run -v ${HOME}/.nni/nasbenchmark:/outputs -v /path/to/your/nni:/nni python:3.7 /bin/bash /nni/examples/nas/benchmarks/nds.sh
git clone https://github.com/microsoft/nni.git
cd nni/examples/nas/benchmarks
```
2. Install dependencies via `pip3 install -r xxx.requirements.txt`. `xxx` can be `nasbench101`, `nasbench201` or `nds`.
3. Generate the database via `./xxx.sh`. The directory that stores the benchmark file can be configured with `NASBENCHMARK_DIR` environment variable, which defaults to `~/.nni/nasbenchmark`. Note that the NAS-Bench-201 dataset will be downloaded from a google drive.
Please make sure there is at least 10GB free disk space and note that the conversion process can take up to hours to complete.
......
docs/en_US/NAS/BenchmarksExample.ipynb
View file @ c4fd7151
{
"nbformat": 4,
"nbformat_minor": 2,
"metadata": {
"language_info": {
"name": "python",
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"version": "3.6.10-final"
},
"orig_nbformat": 2,
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"npconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": 3,
"kernelspec": {
"name": "python361064bitnnilatestcondabff8d66a619a4d26af34fe0fe687c7b0",
"display_name": "Python 3.6.10 64-bit ('nnilatest': conda)"
}
},
"cells": [
{
"cell_type": "markdown",
......@@ -53,6 +30,14 @@
"## NAS-Bench-101"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the following architecture as an example:<br>\n",
"![nas-101](../../img/nas-bench-101-example.png)"
]
},
{
"cell_type": "code",
"execution_count": 2,
......@@ -82,6 +67,13 @@
" pprint.pprint(t)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"An architecture of NAS-Bench-101 could be trained more than once. Each element of the returned generator is a dict which contains one of the training results of this trial config (architecture + hyper-parameters) including train/valid/test accuracy, training time, number of epochs, etc. The results of NAS-Bench-201 and NDS follow similar formats."
]
},
{
"cell_type": "markdown",
"metadata": {},
......@@ -89,6 +81,14 @@
"## NAS-Bench-201"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the following architecture as an example:<br>\n",
"![nas-201](../../img/nas-bench-201-example.png)"
]
},
{
"cell_type": "code",
"execution_count": 3,
......@@ -120,6 +120,16 @@
"## NDS"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the following architecture as an example:<br>\n",
"![nds](../../img/nas-bench-nds-example.png)\n",
"\n",
"Here, `bot_muls`, `ds`, `num_gs`, `ss` and `ws` stand for \"bottleneck multipliers\", \"depths\", \"number of groups\", \"strides\" and \"widths\" respectively."
]
},
{
"cell_type": "code",
"execution_count": 4,
......@@ -273,5 +283,28 @@
"print('Elapsed time: ', time.time() - ti, 'seconds')"
]
}
]
],
"metadata": {
"language_info": {
"name": "python",
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"version": "3.6.10-final"
},
"orig_nbformat": 2,
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"npconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": 3,
"kernelspec": {
"name": "python361064bitnnilatestcondabff8d66a619a4d26af34fe0fe687c7b0",
"display_name": "Python 3.6.10 64-bit ('nnilatest': conda)"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
\ No newline at end of file
docs/en_US/TrainingService/AMLMode.md
View file @ c4fd7151
......@@ -5,12 +5,22 @@ NNI supports running an experiment on [AML](https://azure.microsoft.com/en-us/se
## Setup environment
Step 1. Install NNI, follow the install guide [here](../Tutorial/QuickStart.md).
Step 2. Create AML account, follow the document [here](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-manage-workspace-cli).
Step 2. Create an Azure account/subscription using this [link](https://azure.microsoft.com/en-us/free/services/machine-learning/). If you already have an Azure account/subscription, skip this step.
Step 3. Get your account information.
![](../../img/aml_account.png)
Step 3. Install the Azure CLI on your machine, follow the install guide [here](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest).
Step4. Install AML package environment.
Step 4. Authenticate to your Azure subscription from the CLI. To authenticate interactively, open a command line or terminal and use the following command:
```
az login
```
Step 5. Log into your Azure account with a web browser and create a Machine Learning resource. You will need to choose a resource group and specific a workspace name. Then download `config.json` which will be used later.
![](../../img/aml_workspace.png)
Step 6. Create an AML cluster as the computeTarget.
![](../../img/aml_cluster.png)
Step 7. Open a command line and install AML package environment.
```
python3 -m pip install azureml --user
python3 -m pip install azureml-sdk --user
......@@ -52,9 +62,9 @@ Note: You should set `trainingServicePlatform: aml` in NNI config YAML file if y
Compared with [LocalMode](LocalMode.md) trial configuration in aml mode have these additional keys:
* computeTarget
* required key. The computer cluster name you want to use in your AML workspace.
* required key. The compute cluster name you want to use in your AML workspace. See Step 6.
* image
* required key. The docker image name used in job.
* required key. The docker image name used in job. The image `msranni/nni` of this example only support GPU computeTargets.
amlConfig:
* subscriptionId
......@@ -63,4 +73,15 @@ amlConfig:
* the resourceGroup of your account
* workspaceName
* the workspaceName of your account
\ No newline at end of file
The required information of amlConfig could be found in the downloaded `config.json` in Step 5.
Run the following commands to start the example experiment:
```
git clone https://github.com/microsoft/nni.git
cd nni/examples/trials/mnist-tfv1
# modify config_aml.yml ...
nnictl create --config config_aml.yml
```
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment