"nni provides nnicli module as a python wrapper for its restful APIs, which can be used to retrieve nni experiment and trial job information in your python code. This notebook shows how to use nnicli module.\n",
"\n",
"For a full nnicli API reference, please refer to [this documentation](https://nni.readthedocs.io/en/latest/nnicli_ref.html)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Start nni experiment using specified configuration file\n",
"Let's use a configruation file in nni examples directory to start an experiment. Make sure you have installed nni, seaborn and pytorch in your environment."
"text": "INFO: expand searchSpacePath: search_space.json to /home/xxx/nni/examples/trials/mnist-pytorch/search_space.json\nINFO: expand codeDir: . to /home/xxx/nni/examples/trials/mnist-pytorch/.\nINFO: Starting restful server...\nINFO: Successfully started Restful server!\nINFO: Setting local config...\nINFO: Successfully set local config!\nINFO: Starting experiment...\nINFO: Successfully started experiment!\n------------------------------------------------------------------------------------\nThe experiment id is OhHNEkLQ\nThe Web UI urls are: http://127.0.0.1:8080 http://xxx.xxx.xxx.xxx:8080 http://172.17.0.1:8080\n------------------------------------------------------------------------------------\n\nYou can use these commands to get more information about the experiment\n------------------------------------------------------------------------------------\ncommands description\n1. nnictl experiment show show the information of experiments\n2. nnictl trial ls list all of trial jobs\n3. nnictl top monitor the status of running experiments\n4. nnictl log stderr show stderr log content\n5. nnictl log stdout show stdout log content\n6. nnictl stop stop an experiment\n7. nnictl trial kill kill a trial job by id\n8. nnictl --help get help information about nnictl\n------------------------------------------------------------------------------------\nCommand reference document https://nni.readthedocs.io/en/latest/Tutorial/Nnictl.html\n------------------------------------------------------------------------------------\n\n"
"This simple example is to use NNI NAS 2.0(Retiarii) framework to search for the best neural architecture for tabular data classification task in Azure Machine Learning training platform.\n",
"\n",
"The video demo is https://www.youtube.com/watch?v=PDVqBmm7Cro and https://www.bilibili.com/video/BV1oy4y1W7GF."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1: Prepare the dataset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The first step is to prepare the dataset. Here we use the Titanic dataset as an example."
"Model space is defined by users to express a set of models that they want to explore, which contains potentially good-performing models. In Retiarii(NNI NAS 2.0) framework, a model space is defined with two parts: a base model and possible mutations on the base model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 2.1: Define the Base Model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Defining a base model is almost the same as defining a PyTorch (or TensorFlow) model. Usually, you only need to replace the code ``import torch.nn as nn`` with ``import nni.retiarii.nn.pytorch as nn`` to use NNI wrapped PyTorch modules. Below is a very simple example of defining a base model."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import nni.retiarii.nn.pytorch as nn\n",
"import torch.nn.functional as F\n",
"\n",
"class Net(nn.Module):\n",
"\n",
" def __init__(self, input_size):\n",
" super().__init__()\n",
"\n",
" self.fc1 = nn.Linear(input_size, 16)\n",
" self.bn1 = nn.BatchNorm1d(16)\n",
" self.dropout1 = nn.Dropout(0.0)\n",
"\n",
" self.fc2 = nn.Linear(16, 16)\n",
" self.bn2 = nn.BatchNorm1d(16)\n",
" self.dropout2 = nn.Dropout(0.0)\n",
"\n",
" self.fc3 = nn.Linear(16, 2)\n",
"\n",
" def forward(self, x):\n",
"\n",
" x = self.dropout1(F.relu(self.bn1(self.fc1(x))))\n",
" x = self.dropout2(F.relu(self.bn2(self.fc2(x))))\n",
"A base model is only one concrete model, not a model space. NNI provides APIs and primitives for users to express how the base model can be mutated, i.e., a model space that includes many models. The following will use inline Mutation APIs as a simple example. "
"Besides inline mutations, Retiarii also provides ``mutator``, a more general approach to express complex model space."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3: Explore the Defined Model Space"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the NAS process, the search strategy repeatedly generates new models, and the model evaluator is for training and validating each generated model. The obtained performance of a generated model is collected and sent to the search strategy for generating better models.\n",
"\n",
"Users can choose a proper search strategy to explore the model space, and use a chosen or user-defined model evaluator to evaluate the performance of each sampled model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 3.1: Choose a Search Strategy"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import nni.retiarii.strategy as strategy\n",
"\n",
"simple_strategy = strategy.TPEStrategy()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 3.2: Choose or Write a Model Evaluator"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the context of PyTorch, Retiarii has provided two built-in model evaluators, designed for simple use cases: classification and regression. These two evaluators are built upon the awesome library PyTorch-Lightning."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import nni.retiarii.evaluator.pytorch.lightning as pl\n",
"exp_config.nni_manager_ip = '' # your nni_manager_ip"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Running NNI experiments on the AML(Azure Machine Learning) training service is also simple, you only need to configure the following additional fields:"
kvartet <xuyiru1999@gmail.com>