retrieve_nni_info_with_python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python wrapper for nni restful APIs\n",
    "\n",
    "nni provides nnicli module as a python wrapper for its restful APIs, which can be used to retrieve nni experiment and trial job information in your python code. This notebook shows how to use nnicli module.\n",
    "\n",
    "Following are the functions available in nnicli module:\n",
    "\n",
    "#### start_nni(config_file)\n",
    "Starts nni experiment with specified configuration file\n",
    "\n",
    "#### stop_nni()\n",
    "Stop nni experiment.\n",
    "\n",
    "#### set_endpoint(endpoint)\n",
    "Set nni endpoint for nnicli, the endpoint is showed while nni experiment is started successfully using nnictl command or start_nni function\n",
    "\n",
    "#### version()\n",
    "Returns nni version\n",
    "\n",
    "#### get_experiment_profile()\n",
    "Returns experiment profile.\n",
    "\n",
    "#### get_experiment_status()\n",
    "Returns nni experiment status.\n",
    "\n",
    "#### get_job_metrics(trial_job_id)\n",
    "Returns specified trial job metrics, including final results and intermediate results.\n",
    "\n",
    "#### get_job_statistics()\n",
    "Returns trial job statistics information\n",
    "\n",
    "#### get_trial_job(trial_job_id)\n",
    "Returns information of a specified trial job.\n",
    "\n",
    "#### list_trial_jobs()\n",
    "Returns information of all trial jobs of current experiment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Start nni experiment using specified configuration file\n",
    "Let's use a configruation file in nni examples directory to start an experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "authorName: default\r\n",
      "experimentName: example_mnist\r\n",
      "trialConcurrency: 1\r\n",
      "maxExecDuration: 1h\r\n",
      "maxTrialNum: 10\r\n",
      "#choice: local, remote, pai\r\n",
      "trainingServicePlatform: local\r\n",
      "searchSpacePath: search_space.json\r\n",
      "#choice: true, false\r\n",
      "useAnnotation: false\r\n",
      "tuner:\r\n",
      "  #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner, GPTuner\r\n",
      "  #SMAC (SMAC should be installed through nnictl)\r\n",
      "  builtinTunerName: TPE\r\n",
      "  classArgs:\r\n",
      "    #choice: maximize, minimize\r\n",
      "    optimize_mode: maximize\r\n",
      "trial:\r\n",
      "  command: python3 mnist.py\r\n",
      "  codeDir: .\r\n",
      "  gpuNum: 0\r\n"
     ]
    }
   ],
   "source": [
    "! cat ../trials/mnist/config.yml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO: expand searchSpacePath: search_space.json to /mnt/d/Repos/nni/examples/trials/mnist/search_space.json\n",
      "INFO: expand codeDir: . to /mnt/d/Repos/nni/examples/trials/mnist/.\n",
      "INFO: Starting restful server...\n",
      "INFO: Successfully started Restful server!\n",
      "INFO: Setting local config...\n",
      "INFO: Successfully set local config!\n",
      "INFO: Starting experiment...\n",
      "INFO: Successfully started experiment!\n",
      "-----------------------------------------------------------------------\n",
      "The experiment id is PlUIfDTR\n",
      "The Web UI urls are: http://172.18.17.1:8080   http://10.172.121.40:8080   http://10.0.75.1:8080   http://127.0.0.1:8080\n",
      "-----------------------------------------------------------------------\n",
      "\n",
      "You can use these commands to get more information about the experiment\n",
      "-----------------------------------------------------------------------\n",
      "commands                       description\n",
      "1. nnictl experiment show        show the information of experiments\n",
      "2. nnictl trial ls               list all of trial jobs\n",
      "3. nnictl top                    monitor the status of running experiments\n",
      "4. nnictl log stderr             show stderr log content\n",
      "5. nnictl log stdout             show stdout log content\n",
      "6. nnictl stop                   stop an experiment\n",
      "7. nnictl trial kill             kill a trial job by id\n",
      "8. nnictl --help                 get help information about nnictl\n",
      "-----------------------------------------------------------------------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import nnicli as nc\n",
    "nc.start_nni(config_file='../trials/mnist/config.yml')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Connect nnicli module to started nni experiment\n",
    "Call set_endpoint to connect nnicli moduele to the rest server of started nni experiment. Local mode training serviced is used in this notebook, but nnicli module can connect to any started nni experiment. The endpoint can be found in the output of start_nni function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "nc.set_endpoint('http://127.0.0.1:8080')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Retrieve nni experiment and trial job information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'errors': [], 'status': 'RUNNING'}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nc.get_experiment_status()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'trialJobNumber': 4, 'trialJobStatus': 'SUCCEEDED'},\n",
       " {'trialJobNumber': 1, 'trialJobStatus': 'RUNNING'}]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nc.get_job_statistics()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'execDuration': 1117,\n",
       " 'id': 'PlUIfDTR',\n",
       " 'logDir': '/home/chicm/nni/experiments/PlUIfDTR',\n",
       " 'maxSequenceId': 3,\n",
       " 'params': {'authorName': 'default',\n",
       "  'clusterMetaData': [{'key': 'codeDir',\n",
       "    'value': '/mnt/d/Repos/nni/examples/trials/mnist/.'},\n",
       "   {'key': 'command', 'value': 'python3 mnist.py'}],\n",
       "  'experimentName': 'example_mnist',\n",
       "  'maxExecDuration': 3600,\n",
       "  'maxTrialNum': 10,\n",
       "  'searchSpace': '{\"hidden_size\": {\"_value\": [124, 512, 1024], \"_type\": \"choice\"}, \"batch_size\": {\"_value\": [1, 4, 8, 16, 32], \"_type\": \"choice\"}, \"conv_size\": {\"_value\": [2, 3, 5, 7], \"_type\": \"choice\"}, \"dropout_rate\": {\"_value\": [0.5, 0.9], \"_type\": \"uniform\"}, \"learning_rate\": {\"_value\": [0.0001, 0.001, 0.01, 0.1], \"_type\": \"choice\"}}',\n",
       "  'trainingServicePlatform': 'local',\n",
       "  'trialConcurrency': 1,\n",
       "  'tuner': {'builtinTunerName': 'TPE',\n",
       "   'checkpointDir': '/home/chicm/nni/experiments/PlUIfDTR/checkpoint',\n",
       "   'classArgs': {'optimize_mode': 'maximize'},\n",
       "   'className': 'TPE'},\n",
       "  'versionCheck': True},\n",
       " 'revision': 116,\n",
       " 'startTime': 1564484985839}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nc.get_experiment_profile()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's define an utility function to format json string returned by nnicli module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "def show_json(res):\n",
    "    print(json.dumps(res, indent=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"params\": {\n",
      "        \"searchSpace\": \"{\\\"hidden_size\\\": {\\\"_value\\\": [124, 512, 1024], \\\"_type\\\": \\\"choice\\\"}, \\\"batch_size\\\": {\\\"_value\\\": [1, 4, 8, 16, 32], \\\"_type\\\": \\\"choice\\\"}, \\\"conv_size\\\": {\\\"_value\\\": [2, 3, 5, 7], \\\"_type\\\": \\\"choice\\\"}, \\\"dropout_rate\\\": {\\\"_value\\\": [0.5, 0.9], \\\"_type\\\": \\\"uniform\\\"}, \\\"learning_rate\\\": {\\\"_value\\\": [0.0001, 0.001, 0.01, 0.1], \\\"_type\\\": \\\"choice\\\"}}\",\n",
      "        \"clusterMetaData\": [\n",
      "            {\n",
      "                \"key\": \"codeDir\",\n",
      "                \"value\": \"/mnt/d/Repos/nni/examples/trials/mnist/.\"\n",
      "            },\n",
      "            {\n",
      "                \"key\": \"command\",\n",
      "                \"value\": \"python3 mnist.py\"\n",
      "            }\n",
      "        ],\n",
      "        \"tuner\": {\n",
      "            \"classArgs\": {\n",
      "                \"optimize_mode\": \"maximize\"\n",
      "            },\n",
      "            \"builtinTunerName\": \"TPE\",\n",
      "            \"checkpointDir\": \"/home/chicm/nni/experiments/PlUIfDTR/checkpoint\",\n",
      "            \"className\": \"TPE\"\n",
      "        },\n",
      "        \"maxTrialNum\": 10,\n",
      "        \"maxExecDuration\": 3600,\n",
      "        \"experimentName\": \"example_mnist\",\n",
      "        \"authorName\": \"default\",\n",
      "        \"trialConcurrency\": 1,\n",
      "        \"trainingServicePlatform\": \"local\",\n",
      "        \"versionCheck\": true\n",
      "    },\n",
      "    \"execDuration\": 1192,\n",
      "    \"revision\": 124,\n",
      "    \"logDir\": \"/home/chicm/nni/experiments/PlUIfDTR\",\n",
      "    \"maxSequenceId\": 3,\n",
      "    \"id\": \"PlUIfDTR\",\n",
      "    \"startTime\": 1564484985839\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "show_json(nc.get_experiment_profile())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "    {\n",
      "        \"startTime\": 1564484995992,\n",
      "        \"hyperParameters\": [\n",
      "            \"{\\\"parameter_source\\\":\\\"algorithm\\\",\\\"parameter_id\\\":0,\\\"parameter_index\\\":0,\\\"parameters\\\":{\\\"batch_size\\\":8,\\\"conv_size\\\":3,\\\"hidden_size\\\":1024,\\\"learning_rate\\\":0.0001,\\\"dropout_rate\\\":0.8055724367106529}}\"\n",
      "        ],\n",
      "        \"id\": \"BW0NR\",\n",
      "        \"endTime\": 1564485259753,\n",
      "        \"status\": \"SUCCEEDED\",\n",
      "        \"sequenceId\": 0,\n",
      "        \"finalMetricData\": [\n",
      "            {\n",
      "                \"parameterId\": \"0\",\n",
      "                \"type\": \"FINAL\",\n",
      "                \"trialJobId\": \"BW0NR\",\n",
      "                \"timestamp\": 1564485258774,\n",
      "                \"data\": \"0.9078999757766724\",\n",
      "                \"sequence\": 0\n",
      "            }\n",
      "        ],\n",
      "        \"logPath\": \"file://localhost:/home/chicm/nni/experiments/PlUIfDTR/trials/BW0NR\"\n",
      "    },\n",
      "    {\n",
      "        \"startTime\": 1564485271947,\n",
      "        \"hyperParameters\": [\n",
      "            \"{\\\"parameter_source\\\":\\\"algorithm\\\",\\\"parameter_id\\\":1,\\\"parameter_index\\\":0,\\\"parameters\\\":{\\\"batch_size\\\":4,\\\"conv_size\\\":5,\\\"hidden_size\\\":512,\\\"learning_rate\\\":0.01,\\\"dropout_rate\\\":0.5547528540531742}}\"\n",
      "        ],\n",
      "        \"id\": \"x0P5w\",\n",
      "        \"endTime\": 1564485642784,\n",
      "        \"status\": \"SUCCEEDED\",\n",
      "        \"sequenceId\": 1,\n",
      "        \"finalMetricData\": [\n",
      "            {\n",
      "                \"parameterId\": \"1\",\n",
      "                \"type\": \"FINAL\",\n",
      "                \"trialJobId\": \"x0P5w\",\n",
      "                \"timestamp\": 1564485642072,\n",
      "                \"data\": \"0.10100000351667404\",\n",
      "                \"sequence\": 0\n",
      "            }\n",
      "        ],\n",
      "        \"logPath\": \"file://localhost:/home/chicm/nni/experiments/PlUIfDTR/trials/x0P5w\"\n",
      "    },\n",
      "    {\n",
      "        \"startTime\": 1564485652151,\n",
      "        \"hyperParameters\": [\n",
      "            \"{\\\"parameter_source\\\":\\\"algorithm\\\",\\\"parameter_id\\\":2,\\\"parameter_index\\\":0,\\\"parameters\\\":{\\\"batch_size\\\":8,\\\"conv_size\\\":3,\\\"hidden_size\\\":512,\\\"learning_rate\\\":0.0001,\\\"dropout_rate\\\":0.5584485925416655}}\"\n",
      "        ],\n",
      "        \"id\": \"V9jSG\",\n",
      "        \"endTime\": 1564485917057,\n",
      "        \"status\": \"SUCCEEDED\",\n",
      "        \"sequenceId\": 2,\n",
      "        \"finalMetricData\": [\n",
      "            {\n",
      "                \"parameterId\": \"2\",\n",
      "                \"type\": \"FINAL\",\n",
      "                \"trialJobId\": \"V9jSG\",\n",
      "                \"timestamp\": 1564485916403,\n",
      "                \"data\": \"0.928600013256073\",\n",
      "                \"sequence\": 0\n",
      "            }\n",
      "        ],\n",
      "        \"logPath\": \"file://localhost:/home/chicm/nni/experiments/PlUIfDTR/trials/V9jSG\"\n",
      "    },\n",
      "    {\n",
      "        \"startTime\": 1564485927295,\n",
      "        \"hyperParameters\": [\n",
      "            \"{\\\"parameter_source\\\":\\\"algorithm\\\",\\\"parameter_id\\\":3,\\\"parameter_index\\\":0,\\\"parameters\\\":{\\\"batch_size\\\":8,\\\"conv_size\\\":7,\\\"hidden_size\\\":124,\\\"learning_rate\\\":0.001,\\\"dropout_rate\\\":0.6281630602835235}}\"\n",
      "        ],\n",
      "        \"id\": \"CDlRX\",\n",
      "        \"status\": \"RUNNING\",\n",
      "        \"sequenceId\": 3,\n",
      "        \"logPath\": \"file://localhost:/home/chicm/nni/experiments/PlUIfDTR/trials/CDlRX\"\n",
      "    }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "show_json(nc.list_trial_jobs())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing nni experiment result\n",
    "\n",
    "With the retrieved trial job information, we can do some analysis by visualizing the metric data, below is a simple example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3QAAAF4CAYAAAAczbvpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xl8TNfj//F3MiQoGoQIsdTSVBFSitoiERIkoiFFPnSh0ZbSaq1dkBa1VKuWVqtaSz6WplJLpPiqvSUUReujVKkiIhJbQiWZ5PeHn/mYT0gGE8nl9Xw8PB6Ze889c85xM5P3vefe65CdnZ0tAAAAAIDhOBZ0AwAAAAAAd4ZABwAAAAAGRaADAAAAAIMi0AEAAACAQRHoAAAAAMCgCHQAAAAAYFAEOgCAXcycOVOjRo2yqeyQIUM0ffr0m6777rvv9OKLL9pUT8+ePRUTE2NzG6OiovTUU0/J29tbly5dkre3t06dOmXz9rfy8ccfa8SIEXddz91q3bq14uPjC7oZAIB7qEhBNwAAUPh4e3tbfr5y5YqcnJxkMpkkSZGRkercuXOObQYMGGCX93766af19NNP26WuG129elUTJ05UTEyMateuLUnas2eP3d+nsPj444+VmJioCRMmFHRTAAD5iEAHAMjhxqDj5+ensWPHqnnz5rcsn5mZqSJFCvdXSlJSktLT0y1hrqAYYawAAMbBlEsAwG37+OOP9frrr+uNN96Qt7e3VqxYYTXtMCsrS4MGDVKLFi3UuHFj9e7dW0eOHLGp7ujoaPXu3dvy+ueff1ZoaKgaNWqkbt26ae/evVbl//rrL8v6AQMG6MKFCznqPHLkiIKCgiRdO/vYp08fZWZmytPTUydOnJB0bRro2LFj9eKLL8rb21vdu3fX33//banjvffeU+vWrfXEE0+oa9eu2r17t039+emnn+Tn56dZs2apRYsWevfddyVJP/zwgzp37qzGjRurZ8+eOnTokGWbWbNmqWXLlnriiScUGBhomUb5v1NVr9f9vzZs2KA5c+Zo5cqV8vb2VmhoqGVs/fz85O3trbZt22rVqlU29QEAUHgR6AAAd2TdunUKCgrSrl271LFjxxzr27RpozVr1ujHH39U7dq1NXTo0Nt+j5SUFL300kvq06eP4uPj1atXL/Xr188qtC1btkyTJk3Sli1blJ2drQ8++CBHPTVr1tTy5cslXTv7+NVXX930/VauXKnXXntNO3bskLu7uz755BPLOi8vL61YsUI7duxQQECAXnvtNaWnp9vUj9OnT+vy5cvasGGDRo8erX379undd9/V2LFjFR8fr65du6p///5KT0/X4cOHtWTJEn333XfavXu3Zs+erUqVKt3OsMnX11d9+/ZVcHCw9uzZo5iYGKWmpmrChAn66quvtGfPHi1atEienp63VS8AoPAh0AEA7sgTTzwhPz8/OTo6qlixYlbrHB0dFRoaqpIlS8rZ2VmvvvqqfvvtN12+fPm23mPDhg2qXbu2goKCVKRIEXXp0kUeHh7auHGjpUyXLl1Uq1YtlShRQoMGDdKqVauUnZ19R30KCAhQ/fr1VbRoUQUHB+vgwYNW7+Pi4qIiRYooIiJCqamp+uuvv2yq12Qy6dVXX5WTk5OKFSumb775RuHh4fLy8pLJZFK3bt0kSfv375fJZNLVq1f1xx9/KDMzU1WqVFGVKlXuqD//y8HBQYcPH9bVq1dVoUIF1apVyy71AgAKDpP4AQB3xN3d/ZbrzGazpkyZojVr1ujcuXNydLx2/PDcuXMqUaKEze9x5syZHGenKleurMTExJu2o3LlykpPT9f58+dVpkwZm9/nuvLly1t+Ll68uFUAnT17tpYuXaqkpCQ5ODjoypUrOnfunE31urq6ysnJyfL61KlTWrlypebOnWtZlpGRocTERDVq1EjDhw/XJ598oj///FMtW7bUyJEjrdp2J0qWLKkpU6bo66+/1siRI9WoUSONGDFCjzzyyF3VCwAoWJyhAwDcEQcHh1uuW7ZsmTZv3qx58+Zp165dWrt2rSTd9pmzChUq5HiswKlTp+Tm5mZ5nZCQYLXOyclJLi4ut/U+edm+fbvmzp2r6dOn6+eff9bOnTtVokQJm/vzv2NVsWJFDRgwQD///LPl3969ey1TV0NCQrR48WL98MMPMpvN+uijjyRJJUqU0JUrVyz1JCUl2fyekuTj46O5c+dq69atqlq1qs2PmQAAFF4EOgCA3aWlpVmC1ZUrVzR16tQ7qsfX11eHDx9WXFycMjMztXLlSh0/flxt2rSxlFm+fLmOHDmiy5cva/r06erQoUOuYfNOpKWlyWQyqUyZMsrIyND06dOtgtXteuaZZ7Rw4ULt27dP2dnZSktL0/r163X58mUdOXJE27dvV3p6upydnVWsWDFLfx577DFt2rRJFy5c0JkzZ7RgwYJbvoerq6tOnjxpCZ1nzpzR+vXrdeXKFRUtWlQlSpSwnDkFABgXn+QAALsLDQ1VhQoV1KpVKwUFBVk91+52lC1bVp999plmz56tpk2bau7cuZo1a5YefvhhS5mQkBANHTpULVu2lNls1ltvvWWvblj4+PioefPmat++vfz8/FSyZMm7mgLZsGFDjRkzRmPGjNGTTz6pgIAArVixQpKUnp6uyZMnq2nTpmrZsqUuXLigwYMHS7o2rjVr1pSvr69efPFFderU6Zbv0bFjR2VkZKhJkybq1q2bsrKyNGfOHLVs2VJNmzbV7t27OUMHAPcBh+w7vXIcAIB8sGTJEq1Zs+aWd6IEAAD/xRk6AEChcvjwYXl4eBR0MwAAMATucgkAKDReeuklnTp1yur5bwAA4NaYcgkAAAAABsWUSwAAAAAwKAIdAAAAABgUgQ4AAAAADKpQ3RTl3Lk0ZWVxSR8AAACAB4ujo4PKlHnotrcrVIEuKyubQAcAAAAANmLKJQAAAAAYFIEOAAAAAAyKQAcAAAAABpVnoJs4caL8/Pzk6empQ4cO3bSM2WxWZGSk/P391a5dO0VHR9u9oQAAAAAePEeOHFaHDm3VrJm3OnRoqz///CNHmcTERD37bA/5+DylFi0aKzp6sWVdUlKSwsO7WdYNGzZYmZmZkqQNG35Qu3Y+8vBw1ejRb1vVmdu6wiTPQNe2bVv9+9//VuXKlW9ZZuXKlTp+/LjWrl2rJUuWaPr06Tpx4oRdGwoAAADgwTN06GD16ROh7dv3qE+fCA0Z8nqOMqNHj1SDBt7atGmbli9frfHj39PJk9fyyCeffKjatT21adM2bdy4Tfv2/aJVq1ZIkqpVq66PPpquAQMG5agzt3WFSZ6BrnHjxnJ3d8+1TFxcnMLCwuTo6KiyZcvK399fq1evtlsjAQAAADx4kpKStG/fXoWGhkmSQkPDtG/fXp09e9aq3G+//So/P39Jkqurq+rVq6/ly7+TJDk4OCg1NVVZWVm6evWq0tMzVLFiJUlSjRo1Vb++l0ymnDf/z21dYWKXa+gSEhJUqVIly2t3d3edPn3aHlUDAAAAeECdOnVC7u7uMplMkiSTyaSKFSvq1Cnr2YBeXg313XdLlZ2drb/+OqadO+N14sRxSdIbbwzTn3/+oXr1aqtevdry9W2rpk2b3fO+5JdCFTfLlStZ0E0AAAAAUEiUKfOQTCZHlS9fyrLMZHJUmTIPWS2bOXOaBg8erHbtWqlq1ary9/dXqVIlVL58KcXELFSjRt7avHmjLl26pA4dOmjTpjXq1q2bZfuHHnJWdnaGVZ22rCsM7BLo3N3dderUKXl5eUnKecbOVsnJqfn6YPEjRw7r1Vdf1rlzKSpTpqxmzvxcNWrUsiqTmJiooUNf019//aXMzAy9/voQhYX1sKxfvjxGH300SdnZ2XJwcFB09ApVqFBBkyaN19y5X8rN7dr01CZNmmrixI8s7/vmm6/pwoULSk+/qpCQUA0b9la+9RMAAAC4HxQvXkYnTpzU6dPnZTKZZDabdfLkKRUvXkZJSZduKFlMH3/8meVVz55d1axZKyUlXdLUqZ9o6tSZSk5Ok+Sotm0DFRe3Rj4+AZbyaWlXdfly+v/Umfc6e3J0dLijE1x2mXIZGBio6OhoZWVlKSUlRevWrVNAQEDeG95jd3tB5S+/7NbkyR/om2+Wa/PmeK1cuUalS5e2bBsW1lMbNvyoDRt+tIQ5SYqMHKXg4BBt2PCj1qzZqMWL/63du3/O/w4DAADY4G7vIihdO+jt49NMrVs3lY9PM505c0bStbuhDx/+hp580ktNmjRQVNQ8yzZTpkxUq1ZN5OPzlPz9W2v9+nX521EYTvny5VWvXn3FxFy7i35MTLTq1/eSq6urVbmUlGTLnSu3bNmk//zngOW6u6pVq1n2rfT0dG3evFGPPfb4PexF/soz0I0dO1atW7fW6dOn9cILL6hTp06SpIiICO3fv1+SFBISIg8PD7Vv317PPPOMBgwYoCpVquRvy2+TPS6onDVrpvr3HyQ3NzdJUunSD6tYsWJ5vreDg4MuXrwoSbpy5YocHBzk6lrebn2D8d3tF+mkSeP1+OM15OvbQr6+LTR8+BtWdXfp0lG+vi3UokVjTZo03rLu8uXLioh4Xk2aNFDz5o20du33+dtRAEChlJ8Hvb/9domOHv1T8fG/KC7uB02e/IGOH/9LkuTt3Uhr1mzUpk3bNHXqTPXr94KuXLly7zoOQ5g8earmzPlczZp5a86czzV58lRJ187C/fLLbknSnj271KJFYzVv3kgTJ47TggVLVKJECUnS++9P0Pbt2+Tj00x+fi1Us2ZN9e79vCRp+/ZtatDgMc2aNVPz53+tBg0es4S/3NYVJg7Z2dn5N8fxNuXnlMu9e/fo1Vdf0pYtOyzLWrZ8Up9+OlteXg0tywYM6Kdy5VwVGTlOx4//pYCANgoNDdP48ZPl59dS7dq117ZtPyktLU2dOgVr8OChcnBw0KRJ4/Xvf8+Xi0sZVahQQcOGvaUnn2wqSfr77+Pq1au7zp1L0YUL5zVq1Pvq27dfvvQTxhQaGqSePXspLKyHoqMXa9GiKMXExFqVefnlPqpd21NvvjlcZ8+eVbt2rRUbu1aVK3to0qTxSktLU2TkuBx1P/tsT/n4tFHfvi8pNTVVrVs31ZdfztMTTzTWhx9O0KlTJ/XRR9P1559/KDg4UPHxv6hkSa5nBYAHRVJSkp566gn9/vsxy5Q2T8/q2r59j9VZkFatmmjatM/k7d1IktS7d3c99VRL9e8/UC+/3FetW7dReHjvHPWHh3dTz569FBzcRZI0YsSb8vCoqldffc2qXHZ2tmrVqqItW+JVqdKtH5cF3K8KdMrl/SQycrySks7I17eF3n57mFq1amO5VanZbNaBA78pOnq5li+P0w8//J+++WaRJOm55/rq55/3a9OmbRow4DU991xPpaQkS5Lmz/9aYWE9tG/f79qxY6++/HKWdu3aWWB9ROFij7PHucntDPHy5TF69tkXJEk1atRSw4beWr/+/+zWNwBA4WePuwgeOvS7/vrrqDp3DlTbtq0s9xuQpJMnT8jD478ztypXrpKjbklasmShqld/hDAH3KZCdZfL/FSpkocSEhJkNpstR59Onz6tSpU8rMq5urrqs8++tLzu2bOrPD19JUkeHh4KCgqRs7OznJ2dFRjYSXv27FL37uGWaZiS1KaNnypV8tDBg/9R8+YtNXv2LO3cuU+S5OZWUS1b+mjbtp/UqNGT96DnKOxy+yK98cjo9S/Shg2f0PHjf2nnznhVqVLVsn7ZsqXauHF9jjPEY8dOUK9e3fX1119azhBXrVpNknTixAl5ePy3jsqVPXTy5Ml70W0AgMFERo7Xu++OkK9vC3l4eNzyoHdGRrq6dw9V5coe6t493Ka6f/ppqyZOHKfo6OX52QUUkNIuxeVc9IGJHbm6mpGpi+ftO634gRnZGy+oDAvrkesFlaVLP6wiRYpYLqicM2eBpGtnTtatW6tnnumpzMxMbdmy0TJ9ICHhlNzdr93Zc//+ffr7779Us2ZtSVK1atW0fv3/qXv3cKWmXlJ8/E8KDOxwD3uP+0FuX6TPPddXgwcPVdGiRbVx43o991xPbd26U2XLlrOcIX711deUmHhaXbp0VMOG3hxQAABIyv+D3pUre+jEib8tUzVPnvzb6mDizp3x6t8/QvPnL1KtWrXvQY9xrzkXLaI3vttU0M0oFD562sfudT5QUy7v9oLKp5/uJlfX8mrZ8kn5+bWQp2cd/etfz0qSxo2LVOvWTdWmTXO9+eZAzZz5heWs3bRpn2nevK/Upk1zBQb6qXPnp9W2bfsCGAEURjd+kUrK84t048afFBX1jVJTL8nT8zFJkpubm4oWLSrJ+gyxJM2ePctyhPTGM8TStS/g69NlpGvTYipXZqoLADxI7HEXwdDQMG3cuF7Z2dnKyMjQli0bVbdufUlS585Pa8GCucrKytLZs2f1/ferFBwcIuna3139+r2gOXPmW93TAIDtHpibogCFWZcuHfWvfz1ruSnKwoUL9N13q6zK/O/Z44EDX9ZPP+1SiRIlcpwh7tYtWJs375Cbm5t8fJqpf/9BljPEHTv6a/To99W2bXtNmjRep08nWG6KEhQUoB07flHJkoXzwZm2uttnTi5aFKVZs2bK0dFRWVlm9er1nCIiXslzu6SkJL322is6efKkMjMz1KJFK40fP1lFijwwkyEAGNThw4c0cOBLOn/+vFxcXDRjxheqVau2evbsquHD31bDhk/ohx/W6q23hslkMqls2XL64IMPVb/+tWcQZ2VlafTot7V+/f/J0dFRbdq0VWTkODk6OspsNmvEiCHauPEHSdLAgYMt12+3b++jv/8+rooV//v84pkzv9Djj9e994OAfFO+fCnO0P1/Hz3tc8vn2d3pTVEIdEAhcLdfpK+++pL27ftFjo4mOTkV1bBhb8nf/9qzIPfu3aORI4fq8uXLyszMUJcuXTVkyAhJUlpamgYNekX79++VyWTSqFHvq0OHTgU2DvZyt3cNvXTpokqWLCUHBwelpl5S69bNtGDBEtWtWy/X7d55Z7hMpiKKjBynjIwMBQe31yuvDFRISGgBjQQAAAWPQPdf+RHoDHPYuFTpYirmXLSgm1Eo/HM1Q5cu/lPQzYAd1a79qFav3pBj+aJFSy0/t23bXvHxN5+qO2PG57esu0EDb8XF3fyZKQ899JDmzJl/m60t3K7fNfT6hfWhoWEaOXKozp49azV96LffftVLLw2QZH3X0P79B6pUqdKWcpcvX1FGRoYcHBzy3O5aAExVVlaWrl69qvT0DKujzgCAB0N+zhS57o8/Dqtt25Z6/vkXLY8tio5erBkzPtGhQwc1duwE9e370r3pMAqUYQJdMeeiCh/274JuRqGwcNK/dEkEOuBm7HXX0NWr4zRu3BgdO3ZUb7892jL9J7ft3nhjmPr06a169Wrr8uXL6tu3n5o2bXYPew/gQfXww8Xk5MSBb0lKT8/QhQsF+3fS9Qe1X58pMmTI6zlmilx/UPv8+YstMz6aN2+pypU9FBTUWT16/Mtqpkjz5q1Ut249Sdeutx8y5LUcs2rq1fPSF198rWnTPrpnfUXBM0ygAwqz0g87y9nJqaCbUShcTU/XxQtXC7oZecrtrqGSFBjYUYGBHXXixN967rlw+fsHqFat2rlut2LFMj3+eF0tXbpSqamX1KNHV61cucxyN1wAyC9OTkU1ZcqUgm5GofDmm29KBXjgO79nikjStGkfqV27QKWlpSotLc2yvE6dxyVJjo4P1H0PH3gEOsAOnJ2c9PzXrxV0MwqFuS98IqngAp09br99Iw+PKvL2bqS1a1erVq3auW43Z87nmjr12hSZ0qUfVmBgJ23duplABwAPkPyeKfLrr/u1YcMP+u67VZoyZeK97RwKJeI7gPuKPW6/fejQ75ZyycnJ+vHHzZajnrltV7VqNa1ff+16xfT0dG3evFGPPfZ4PvYWAGBUkZHjlZR0Rr6+LfT228NuOlNky5Yd2rZtt6Kjl+iPPw4rIyNDQ4YM0uTJUy2BEeAMHYD7zuTJUzVw4EuaMmWi5a6hkqzuGrpnzy6ru4be+MzJBQu+1saN61WkSFFlZ2erb99+8vVtK0m5bvf++xM0dOhg+fg0k9lsVosWrdS79/MFMgYAgIKRnzNFOnfuomPHjio8vJsk6cKFC8rOzlZq6iVNmTItfzuGQotAB+C+c7d3DX3//Qm3rDu37R55pIa+/Xb5bbYWAHA/uXGmSFhYj1xnitz4fNn//OeA5sxZIOnaTJFHH/WU9N+ZIp06BcvDo4oOHjxmqWPSpPFKS0uz3OUSDyYCHQAAAGBH+TlTJDcxMdGKjHxXFy6c1+rVcZo27WN9880yeXo+lq/9RcEi0AEodFxKOaloMeeCbkahkPHPVZ2/lF7QzQAA3Ib8nClyo2HD3rJ6HRoaZrmuGw8OAh2AQqdoMWfFPftCQTejUOg4/2uJQAcAAG6BQAcAAADcoIyLs4oU5fmykpSZka5z5wv/82UfZAQ6AAAA4AZFijppc+yYgm5GodA6aIwK8vmyyBvPoQMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUEVsKXT06FGNGDFC58+fl4uLiyZOnKjq1atblUlOTtbIkSOVkJCgzMxMNW3aVO+8846KFLHpLQAAAAAAt8mmM3SjR49WeHi41qxZo/DwcI0aNSpHmVmzZqlmzZpauXKlVqxYod9++01r1661e4MBAAAAANfkGeiSk5N14MABBQUFSZKCgoJ04MABpaSkWJVzcHBQWlqasrKylJ6eroyMDLm5ueVPqwEAAAAAeQe6hIQEubm5yWQySZJMJpMqVKighIQEq3L9+/fX0aNH1bJlS8u/Ro0a5U+rAQAAAAC2XUNni9WrV8vT01Pz5s1TWlqaIiIitHr1agUGBtpcR7lyJe3VnPte+fKlCroJwC2xf9oX4wkAt4fPTftiPO3L3uOZZ6Bzd3dXYmKizGazTCaTzGazzpw5I3d3d6tyUVFRGj9+vBwdHVWqVCn5+fkpPj7+tgJdcnKqsrKyb7qOHclaUtKlgm4CbsD+ae1u90/G0xq/7wDywuemNb6H7IvxtK9bjaejo8MdneDKc8pluXLlVKdOHcXGxkqSYmNjVadOHZUtW9aqnIeHhzZv3ixJSk9P17Zt21S7du3bbhAAAAAAwDY23eVyzJgxioqKUkBAgKKiohQZGSlJioiI0P79+yVJb731lnbt2qXg4GB16dJF1atX1zPPPJN/LQcAAACAB5xN19DVrFlT0dHROZbPnj3b8nPVqlX19ddf269lAAAAAIBc2XSGDgAAAABQ+BDoAAAAAMCgCHQAAAAAYFAEOgAAAAAwKAIdAAAAABgUgQ4AAAAADIpABwAAAAAGRaADAAAAAIOy6cHiAIAH05Ejh/Xqqy/r3LkUlSlTVjNnfq4aNWpZlRkwoJ8OHPjN8vrAgV81b94iBQZ21JQpE7Vs2VI5OppUtGhRvfXWKPn5+UuSunbtrJSUZEmS2Zypgwf/ow0bflLduvV05Mhhvfnma7pw4YLS068qJCRUw4a9de86DgCAQRDoAAC3NHToYPXpE6GwsB6Kjl6sIUNeV0xMrFWZmTO/sPz866/71bVrkHx920qSvL0b6ZVXBqpEiRL69df96tKlo/bvP6TixYtr6dIVlu3i4mI1YcL7qlu3niQpMnKUgoND1LfvS0pNTVXr1k3l799eTzzR+B70GgAA42DKJQDgppKSkrRv316FhoZJkkJDw7Rv316dPXv2ltssXDhfXbs+I2dnZ0mSn5+/SpQoIUmqW7eesrOzde5cSo7tFi1aoJ49e1teOzg46OLFi5KkK1euyMHBQa6u5e3WNwAA7hcEOgDATZ06dULu7u4ymUySJJPJpIoVK+rUqRM3LZ+enq6YmGirYHajJUsWqnr1R1SpUmWr5YmJidq8eaPCwnpYlo0dO0HLlsXIy8tTjRvXU//+g1S1ajU79QwAgPsHgQ4AYBfffx+rypWrqH59rxzrfvppqyZOHKfPP/8qx7pvvlkkX19/ubq6WpbNn/+1wsJ6aN++37Vjx159+eUs7dq1M1/bDwCAERHoAAA3VamShxISEmQ2myVJZrNZp0+fVqVKHjctv3DhAoWH98qxfOfOePXvH6F58xaqVq3aOdYvXhyVY7vZs2epe/dwSZKbW0W1bOmjbdt+utsuAQBw3yHQAQBuqnz58qpXr75iYqIlSTEx0apf38vqTNp1p06dVHz8NnXt+ozV8j17dqlfvxc0Z858eXk1zLHdjh3xunjxotq2bW+1vFq1alq//v8kSamplxQf/5Pq1Kljr64BAHDfINABAG5p8uSpmjPnczVr5q05cz7X5MlTJUk9e3bVL7/stpRbsmSh2rcPlItLGavthw9/Q//8c0VDhrwuX98W8vVtYfWIg8WLo/TMMz0t1+ldN23aZ5o37yu1adNcgYF+6tz56RyhDwAA8NgCAEAuatd+VKtXb8ixfNGipVavBw8eetPt167dlGv9H300/abLGzTwVlzcOhtbCQDAg4szdAAAAABgUJyhA4D72MOli8vJmY96SUq/mqkLF68UdDMAALArvuUB4D7m5FxE49/+tqCbUSi8Na5bQTcBAAC7Y8olAAAAABgUgQ4AAAAADIpABwAAAAAGRaADAAAAAIMi0AEAAACAQRHoAAAAAMCgCHQAAAAAYFAEOgAAAAAwKAIdAAAAABgUgQ4AAAAADIpABwAAAAAGRaADAAAAAIMZtZ4OAAAgAElEQVQi0AEAAACAQRHoAAAAAMCgCHQAAAAAYFAEOgAAAAAwKAIdAAAAABgUgQ4AAAAADIpABwAAAAAGRaADAAAAAIMi0AEAAACAQRHoAAAAAMCgCHQAAAAAYFAEOgAAAAAwKAIdAAAAABgUgQ4AAAAADIpABwAAAAAGRaADAAAAAIMi0AEAAACAQRHoAAAAAMCgCHQAAAAAYFAEOgAAAAAwKAIdAAAAABgUgQ4AAAAADMqmQHf06FF1795dAQEB6t69u44dO3bTcnFxcQoODlZQUJCCg4N19uxZe7YVAAAAAHCDIrYUGj16tMLDwxUSEqLly5dr1KhRmj9/vlWZ/fv3a8aMGZo3b57Kly+vS5cuycnJKV8aDQAAAACw4QxdcnKyDhw4oKCgIElSUFCQDhw4oJSUFKtyc+fOVZ8+fVS+fHlJUqlSpeTs7JwPTQYAAAAASDYEuoSEBLm5uclkMkmSTCaTKlSooISEBKtyR44c0d9//61//etfevrpp/Xpp58qOzs7f1oNAAAAALBtyqUtzGazfv/9d3399ddKT0/Xiy++qEqVKqlLly4211GuXEl7Nee+V758qYJuAnBL7J/2xXjaD2MJPBj4XbcvxtO+7D2eeQY6d3d3JSYmymw2y2QyyWw268yZM3J3d7cqV6lSJQUGBsrJyUlOTk5q27at9u3bd1uBLjk5VVlZNz+rx45kLSnpUkE3ATdg/7R2t/sn42ntbsaTsbTGZyfuV/yuW+N7yL4YT/u61Xg6Ojrc0QmuPKdclitXTnXq1FFsbKwkKTY2VnXq1FHZsmWtygUFBWnr1q3Kzs5WRkaGtm/frscee+y2GwQAAAAAsI1Njy0YM2aMoqKiFBAQoKioKEVGRkqSIiIitH//fklSp06dVK5cOXXs2FFdunRRrVq11K1bt/xrOQAAAAA84Gy6hq5mzZqKjo7OsXz27NmWnx0dHTVy5EiNHDnSfq0DAAAAANySTWfoAAAAAACFD4EOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQBDoAAAAAMCgCHQAAAAAYFIEOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQBDoAAAAAMCgCHQAAAAAYFIEOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQBDoAAAAAMCgCHQAAAAAYFIEOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQBDoAAAAAMCgCHQAAAAAYFIEOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQBDoAAAAAMCgCHQAAAAAYFIEOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQBDoAAAAAMCgCHQAAAAAYFIEOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQNgW6o0ePqnv37goICFD37t117NixW5b9888/1aBBA02cONFebQQAAAAA3IRNgW706NEKDw/XmjVrFB4erlGjRt20nNls1ujRo+Xv72/XRgIAAAAAcsoz0CUnJ+vAgQMKCgqSJAUFBenAgQNKSUnJUfaLL75QmzZtVL16dbs3FAAAAABgLc9Al5CQIDc3N5lMJkmSyWRShQoVlJCQYFXu4MGD2rp1q55//vl8aSgAAAAAwFoRe1SSkZGhd999Vx988IEl+N2JcuVK2qM5D4Ty5UsVdBOAW2L/tC/G034YS+DBwO+6fTGe9mXv8cwz0Lm7uysxMVFms1kmk0lms1lnzpyRu7u7pUxSUpKOHz+ufv36SZIuXryo7Oxspaam6v3337e5McnJqcrKyr7pOnYka0lJlwq6CbgB+6e1u90/GU9rdzOejKU1Pjtxv+J33RrfQ/bFeNrXrcbT0dHhjk5w5RnoypUrpzp16ig2NlYhISGKjY1VnTp1VLZsWUuZSpUqKT4+3vJ6+vTpunz5soYPH37bDQIAAAAA2Mamu1yOGTNGUVFRCggIUFRUlCIjIyVJERER2r9/f742EAAAAABwczZdQ1ezZk1FR0fnWD579uyblh84cODdtQoAAAAAkCebztABAAAAAAofAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBEegAAAAAwKAIdAAAAABgUAQ6AAAAADAoAh0AAAAAGBSBDgAAAAAMikAHAAAAAAZFoAMAAAAAgyLQAQAAAIBBFbGl0NGjRzVixAidP39eLi4umjhxoqpXr25VZubMmYqLi5Ojo6OKFi2qwYMHq1WrVvnRZgAAAACAbAx0o0ePVnh4uEJCQrR8+XKNGjVK8+fPtyrj5eWlPn36qHjx4jp48KB69eqlrVu3qlixYvnScAAAAAB40OU55TI5OVkHDhxQUFCQJCkoKEgHDhxQSkqKVblWrVqpePHikiRPT09lZ2fr/Pnz+dBkAAAAAIBkQ6BLSEiQm5ubTCaTJMlkMqlChQpKSEi45TbLli1T1apVVbFiRfu1FAAAAABgxaYpl7djx44d+uSTT/TVV1/d9rblypW0d3PuW+XLlyroJgC3xP5pX4yn/TCWwIOB33X7Yjzty97jmWegc3d3V2Jiosxms0wmk8xms86cOSN3d/ccZffs2aOhQ4fq008/VY0aNW67McnJqcrKyr7pOnYka0lJlwq6CbgB+6e1u90/GU9rdzOejKU1Pjtxv+J33RrfQ/bFeNrXrcbT0dHhjk5w5Tnlsly5cqpTp45iY2MlSbGxsapTp47Kli1rVW7fvn0aPHiwpk2bprp16952QwAAAAAAt8em59CNGTNGUVFRCggIUFRUlCIjIyVJERER2r9/vyQpMjJS//zzj0aNGqWQkBCFhITo999/z7+WAwAAAMADzqZr6GrWrKno6Ogcy2fPnm35eenSpfZrFQAAAAAgTzadoQMAAAAAFD4EOgAAAAAwKAIdAAAAABgUgQ4AAAAADIpABwAAAAAGRaADAAAAAIMi0AEAAACAQRHoAAAAAMCgCHQAAAAAYFAEOgAAAAAwKAIdAAAAABgUgQ4AAAAADIpABwAAAAAGRaADAAAAAIMi0AEAAACAQRHoAAAAAMCgCHS4I0eOHFaHDm3VrJm3OnRoqz///CNHmQ0bflC7dj7y8HDV6NFvW60zm80aPvwNPfmkl5o0aaCoqHmWdYsWRcnH5yn5+raQj08zzZ79mWXdlCkT1apVE/n4PCV//9Zav35d/nUSAAAAKOQIdLgjQ4cOVp8+Edq+fY/69InQkCGv5yhTrVp1ffTRdA0YMCjHum+/XaKjR/9UfPwviov7QZMnf6Djx/+SJAUFddbGjT9pw4YftWrV/+mzz2bot99+lSR5ezfSmjUbtWnTNk2dOlP9+r2gK1eu5G9nAcBObDkYltsBr8TERD37bA/5+DylFi0aKzp6sWVdUlKSwsO7WdYNGzZYmZmZedYJADA2Ah1uW1JSkvbt26vQ0DBJUmhomPbt26uzZ89alatRo6bq1/eSyVQkRx3Ll8eod+/n5ejoKFdXV3Xo0EkrViyTJJUqVVoODg6SpMuXrygjI8Py2s/PXyVKlJAk1a1bT9nZ2Tp3LiXf+goA9mTLwbDcDniNHj1SDRp4a9OmbVq+fLXGj39PJ0+ekCR98smHql3bU5s2bdPGjdu0b98vWrVqRZ51Gll+BuQBA/rJ17eF5Z+b28NavTrOqu4//jisatXccsxCAYB7iUCH23bq1Am5u7vLZDJJkkwmkypWrKhTp07YXMfJkyfk4VHF8rpy5SpW269eHadWrZqoUaO6GjBgkB5/vG6OOpYsWajq1R9RpUqV76I3AHBv2HowLLcDXr/99qv8/PwlSa6urqpXr76WL/9OkuTg4KDU1FRlZWXp6tWrSk/PUMWKlfKs08jyMyDPnPmFNmz4URs2/Kjp02fJxcVFvr5tLfWazWYNGfKaOnTodG86CwC3QKBDoRQY2FFbtuzQtm27FR29RH/8cdhq/U8/bdXEieP0+edfFVALAeD22HowLLcDXl5eDfXdd0uVnZ2tv/46pp0743XixHFJ0htvDNOff/6hevVqq1692vL1baumTZvlWadR5XdAvtHChfPVteszcnZ2tiybNu0jtWsXqBo1auVXFwHAJgQ63LZKlTyUkJAgs9ks6dpRytOnT6tSJQ+b66hc2UMnTvxteX3y5N833d7Do4q8vRtp7drVlmU7d8arf/8IzZu3ULVq1b6LngCAsURGjldS0hn5+rbQ228PU6tWbSzT2lesWKbHH6+rX389rH37Dmrbth+1cqXxz8LdSn4H5OvS09MVExOtnj17W5b9+ut+bdjwg15+eUB+dQ8AbEagw20rX7686tWrr5iYaElSTEy06tf3kqurq811dO78tBYsmKusrCydPXtW33+/SsHBIZKkQ4d+t5RLTk7Wjz9uVp06j0uS9uzZpX79XtCcOfPl5dXQjr0CgPxl68Gw3A54ubq66rPPvtTGjT8pKuobpaZekqfnY5KkOXM+V9euz8jR0VGlSz+swMBO2rp1c551PshyC8jXff99rCpXrqL69b0kSRkZGRoyZJAmT55qCZMAUJAIdLgjkydP1Zw5n6tZM2/NmfO5Jk+eKknq2bOrfvlltyRp+/ZtatDgMc2aNVPz53+tBg0eszxmICysh6pVe0RNmzZUhw5+evPN4apWrbokacGCr9WqVRP5+rZQ167B6tu3n+W6heHD39A//1zRkCGvWy5UP3Dgt3s/AABwm2w9GJbbAa+UlGTLnSu3bNmk//zngGXKYdWq1Syfsenp6dq8eaMee+zxPOs0qvwOyNctXLhA4eG9LK8TE0/r2LGjCg/vpkaN6umLLz5TVNQ8vflmzjs6A8C9kPP2g4ANatd+VKtXb8ixfNGipZafmzV7Snv3Hrzp9iaTSZMnf3zTde+/P+GW77t27abbbCkAFB6TJ0/VwIEvacqUiXJxcdGMGV9IunYwbPjwt9Ww4RMKC+uhXbt+VtOm12Yh3HjAa8+eXXrrrWEymUwqW7acFixYYrnz7/vvT9DQoYPl49NMZrNZLVq0Uu/ez0tSrnUa1Y0BOSysR54BuVOnzkpJSdH336/SihXXpvGnpCSrdOmHVaRIEUtAnjNngWXbU6dOKj5+m9X12h4eVXTw4DHL60mTxistLU2RkePyt8MAcAsEOgAA7hFbDobldsCrbdv2io9vf9N1jzxSQ99+u/ym63Kr08jyMyBL1+6m3L59oFxcytzzvgGArQh0D6gyDzupiJNz3gUfAJnpV3XuQnpBNwMAcJvyMyBL0uDBQ/Nsw7Bhb9nQUgDIPwS6B1QRJ2ftmvRiQTejUGg07EtJBDoAeXu4tJOcnDkYJknpV6/qwsW7++ws83BxFXHiTxFJykzP1LkLVwq6GQAMiE9RAABs5OTsrI9GvlTQzSgU3vjgc93twbAiTkW099ONdmmP0TXo36agmwDAoLjLJQAAAAAYFIEOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQBDoAAAAAMCgCHQAAAAAYFIEOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQBDoAAAAAMCgCHQAAAAAYFIEOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6AAAAADAoAh0AAAAAGBQBDoAAAAAMCgCHQAAAAAYlE2B7ujRo+revbsCAgLUvXt3HTt2LEcZs9msyMhI+fv7q127doqOjrZ3WwEAAAAAN7Ap0I0ePVrh4eFas2aNwsPDNWrUqBxlVq5cqePHj2vt2rVasmSJpk+frhMnTti9wQAAAACAa/IMdMnJyTpw4ICCgoIkSUFBQTpw4IBSUlKsysXFxSksLEyOjo4qW7as/P39tXr16vxpNQAAAABARfIqkJCQIDc3N5lMJkmSyWRShQoVlJCQoLJly1qVq1SpkuW1u7u7Tp8+fVuNcXR0yHW9a5mHbqu++1leY2ULp9Ll7NCS+4M9xtO1ZNm8Cz0g7DGexV3ZP6+72/F82KWEnVpifPbYN0u7sG9eZ4/xLFqqmB1acn+42/EsXbq0nVpifPbYN52Lu9ihJfcHe4xnmRLOdmjJ/eFW43mn4+yQnZ2dnVuBX3/9VcOHD9eqVassyzp27KjJkyerbt26lmXBwcEaN26cvLy8JEmzZ89WYmKi3nnnnTtqGAAAAAAgd3lOuXR3d1diYqLMZrOkazc/OXPmjNzd3XOUO3XqlOV1QkKCKlasaOfmAgAAAACuyzPQlStXTnXq1FFsbKwkKTY2VnXq1LGabilJgYGBio6OVlZWllJSUrRu3ToFBATkT6sBAAAAAHlPuZSkI0eOaMSIEbp48aJKly6tiRMnqkaNGoqIiNCgQYNUv359mc1mvffee/rxxx8lSREREerevXu+dwAAAAAAHlQ2BToAAAAAQOFj03PoAAAAAACFD4EOAAAAAAyKQAcAAAAABkWgAwAAAACDItABAAAAgEER6Gy0bt06Pf300woMDJS/v78mTJig9PT0PLeLj4/X1q1b70ELC8aFCxfk5eWlsWPHWpbFxMRo0KBBOcreanleMjIyNH36dAUEBKhTp07q3LmzBg0apD/++OOu2l6YpKena8KECfL391dgYKC6dOmidevWSbq2D4WGhlqVP3TokPz8/HLUM2PGDHl6eurQoUNWy48cOSJPT0/NnTvXavnJkyfVr18/BQcHKzg4WKGhoTm2vZ/87ziHhITo+++/l3RtnBs0aKCQkBDLv2HDhlnW3fh/4OnpqbS0NElS7969tWHDhnvfmXvEz89PgYGB6ty5s9q1a6dXXnlFu3fvtqyfOXOm/P395e/vr5kzZ9pc77Zt2xQaGqpOnTqpU6dOOnjwYH403xCmT5+e5/dJ79691bZtW6v98/r/g5+f3y1/b3Nbdz+53s8RI0aodevWCgkJUUBAgD788ENLGU9PTwUHB6tz584KDg7WDz/8YFk3ffp0TZw48aZ1x8TEyNPTU3FxcVbL7uT77EF14/j+72dtWFhYAbfu3rrx+yMiIkLHjx+XJI0YMUJRUVGSru1fjRs3VkhIiDp27Kh+/fopKSnJqh6z2axWrVrplVdeubcdKGQSExPVu3dvSdL58+fVunVr7du3z7J+1qxZGjhwoK5cuaJ69erp7NmzlnWhoaFWv8f79++Xj4+PpGv7rKenp/bu3WtZn9vnREErUtANMIKdO3fqvffe0+zZs+Xp6amrV69q+PDhioyM1Lhx43LddseOHbp8+bJatmx5j1p7b8XGxqpBgwZatWqVhg0bJicnJ7u/x8iRI/XPP/8oOjpapUuXVnZ2tjZt2qSjR4+qVq1adn+/gjBmzBhdvnxZq1atkrOzsw4dOqQXX3xRDz/8sM11/Pbbb/rll19UuXLlHOu+/fZbNWvWTEuXLtXzzz9vWR4ZGanWrVurV69ekq59MBYpcv9+LNxsnPv27SsXFxc5OjqqZs2aiomJKehmFjrTpk3To48+Kklau3at+vXrpzlz5ig9PV2rV69WbGysJCksLExNmjTRk08+mWt9iYmJevvtt/Xll1+qRo0a+ueff5SZmZnv/SisZsyYoT59+uT5+fnOO+/I19f3HrXKuPr166devXrp0qVLCgkJkbe3t9q2bStJWrx4sR566CFt2rRJr7/+unbu3GnTZ17lypX1ySefqH379vf1Z+S9wmftNbNnz77luubNm2vatGnKzs7WG2+8oRkzZigyMtKyfsuWLapQoYJ2796ts2fPytXV9V40udBxc3PTggULJEkuLi4aNWqURo4cqe+++05Hjx7Vv//9by1btkzFixeXl5eXduzYoY4dOyo1NVX//POP1QGvHTt2qEmTJpbXlStX1pQpUzR//vx73q/b9cCdofv00081fvx4y+tz586padOmGjRokOXIiGR9pGT69Ol65ZVX5OnpKUlydnbWmDFjFBcXp5MnT+rIkSPy8fHRyZMnJV37ch48eLB+//13LV68WMuWLVNISIi++OKLe9jTe2Pp0qXq37+/PD09rY525iUhIUGhoaGWI55r1qyxnJmaNWuW5QjWsWPHtG7dOo0bN06lS5eWJDk4OKhNmzZq165dvvTpXjt58qS+//57jRkzRs7OzpKkRx99VC+//LJmzJhhUx3p6el67733NGbMmBzrMjMztXLlSr333nu6evWq1ZGr06dPy83NzfLazc1N5cqVu7sOFVK3GudXXnnF5nGG1L59e/Xo0UNz5sxRXFycunTpomLFiqlYsWLq0qWL5Xf60qVLGjRokAIDA9W7d28NHTrUcmRz4cKFCgkJUY0aNSRJxYoVU8mSJSVJrVq1UnJysqRrR6/79esnSUpOTlbr1q3vdXfz3fU/0Hr06KGQkBC1a9fOcuaiRYsWGjBggE31rFixQqGhoWrXrp3Vd9mDrFSpUqpfv76OHj2aY13Tpk11+fJlXbx40bIsMTFRERERCgwMVL9+/XTlyhXLunr16umRRx7Rt99+e0/aXhisXbvWMpNh5syZlu/lW31f3+rvq8uXLxdgLwo3W86gOzg46Mknn1RCQoLV8qVLl6pHjx7y9/fXsmXL8rOZhcbevXvVu3dvhYaGKjQ0VBs3btSJEyfUtGlTSxl/f3/VqVNHH374oUaMGKGRI0da/q5p0qSJ4uPjJUm7du1S48aNVb16dR0+fFjStUB3Y13t27fX+fPntWXLlnvYyzvzwAW6639wXD8aHBsbKz8/P5UoUeKW2/z+++9q2LCh1TIXFxdVqVJFhw4dUs2aNTV48GANHjxYW7du1cqVK/X+++/L09NTPXr0UJcuXbR8+XLLHyb3i4MH/1979x5TZf0HcPx9BghMEMXCJBHyBqHjYigHZxuB0yMUcEjLrHCRKIph0ooNEHA1WDlNKi6jDLUtGnIOHMfqlEXguMoERcd+GDZMUmSMWBgSxyO/P8545iPgJTPh8H1t/PPceJ4vj8/39vl8/R+9vb0olUoiIyPRaDT3fN6WLVtISkoiJCSE7u5uUlNTycvLo7S0FBsbG+nYlpYWXF1d72umaqI5f/48c+fOZfr06bLtPj4+UhjahQsXZKFWt4f6ZGVlERYWxpw5c0Zcv6KiAldXV1xdXVGr1bK/0+bNm3nvvfd47bXX2Lt3r6yzZ27uVM7DFert5Sw6eqPz9vamra2NK1eu4OzsLG2fPXu21OjIzs5m6tSp6PV6srKyaGhokI5ra2vjr7/+IioqioiICDIzM6WQQ39/f+rq6jAYDHR0dNDR0YHBYKC2tlZW0ZqLtLQ0wDRzpNPpOH78ODqdjtzcXGxtbYmJiZGO/eCDD2Tv53DHF0wdXq1WS2FhIXl5eZM6hHXY1atXaWxsxNPTc8S+48ePo1QqcXR0lLadO3eOffv28d1330kDYbfatWsXubm5DAwMPPR7f9S6u7vZvXs3OTk56HQ6afa4t7d3zPr6ftpX7e3tqNVq1q9fT0lJyX/zUBPU4OAgJ06cICQkRNrW09NDXV0da9euJTIyclLMdv7555+kpaWxb98+tFoteXl5pKam0tfXN+LY3bt3U1xczJNPPikrN39/f06ePAmYQn+XL1+On58fJ0+exGg0curUKVk9o1AoSEhI4OOPP2ZoaOjhP+QDmHQdOmdnZxYsWEBlZSUAJSUlI/KT/omIiAjmzZtHXFwc+/btk0abzVlxcTHh4eEoFApWr15Nc3MzV69eveM5ra2t7NixgwMHDuDn5weYRlw8PT1xc3MD4MUXXxzz/La2Nik34ta8vYnsXj4S8+fPR6fTST+ffPKJtK+pqYlz586xcePGUc/VaDSo1WrA9J7q9Xr+/vtvAMLCwigvL2fjxo1cv36dTZs2SeFz5uZO5axQKICR5bxjx47/6vYmlHt5Z+vr61m3bh0Ajo6Oshl1o9FIY2Mj2dnZFBUVceXKFSmCISAggJqaGs6cOYOPjw9eXl6cOXOGmpoalErlw3mgcaavr4/Y2Fjeffdd2WBiSkqK7P28dTZ9uKwfe+wxAgMDpUbLZJSfn094eDjbtm1j8+bNrFixQtq3YcMGgoKCSE5O5u2335adt3LlSqZNm4ZCocDLy0vKbRrm7u7OsmXLpPAuczZcLw/Por/88suAaUB2rPr6XttXixcvprKykpKSEvbv3092djY1NTUP+YkmnpqaGsLDwwkICKCnp4e1a9dK+44dO8Zzzz2HnZ0dzzzzDEajkaampkd4tw9fU1MTHR0dxMTEEB4eTkxMDAqFYtRw/draWuzs7Pj1119l+cm+vr50dHTQ3d1NQ0MDy5cvl2btWlpasLe3x8XFRXatwMBArK2tpXz78WrSdegA1Go1paWltLAYJCIAAAefSURBVLa20tfXh5+fHxYWFty8eVM6ZrjBC6aP+OnTp2XX6O3t5dKlSyxcuBAwjaD88ssv2Nvby0ZNzdXg4CBlZWVoNBqCgoIICQnBYDDcdZRo1qxZODg4SFPed+Pp6cnFixelsJgFCxag0+l4/fXXuXbt2gM/x3iwaNEifvvtN3p7e2XbT58+LYX53klDQwMXLlwgODiYoKAgOjs7efPNN6mqqqK7u5uqqio+++wzgoKCePXVV7l+/Trff/+9dP6MGTMICQkhNTWVbdu2mW2H7k7l7Ovr+4juamI6e/YsCxcuZPbs2Vy+fFnafuXKFWbPnn3X852dnQkMDMTe3p4pU6agUqk4e/YsAEqlktraWmpra1EqlSiVSurq6qirqyMgIOChPdN4YTAYeOutt1Cr1axZs+ZR386EtGXLFnQ6HVqtlk2bNsn2ffPNN5SXl7Nz504SEhJkdf1wKDaAhYUFRqNxxLV37tzJoUOHRp0VEEZvX93Ozs4Oe3t7AFxcXFi1apVsoSXBZMWKFeh0OiorK1EoFGRlZUn7NBoN1dXVBAUFERQURE9Pzz1HSU1UQ0NDuLu7ywa1KisrmTFjhuy4np4eMjIyyM/PZ8mSJbIBcBsbG7y9vfn555/p7+/HyckJT09PWlpaRuTP3eqdd94hKytrXOd6T8oO3erVq2loaKCgoAC1Wo1CocDV1VVqUHR1dck6HHFxceTm5tLa2gqYOnvp6emoVCopxO2jjz5i8eLFFBQUkJaWRmdnJ2D6cJnjh/+nn37iqaee4sSJE5SXl1NeXs6XX35519CJ6dOnc+jQIY4dO0ZBQQFgCt9qaWmRRkNvvYabmxvBwcGkpKTIytGcYvLnzJmDSqUiPT1dalycP3+evLy8e5oh2rJlC1VVVdLf4YknnuDgwYOsXLmS0tJS1qxZQ0VFhbQ/IyND+vBXVFRIv9NoNNLa2jpq2KY5GKucDx8+PGKkXhjbjz/+SGFhIdHR0ahUKkpLSxkYGGBgYIDS0lJpFFmpVEoDPH/88Ye0aivA888/T319PYODgwwNDVFVVYWHhwdgSkK3sLCgpKSEgIAAAgIC0Gq1WFpaysI7zcnUqVOlAarU1FTmzZvHG2+8cV/XGP5u9vT0UFlZaZbhqf+m6OhoZs6cSWFh4X2d5+Liwpo1azh8+PBDurPxwcfHh5aWFtrb2wE4evQoAB4eHmPW1zB6++p2XV1d0ix/b28v1dXV0r9/YSQ7Ozv27NlDYWEhXV1dNDc309fXJ6v3y8rK0Ov1srxPc+Pr68vFixepq6uTtjU3N4+IGNmzZw8vvfQSHh4eJCcnU1ZWJrXvwZRH98UXX7B06VIALC0tmTt3LkVFRWN+N4dz7W4Pwx5PJuVSTba2tgQHB6PVaqWFPNavX098fDwhISG4ubnh5eUlHe/v709KSgqJiYkMDAxgMBgIDg4mISEBMDVwTp48ydGjR7G2tiYuLo6EhASOHDkiJauGh4cTGhpqNnl0Go2GF154QbbN19eXmzdvcvnyZSorK2ULGERGRjJ37lzAlKh+8OBBYmNj6e/vJy4ujvT0dGJiYrC1tSUwMBArKytsbW0ByMzMJCcnh3Xr1mFpacm0adNwcnIym7IEUx7N/v37CQkJwcrKCmtra5KTk2UJvP+EVqslMTFRti04OJi0tDQ6Ojqor6/nww8/xNLSEqPRyJIlS9i5c+eDPs64dWs5KxQKrl69SlFREU8//TT19fVSDt0wJyenO65CNlnEx8czZcoUrl+/zvz588nPz8fb2xswNeBCQ0MBU0jv8Ajn9u3bSUpKQqVS8fjjj8tG6pcuXcqzzz5LREQEFhYWeHp6snXrVml/QEAAp06dwsnJCTCNqo420m8uoqOjiYqKwmAw8Pvvv7No0SLpPfT39ycpKQkw5dAdOHBAOi8+Pl5avXHGjBlERkbS19fH1q1b72l235zcuHFDNsN2NwqFgsTERHbt2sWGDRvu63dt377d7PO+Zs6cyfvvv09sbCw2NjasXr0aMA3K3qm+Hq19dbsffviBwsJCqd6JiIhg1apV/9mzjTfDnd47vcMeHh6oVCo+//xzBgcHCQ0NlXWWZ82ahaenJ3q9XkqxMDcODg7k5OSwd+9eMjIyMBgMuLi4sHv3bumYb7/9lvb2dum/K3FwcCA1NZWkpCQ0Gg1TpkzB39+f7Oxs2YJTy5Yto7q6eswZOjDl0I7nslUMjfcsP2FSuHbtmpR3qNFoKC4uvu+RU0G4HwaDgdTUVDo7O8nLy7uvxqBw/z799FP6+/tHDDAIwoPq6upi7dq1VFdXyxbpEP5d7u7uNDY2MjQ0JOrrf8GNGzfw9/dHr9fj6OiIWq0mJSXljp0KQRjLpJyhE8afr776Cr1ej9FoxMHBwWwWPBHGLysrKzIzMx/1bQiC8ACOHDnC119/TWJioujM/UdEff3guru7eeWVVwgLC+PSpUtERUXh5+cnOnPCPyZm6ARBEARBEARBECaoSbkoiiAIgiAIgiAIgjkQHTpBEARBEARBEIQJSnToBEEQBEEQBEEQJijRoRMEQRAEQRAEQZigRIdOEARBEARBEARhghIdOkEQBEEQBEEQhAnq/9qDRoCeJ1AFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1080x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "sns.set(style=\"whitegrid\")\n",
    "\n",
    "jobs = nc.list_trial_jobs()\n",
    "job_ids = [x['id'] for x in jobs]\n",
    "final_metrics = [float(x['finalMetricData'][0]['data']) for x in jobs]\n",
    "\n",
    "data = {'job id': job_ids, 'final metrics': final_metrics}\n",
    "sns.set(rc={'figure.figsize':(15, 6)})\n",
    "\n",
    "plt.title('Trial job final results')\n",
    "ax = sns.barplot(x='job id', y='final metrics', data=data) \n",
    "\n",
    "for i,p in enumerate(ax.patches):\n",
    "    ax.annotate('{:.4f}'.format(p.get_height()), (p.get_x() + p.get_width() / 2., p.get_height()),\n",
    "                 ha='center', va='center', fontsize=11, color='black', rotation=0, xytext=(0, 5),\n",
    "                 textcoords='offset points') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stop nni experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO: Stoping experiment PlUIfDTR\n",
      "INFO: Stop experiment success.\n"
     ]
    }
   ],
   "source": [
    "nc.stop_nni()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}