Merge pull request #155 from Microsoft/master

merge master

README.md

@@ -17,7 +17,7 @@
 NNI (Neural Network Intelligence) is a toolkit to help users run automated machine learning (AutoML) experiments. 
 The tool dispatches and runs trial jobs generated by tuning algorithms to search the best neural architecture and/or hyper-parameters in different environments like local machine, remote servers and cloud.
 
-### **NNI [v0.5.2](https://github.com/Microsoft/nni/releases) has been released!**
+### **NNI [v0.6](https://github.com/Microsoft/nni/releases) has been released!**
 <p align="center">
   <a href="#nni-v05-has-been-released"><img src="docs/img/overview.svg" /></a>
 </p>
@@ -115,7 +115,7 @@ Note:
 * We support Linux (Ubuntu 16.04 or higher), MacOS (10.14.1) in our current stage. 
 * Run the following commands in an environment that has `python >= 3.5`, `git` and `wget`.
 ```bash	
-    git clone -b v0.5.2 https://github.com/Microsoft/nni.git
+    git clone -b v0.6 https://github.com/Microsoft/nni.git
     cd nni	
     source install.sh	
 ```
@@ -127,7 +127,7 @@ For the system requirements of NNI, please refer to [Install NNI](docs/en_US/Ins
 The following example is an experiment built on TensorFlow. Make sure you have **TensorFlow installed** before running it.	
 * Download the examples via clone the source code.	
 ```bash	
-    git clone -b v0.5.2 https://github.com/Microsoft/nni.git
+    git clone -b v0.6 https://github.com/Microsoft/nni.git
 ```
 * Run the mnist example.
 ```bash

README_zh_CN.md

@@ -10,7 +10,7 @@
 
 NNI (Neural Network Intelligence) 是自动机器学习（AutoML）的工具包。 它通过多种调优的算法来搜索最好的神经网络结构和（或）超参，并支持单机、本地多机、云等不同的运行环境。
 
-### **NNI [v0.5.2](https://github.com/Microsoft/nni/releases) 已发布！**
+### **NNI [v0.6](https://github.com/Microsoft/nni/releases) 已发布！**
 
 <p align="center">
   <a href="#nni-v05-has-been-released"><img src="docs/img/overview.svg" /></a>
@@ -116,7 +116,7 @@ NNI (Neural Network Intelligence) 是自动机器学习（AutoML）的工具包
 * 在 `python >= 3.5` 的环境中运行命令： `git` 和 `wget`，确保安装了这两个组件。
 
 ```bash
-    git clone -b v0.5.2 https://github.com/Microsoft/nni.git
+    git clone -b v0.6 https://github.com/Microsoft/nni.git
     cd nni  
     source install.sh   
 ```
@@ -130,7 +130,7 @@ NNI (Neural Network Intelligence) 是自动机器学习（AutoML）的工具包
 * 通过克隆源代码下载示例。 
 
 ```bash
-    git clone -b v0.5.2 https://github.com/Microsoft/nni.git
+    git clone -b v0.6 https://github.com/Microsoft/nni.git
 ```
 
 * 运行 mnist 示例。

deployment/docker/README.md

@@ -12,7 +12,7 @@ PyTorch 0.4.1
 scikit-learn 0.20.0
 pandas 0.23.4
 lightgbm 2.2.2
-NNI v0.5.1
+NNI v0.6
 ```
 You can take this Dockerfile as a reference for your own customized Dockerfile. 
 

deployment/docker/README_zh_CN.md

@@ -12,7 +12,7 @@
     scikit-learn 0.20.0
     pandas 0.23.4
     lightgbm 2.2.2
-    NNI v0.5.1
+    NNI v0.6
     
 
 此 Dockerfile 可作为定制的参考。

docs/en_US/Contribution.rst

@@ -4,5 +4,4 @@ Contribute to NNI
 
 ..  toctree::
     Development Setup<SetupNNIDeveloperEnvironment>
-    Contribution Guide<CONTRIBUTING>
-    Debug HowTo<HowToDebug>
+    Contribution Guide<CONTRIBUTING>
\ No newline at end of file
\ No newline at end of file

docs/en_US/FrameworkControllerMode.md

@@ -97,4 +97,7 @@ Trial configuration in frameworkcontroller mode have the following configuration
    * frameworkAttemptCompletionPolicy: the policy to run framework, please refer the [user-manual](https://github.com/Microsoft/frameworkcontroller/blob/master/doc/user-manual.md#frameworkattemptcompletionpolicy) to get the specific information. Users could use the policy to control the pod, for example, if ps does not stop, only worker stops, this completionpolicy could helps stop ps.
 
 ## How to run example
-After you prepare a config file, you could run your experiment by nnictl. The way to start an experiment on frameworkcontroller is similar to kubeflow, please refer the [document](./KubeflowMode.md) for more information.
+After you prepare a config file, you could run your experiment by nnictl. The way to start an experiment on frameworkcontroller is similar to kubeflow, please refer the [document](./KubeflowMode.md) for more information.
\ No newline at end of file
+
+## version check
+NNI support version check feature in since version 0.6, [refer](PAIMode.md)
\ No newline at end of file

docs/en_US/Installation.md

@@ -15,7 +15,7 @@ Currently we only support installation on Linux & Mac.
 
   Prerequisite: `python >=3.5, git, wget`
   ```bash
-  git clone -b v0.5.2 https://github.com/Microsoft/nni.git
+  git clone -b v0.6 https://github.com/Microsoft/nni.git
   cd nni
   ./install.sh
   ```

docs/en_US/KubeflowMode.md

@@ -196,4 +196,7 @@ Notice: In kubeflow mode, NNIManager will start a rest server and listen on a po
 
 Once a trial job is completed, you can goto NNI WebUI's overview page (like http://localhost:8080/oview) to check trial's information. 
 
+## version check
+NNI support version check feature in since version 0.6, [refer](PAIMode.md)
+
 Any problems when using NNI in kubeflow mode, please create issues on [NNI Github repo](https://github.com/Microsoft/nni).

docs/en_US/PAIMode.md

@@ -83,3 +83,13 @@ You can see there're three fils in output folder: stderr, stdout, and trial.log
 If you also want to save trial's other output into HDFS, like model files, you can use environment variable `NNI_OUTPUT_DIR` in your trial code to save your own output files, and NNI SDK will copy all the files in `NNI_OUTPUT_DIR` from trial's container to HDFS. 
 
 Any problems when using NNI in pai mode, please create issues on [NNI github repo](https://github.com/Microsoft/nni).
+
+## version check
+NNI support version check feature in since version 0.6. It is a policy to insure the version of NNIManager is consistent with trialKeeper, and avoid errors caused by version incompatibility.  
+Check policy:  
+1. NNIManager before v0.6 could run any version of trialKeeper, trialKeeper support backward compatibility.
+2. Since version 0.6, NNIManager version should keep same with triakKeeper version. For example, if NNIManager version is 0.6, trialKeeper version should be 0.6 too.  
+3. Note that the version check feature only check first two digits of version.For example, NNIManager v0.6.1 could use trialKeeper v0.6 or trialKeeper v0.6.2, but could not use trialKeeper v0.5.1 or trialKeeper v0.7.  
+  
+If you could not run your experiment and want to know if it is caused by version check, you could check your webUI, and there will be an error message about version check.  
+![](../img/version_check.png)
\ No newline at end of file

docs/en_US/RemoteMachineMode.md

@@ -63,3 +63,6 @@ nnictl create --config ~/nni/examples/trials/mnist-annotation/config_remote.yml
 ```
 
 to start the experiment.
+
+## version check
+NNI support version check feature in since version 0.6, [refer](PAIMode.md)
\ No newline at end of file

docs/en_US/Trials.md

@@ -125,6 +125,34 @@ In the YAML configure file, you need to set *useAnnotation* to true to enable NN
 useAnnotation: true
 ```
 
+
+## Where are my trials?
+
+### Local Mode
+
+In NNI, every trial has a dedicated directory for them to output their own data. In each trial, an environment variable called `NNI_OUTPUT_DIR` is exported. Under this directory, you could find each trial's code, data and other possible log. In addition, each trial's log (including stdout) will be re-directed to a file named `trial.log` under that directory.
+
+If NNI Annotation is used, trial's converted code is in another temporary directory. You can check that in a file named `run.sh` under the directory indicated by `NNI_OUTPUT_DIR`. The second line (i.e., the `cd` command) of this file will change directory to the actual directory where code is located. Below is an example of `run.sh`:
+```shell
+#!/bin/bash
+cd /tmp/user_name/nni/annotation/tmpzj0h72x6 #This is the actual directory
+export NNI_PLATFORM=local
+export NNI_SYS_DIR=/home/user_name/nni/experiments/$experiment_id$/trials/$trial_id$
+export NNI_TRIAL_JOB_ID=nrbb2
+export NNI_OUTPUT_DIR=/home/user_name/nni/experiments/$eperiment_id$/trials/$trial_id$
+export NNI_TRIAL_SEQ_ID=1
+export MULTI_PHASE=false
+export CUDA_VISIBLE_DEVICES=
+eval python3 mnist.py 2>/home/user_name/nni/experiments/$experiment_id$/trials/$trial_id$/stderr
+echo $? `date +%s000` >/home/user_name/nni/experiments/$experiment_id$/trials/$trial_id$/.nni/state
+```
+
+### Other Modes
+
+When runing trials on other platform like remote machine or PAI, the environment variable `NNI_OUTPUT_DIR` only refers to the output directory of the trial, while trial code and `run.sh` might not be there. However, the `trial.log` will be transmitted back to local machine in trial's directory, which defaults to `~/nni/experiments/$experiment_id$/trials/$trial_id$/`
+
+For more information, please refer to [HowToDebug](HowToDebug.md)
+
 <a name="more-examples"></a>
 ## More Trial Examples
 

docs/en_US/Tutorials.rst

@@ -12,4 +12,5 @@ Tutorials
     WebUI
     Training Platform<training_services>
     How to use docker<HowToUseDocker>
-    advanced
+    advanced
\ No newline at end of file
+    Debug HowTo<HowToDebug>
\ No newline at end of file

docs/en_US/WebUI.md

@@ -33,6 +33,17 @@ Click the tab "Trial Duration" to see the bar graph.
 
 ![](../img/trial_duration.png)
 
+## View Trial Intermediate Result Graph
+
+Click the tab "Intermediate Result" to see the lines graph.
+
+![](../img/webui-img/trials_intermeidate.png)
+
+The graph has a filter function. You can open the filter button. And then enter your focus point
+in the scape input. Simultaneously, intermediate result inputs can limit the intermediate's range.
+
+![](../img/webui-img/filter_intermediate.png)
+
 ## View trials status 
 
 Click the tab "Trials Detail" to see the status of the all trials. Specifically:

docs/en_US/conf.py

@@ -28,7 +28,7 @@ author = 'Microsoft'
 # The short X.Y version
 version = ''
 # The full version, including alpha/beta/rc tags
-release = 'v0.5.1'
+release = 'v0.6'
 
 # -- General configuration ---------------------------------------------------
 

docs/en_US/networkmorphismTuner.md

@@ -54,6 +54,52 @@ net = build_graph_from_json(RCV_CONFIG)
 nni.report_final_result(best_acc)
 ```
 
+If you want to save and **load the best model**, the following methods are recommended. 
+
+```python
+# 1. Use NNI API
+## You can get the best model ID from WebUI 
+## or `nni/experiments/experiment_id/log/model_path/best_model.txt'
+
+## read the json string from model file and load it with NNI API
+with open("best-model.json") as json_file:
+    json_of_model = json_file.read()
+model = build_graph_from_json(json_of_model)
+
+# 2. Use Framework API (Related to Framework) 
+## 2.1 Keras API
+
+## Save the model with Keras API in the trial code
+## it's better to save model with id in nni local mode
+model_id = nni.get_sequence_id()
+## serialize model to JSON
+model_json = model.to_json()
+with open("model-{}.json".format(model_id), "w") as json_file:
+    json_file.write(model_json)
+## serialize weights to HDF5
+model.save_weights("model-{}.h5".format(model_id))
+
+## Load the model with Keras API if you want to reuse the model
+## load json and create model
+model_id = "" # id of the model you want to reuse
+with open('model-{}.json'.format(model_id), 'r') as json_file:
+    loaded_model_json = json_file.read()
+loaded_model = model_from_json(loaded_model_json)
+## load weights into new model
+loaded_model.load_weights("model-{}.h5".format(model_id))
+
+## 2.2 PyTorch API
+
+## Save the model with PyTorch API in the trial code
+model_id = nni.get_sequence_id()
+torch.save(model, "model-{}.pt".format(model_id))
+
+## Load the model with PyTorch API if you want to reuse the model
+model_id = "" # id of the model you want to reuse
+loaded_model = torch.load("model-{}.pt".format(model_id))
+
+```
+
 ## 3. File Structure
 
 The tuner has a lot of different files, functions and classes. Here we will only give most of those files a brief introduction: