Merge branch 'master' of git://github.com/microsoft/nni

docs/zh_CN/NniOnWindows.md

@@ -4,33 +4,9 @@
 
 ## **在 Windows 上安装**
 
-**强烈推荐使用 Anaconda 或 Miniconda Python（64位）。**
+详细信息参考[安装](Installation.md#installation-on-windows)。
 
-在第一次使用 PowerShell 运行脚本时，需要用**使用管理员权限**运行如下命令：
-
-```bash
-Set-ExecutionPolicy -ExecutionPolicy Unrestricted
-```
-
-* **通过 pip 命令安装 NNI**
-    
-    先决条件：`python(64-bit) >= 3.5`
-
-```bash
-  python -m pip install --upgrade nni
-```
-
-* __通过代码安装 NNI__
-
-  先决条件: `python >=3.5`, `git`, `PowerShell`
-
-  ```bash
-  git clone -b v0.8 https://github.com/Microsoft/nni.git
-  cd nni
-  powershell -file install.ps1
-  ```
-
-运行完以上脚本后，从命令行使用 **config_windows.yml** 来启动 Experiment，完成安装验证。
+完成操作后，使用 **config_windows.yml** 配置来开始 Experiment 进行验证。
 
 ```bash
 nnictl create --config nni\examples\trials\mnist\config_windows.yml
@@ -85,4 +61,4 @@ Set-ExecutionPolicy -ExecutionPolicy Unrestricted
 
 注意：
 
-* 如果遇到 `Segmentation fault` 这样的错误，参考[常见问答](FAQ.md)。
+* 如果遇到如 `Segmentation fault` 这样的任何错误，参考[常见问题](FAQ.md)。
\ No newline at end of file

docs/zh_CN/QuickStart.md

@@ -164,7 +164,7 @@ trial:
 **注意**：如果使用 Windows，则需要在 config.yml 文件中，将 `python3` 改为 `python`，或者使用 config_windows.yml 来开始 Experiment。
 
 ```bash
-    nnictl create --config nni/examples/trials/mnist/config_windows.yml
+    nnictl create --config nni\examples\trials\mnist\config_windows.yml
 ```
 
 注意：**nnictl** 是一个命令行工具，用来控制 NNI Experiment，如启动、停止、继续 Experiment，启动、停止 NNIBoard 等等。 查看[这里](Nnictl.md)，了解 `nnictl` 更多用法。

docs/zh_CN/SearchSpaceSpec.md

@@ -29,16 +29,16 @@
   
   * 表示变量的值是选项之一。 这里的 'options' 是一个数组。 选项的每个元素都是字符串。 也可以是嵌套的子搜索空间。此子搜索空间仅在相应的元素选中后才起作用。 该子搜索空间中的变量可看作是条件变量。
   
-  * 这是个简单的 [nested] 搜索空间定义的[示例](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nested-search-space/search_space.json)。 如果选项列表中的元素是 dict，则它是一个子搜索空间，对于内置的 Tuner，必须在此 dict 中添加键 “_name”，这有助于标识选中的元素。 相应的，这是从 NNI 中获得的嵌套搜索空间定义的[示例](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nested-search-space/sample.json)。 以下 Tuner 支持嵌套搜索空间：
+  * [nested] 搜索空间定义的简单[示例](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nested-search-space/search_space.json)。 如果选项列表中的元素是 dict，则它是一个子搜索空间，对于内置的 Tuner，必须在此 dict 中添加键 “_name”，这有助于标识选中的元素。 相应的，这是使用从 NNI 获得的嵌套搜索空间的[示例](https://github.com/microsoft/nni/tree/master/examples/trials/mnist-nested-search-space/sample.json)。 以下 Tuner 支持嵌套搜索空间：
     
     * Random Search（随机搜索） 
     * TPE
     * Anneal（退火算法）
     * Evolution
 
-* {"_type":"randint","_value":[upper]}
+* {"_type":"randint","_value":[lower, upper]}
   
-  * 此变量为范围 [0, upper) 之间的随机整数。 这种分布的语义，在较远整数与附近整数之间的损失函数无太大关系， 这是用来描述随机种子的较好分布。 如果损失函数与较近的整数更相关，则应该使用某个"quantized"的连续分布，如quniform, qloguniform, qnormal 或 qlognormal。 注意，如果需要改动数字下限，可以使用 `quniform`。
+  * 当前实现的是 "quniform" 的 "randint" 分布，随机变量的分布函数是 round(uniform(lower, upper))。 所选择值的类型是 float。 如果要使用整数，需要显式转换。
 
 * {"_type":"uniform","_value":[low, high]}
   
@@ -92,9 +92,19 @@
 |  Hyperband Advisor  | ✓ | ✓ | ✓ | ✓ |  ✓  |  ✓   | ✓ | ✓ | ✓  |  ✓  |
 |     Metis Tuner     | ✓ | ✓ | ✓ | ✓ |            |             |          |          |           |            |
 
-注意，在 Grid Search Tuner 中，为了使用方便 `quniform` 和 `qloguniform` 的定义也有所改变，其中的 q 表示采样值的数量。 详情如下：
+已知的局限：
 
-* 类型 'quniform' 接收三个值 [low, high, q]， 其中 [low, high] 指定了范围，而 'q' 指定了会被均匀采样的值的数量。 注意 q 至少为 2。 它的第一个采样值为 'low'，每个采样值都会比前一个大 (high-low)/q 。
-* 类型 'qloguniform' 的行为与 'quniform' 类似，不同处在于首先将范围改为 [log(low), log(high)] 采样后，再将数值还原。
+* 注意，在 Grid Search Tuner 中，为了使用方便 `quniform` 和 `qloguniform` 的定义也有所改变，其中的 q 表示采样值的数量。 详情如下：
+  
+      * 类型 'quniform' 接收三个值 [low, high, q]， 其中 [low, high] 指定了范围，而 'q' 指定了会被均匀采样的值的数量。 注意 q 至少为 2。 它的第一个采样值为 'low'，每个采样值都会比前一个大 (high-low)/q 。
+      
+      * 类型 'qloguniform' 的行为与 'quniform' 类似，不同处在于首先将范围改为 [log(low), log(high)] 采样后，再将数值还原。
+      
+
+* 注意 Metis Tuner 当前仅支持在 `choice` 中使用数值。
 
-注意 Metis Tuner 当前仅支持在 `choice` 中使用数值。
+* 请注意，对于嵌套搜索空间：
\ No newline at end of file
+  
+      * 只有 随机搜索/TPE/Anneal/Evolution Tuner 支持嵌套搜索空间
+      
+      * 不支持嵌套搜索空间 "超参" 并行图，对其的改进通过 #1110(https://github.com/microsoft/nni/issues/1110) 来跟踪 。欢迎任何建议和贡献。
\ No newline at end of file

examples/trials/NAS/README_zh_CN.md

+**在 NNI 中运行神经网络架构搜索**  
+===
+
+参考 [NNI-NAS-Example](https://github.com/Crysple/NNI-NAS-Example)，来使用贡献者提供的 NAS 接口。
+
+谢谢可爱的贡献者！
+
+欢迎越来越多的人加入我们！
\ No newline at end of file

examples/trials/ga_squad/README_zh_CN.md

@@ -14,7 +14,7 @@
 6. ADD-SKIP (在随机层之间一致).
 7. REMOVE-SKIP (移除随机跳过).
 
-![ga-squad-logo](../../../../examples/trials/ga_squad/ga_squad.png)
+![ga-squad-logo](./ga_squad.png)
 
 ## 新版本
 

examples/trials/ga_squad/trial.py

@@ -79,9 +79,7 @@ def get_id(word_dict, word):
     '''
     Return word id.
     '''
-    if word in word_dict.keys():
-        return word_dict[word]
-    return word_dict['<unk>']
+    return word_dict.get(word, word_dict['<unk>'])
 
 
 def load_embedding(path):

examples/trials/mnist-nas/config.yml

+authorName: default
+experimentName: example_mnist
+trialConcurrency: 1
+maxExecDuration: 1h
+maxTrialNum: 10
+#choice: local, remote, pai
+trainingServicePlatform: local
+#choice: true, false
+useAnnotation: true
+tuner:
+  #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner
+  #SMAC (SMAC should be installed through nnictl)
+  #codeDir: ~/nni/nni/examples/tuners/random_nas_tuner
+  codeDir: ../../tuners/random_nas_tuner
+  classFileName: random_nas_tuner.py
+  className: RandomNASTuner
+trial:
+  command: python3 mnist.py
+  codeDir: .
+  gpuNum: 0

examples/trials/mnist-nas/mnist.py

+"""A deep MNIST classifier using convolutional layers."""
+
+import argparse
+import logging
+import math
+import tempfile
+import time
+
+import tensorflow as tf
+from tensorflow.examples.tutorials.mnist import input_data
+
+import operators as op
+
+FLAGS = None
+
+logger = logging.getLogger('mnist_AutoML')
+
+
+class MnistNetwork(object):
+    '''
+    MnistNetwork is for initializing and building basic network for mnist.
+    '''
+    def __init__(self,
+                 channel_1_num,
+                 channel_2_num,
+                 conv_size,
+                 hidden_size,
+                 pool_size,
+                 learning_rate,
+                 x_dim=784,
+                 y_dim=10):
+        self.channel_1_num = channel_1_num
+        self.channel_2_num = channel_2_num
+        self.conv_size = conv_size
+        self.hidden_size = hidden_size
+        self.pool_size = pool_size
+        self.learning_rate = learning_rate
+        self.x_dim = x_dim
+        self.y_dim = y_dim
+
+        self.images = tf.placeholder(tf.float32, [None, self.x_dim], name='input_x')
+        self.labels = tf.placeholder(tf.float32, [None, self.y_dim], name='input_y')
+        self.keep_prob = tf.placeholder(tf.float32, name='keep_prob')
+
+        self.train_step = None
+        self.accuracy = None
+
+    def build_network(self):
+        '''
+        Building network for mnist, meanwhile specifying its neural architecture search space
+        '''
+
+        # Reshape to use within a convolutional neural net.
+        # Last dimension is for "features" - there is only one here, since images are
+        # grayscale -- it would be 3 for an RGB image, 4 for RGBA, etc.
+        with tf.name_scope('reshape'):
+            try:
+                input_dim = int(math.sqrt(self.x_dim))
+            except:
+                print(
+                    'input dim cannot be sqrt and reshape. input dim: ' + str(self.x_dim))
+                logger.debug(
+                    'input dim cannot be sqrt and reshape. input dim: %s', str(self.x_dim))
+                raise
+            x_image = tf.reshape(self.images, [-1, input_dim, input_dim, 1])
+
+        """@nni.mutable_layers(
+            {
+                layer_choice: [op.conv2d(size=1, in_ch=1, out_ch=self.channel_1_num),
+                               op.conv2d(size=3, in_ch=1, out_ch=self.channel_1_num),
+                               op.twice_conv2d(size=3, in_ch=1, out_ch=self.channel_1_num),
+                               op.twice_conv2d(size=7, in_ch=1, out_ch=self.channel_1_num),
+                               op.dilated_conv(in_ch=1, out_ch=self.channel_1_num),
+                               op.separable_conv(size=3, in_ch=1, out_ch=self.channel_1_num),
+                               op.separable_conv(size=5, in_ch=1, out_ch=self.channel_1_num),
+                               op.separable_conv(size=7, in_ch=1, out_ch=self.channel_1_num)],
+                fixed_inputs: [x_image],
+                layer_output: conv1_out
+            },
+            {
+                layer_choice: [op.post_process(ch_size=self.channel_1_num)],
+                fixed_inputs: [conv1_out],
+                layer_output: post1_out
+            },
+            {
+                layer_choice: [op.max_pool(size=3),
+                               op.max_pool(size=5),
+                               op.max_pool(size=7),
+                               op.avg_pool(size=3),
+                               op.avg_pool(size=5),
+                               op.avg_pool(size=7)],
+                fixed_inputs: [post1_out],
+                layer_output: pool1_out
+            },
+            {
+                layer_choice: [op.conv2d(size=1, in_ch=self.channel_1_num, out_ch=self.channel_2_num),
+                               op.conv2d(size=3, in_ch=self.channel_1_num, out_ch=self.channel_2_num),
+                               op.twice_conv2d(size=3, in_ch=self.channel_1_num, out_ch=self.channel_2_num),
+                               op.twice_conv2d(size=7, in_ch=self.channel_1_num, out_ch=self.channel_2_num),
+                               op.dilated_conv(in_ch=self.channel_1_num, out_ch=self.channel_2_num),
+                               op.separable_conv(size=3, in_ch=self.channel_1_num, out_ch=self.channel_2_num),
+                               op.separable_conv(size=5, in_ch=self.channel_1_num, out_ch=self.channel_2_num),
+                               op.separable_conv(size=7, in_ch=self.channel_1_num, out_ch=self.channel_2_num)],
+                fixed_inputs: [pool1_out],
+                optional_inputs: [post1_out],
+                optional_input_size: [0, 1],
+                layer_output: conv2_out
+            },
+            {
+                layer_choice: [op.post_process(ch_size=self.channel_2_num)],
+                fixed_inputs: [conv2_out],
+                layer_output: post2_out
+            },
+            {
+                layer_choice: [op.max_pool(size=3),
+                               op.max_pool(size=5),
+                               op.max_pool(size=7),
+                               op.avg_pool(size=3),
+                               op.avg_pool(size=5),
+                               op.avg_pool(size=7)],
+                fixed_inputs: [post2_out],
+                optional_inputs: [post1_out, pool1_out],
+                optional_input_size: [0, 1],
+                layer_output: pool2_out
+            }
+        )"""
+
+        # Fully connected layer 1 -- after 2 round of downsampling, our 28x28 image
+        # is down to 7x7x64 feature maps -- maps this to 1024 features.
+        last_dim_list = pool2_out.get_shape().as_list()
+        assert(last_dim_list[1] == last_dim_list[2])
+        last_dim = last_dim_list[1]
+        with tf.name_scope('fc1'):
+            w_fc1 = op.weight_variable(
+                [last_dim * last_dim * self.channel_2_num, self.hidden_size])
+            b_fc1 = op.bias_variable([self.hidden_size])
+
+        h_pool2_flat = tf.reshape(
+            pool2_out, [-1, last_dim * last_dim * self.channel_2_num])
+        h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, w_fc1) + b_fc1)
+
+        # Dropout - controls the complexity of the model, prevents co-adaptation of features.
+        with tf.name_scope('dropout'):
+            h_fc1_drop = tf.nn.dropout(h_fc1, self.keep_prob)
+
+        # Map the 1024 features to 10 classes, one for each digit
+        with tf.name_scope('fc2'):
+            w_fc2 = op.weight_variable([self.hidden_size, self.y_dim])
+            b_fc2 = op.bias_variable([self.y_dim])
+            y_conv = tf.matmul(h_fc1_drop, w_fc2) + b_fc2
+
+        with tf.name_scope('loss'):
+            cross_entropy = tf.reduce_mean(
+                tf.nn.softmax_cross_entropy_with_logits(labels=self.labels, logits=y_conv))
+        with tf.name_scope('adam_optimizer'):
+            self.train_step = tf.train.AdamOptimizer(
+                self.learning_rate).minimize(cross_entropy)
+
+        with tf.name_scope('accuracy'):
+            correct_prediction = tf.equal(
+                tf.argmax(y_conv, 1), tf.argmax(self.labels, 1))
+            self.accuracy = tf.reduce_mean(
+                tf.cast(correct_prediction, tf.float32))
+
+
+def download_mnist_retry(data_dir, max_num_retries=20):
+    """Try to download mnist dataset and avoid errors"""
+    for _ in range(max_num_retries):
+        try:
+            return input_data.read_data_sets(data_dir, one_hot=True)
+        except tf.errors.AlreadyExistsError:
+            time.sleep(1)
+    raise Exception("Failed to download MNIST.")
+
+def main(params):
+    '''
+    Main function, build mnist network, run and send result to NNI.
+    '''
+    # Import data
+    mnist = download_mnist_retry(params['data_dir'])
+    print('Mnist download data done.')
+    logger.debug('Mnist download data done.')
+
+    # Create the model
+    # Build the graph for the deep net
+    mnist_network = MnistNetwork(channel_1_num=params['channel_1_num'],
+                                 channel_2_num=params['channel_2_num'],
+                                 conv_size=params['conv_size'],
+                                 hidden_size=params['hidden_size'],
+                                 pool_size=params['pool_size'],
+                                 learning_rate=params['learning_rate'])
+    mnist_network.build_network()
+    logger.debug('Mnist build network done.')
+
+    # Write log
+    graph_location = tempfile.mkdtemp()
+    logger.debug('Saving graph to: %s', graph_location)
+    train_writer = tf.summary.FileWriter(graph_location)
+    train_writer.add_graph(tf.get_default_graph())
+
+    test_acc = 0.0
+    with tf.Session() as sess:
+        sess.run(tf.global_variables_initializer())
+        for i in range(params['batch_num']):
+            batch = mnist.train.next_batch(params['batch_size'])
+            mnist_network.train_step.run(feed_dict={mnist_network.images: batch[0],
+                                                    mnist_network.labels: batch[1],
+                                                    mnist_network.keep_prob: 1 - params['dropout_rate']}
+                                        )
+
+            if i % 100 == 0:
+                test_acc = mnist_network.accuracy.eval(
+                    feed_dict={mnist_network.images: mnist.test.images,
+                               mnist_network.labels: mnist.test.labels,
+                               mnist_network.keep_prob: 1.0})
+
+                """@nni.report_intermediate_result(test_acc)"""
+                logger.debug('test accuracy %g', test_acc)
+                logger.debug('Pipe send intermediate result done.')
+
+        test_acc = mnist_network.accuracy.eval(
+            feed_dict={mnist_network.images: mnist.test.images,
+                       mnist_network.labels: mnist.test.labels,
+                       mnist_network.keep_prob: 1.0})
+
+        """@nni.report_final_result(test_acc)"""
+        logger.debug('Final result is %g', test_acc)
+        logger.debug('Send final result done.')
+
+def get_params():
+    ''' Get parameters from command line '''
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--data_dir", type=str, default='/tmp/tensorflow/mnist/input_data', help="data directory")
+    parser.add_argument("--dropout_rate", type=float, default=0.5, help="dropout rate")
+    parser.add_argument("--channel_1_num", type=int, default=32)
+    parser.add_argument("--channel_2_num", type=int, default=64)
+    parser.add_argument("--conv_size", type=int, default=5)
+    parser.add_argument("--pool_size", type=int, default=2)
+    parser.add_argument("--hidden_size", type=int, default=1024)
+    parser.add_argument("--learning_rate", type=float, default=1e-4)
+    parser.add_argument("--batch_num", type=int, default=2000)
+    parser.add_argument("--batch_size", type=int, default=32)
+
+    args, _ = parser.parse_known_args()
+    return args
+
+if __name__ == '__main__':
+    try:
+        params = vars(get_params())
+        main(params)
+    except Exception as exception:
+        logger.exception(exception)
+        raise

examples/trials/mnist-nas/operators.py

+import tensorflow as tf
+import math
+
+
+def weight_variable(shape):
+    """weight_variable generates a weight variable of a given shape."""
+    initial = tf.truncated_normal(shape, stddev=0.1)
+    return tf.Variable(initial)
+
+def bias_variable(shape):
+    """bias_variable generates a bias variable of a given shape."""
+    initial = tf.constant(0.1, shape=shape)
+    return tf.Variable(initial)
+
+def sum_op(inputs):
+    """sum_op"""
+    fixed_input = inputs[0][0]
+    optional_input = inputs[1][0]
+    fixed_shape = fixed_input.get_shape().as_list()
+    optional_shape = optional_input.get_shape().as_list()
+    assert fixed_shape[1] == fixed_shape[2]
+    assert optional_shape[1] == optional_shape[2]
+    pool_size = math.ceil(optional_shape[1] / fixed_shape[1])
+    pool_out = tf.nn.avg_pool(optional_input, ksize=[1, pool_size, pool_size, 1], strides=[1, pool_size, pool_size, 1], padding='SAME')
+    conv_matrix = weight_variable([1, 1, optional_shape[3], fixed_shape[3]])
+    conv_out = tf.nn.conv2d(pool_out, conv_matrix, strides=[1, 1, 1, 1], padding='SAME')
+    return fixed_input + conv_out
+
+
+def conv2d(inputs, size=-1, in_ch=-1, out_ch=-1):
+    """conv2d returns a 2d convolution layer with full stride."""
+    if not inputs[1]:
+        x_input = inputs[0][0]
+    else:
+        x_input = sum_op(inputs)
+    if size in [1, 3]:
+        w_matrix = weight_variable([size, size, in_ch, out_ch])
+        return tf.nn.conv2d(x_input, w_matrix, strides=[1, 1, 1, 1], padding='SAME')
+    else:
+        raise Exception("Unknown filter size: %d." % size)
+
+def twice_conv2d(inputs, size=-1, in_ch=-1, out_ch=-1):
+    """twice_conv2d"""
+    if not inputs[1]:
+        x_input = inputs[0][0]
+    else:
+        x_input = sum_op(inputs)
+    if size in  [3, 7]:
+        w_matrix1 = weight_variable([1, size, in_ch, int(out_ch/2)])
+        out = tf.nn.conv2d(x_input, w_matrix1, strides=[1, 1, 1, 1], padding='SAME')
+        w_matrix2 = weight_variable([size, 1, int(out_ch/2), out_ch])
+        return tf.nn.conv2d(out, w_matrix2, strides=[1, 1, 1, 1], padding='SAME')
+    else:
+        raise Exception("Unknown filter size: %d." % size)
+
+def dilated_conv(inputs, size=3, in_ch=-1, out_ch=-1):
+    """dilated_conv"""
+    if not inputs[1]:
+        x_input = inputs[0][0]
+    else:
+        x_input = sum_op(inputs)
+    if size == 3:
+        w_matrix = weight_variable([size, size, in_ch, out_ch])
+        return tf.nn.atrous_conv2d(x_input, w_matrix, rate=2, padding='SAME')
+    else:
+        raise Exception("Unknown filter size: %d." % size)
+
+def separable_conv(inputs, size=-1, in_ch=-1, out_ch=-1):
+    """separable_conv"""
+    if not inputs[1]:
+        x_input = inputs[0][0]
+    else:
+        x_input = sum_op(inputs)
+    if size in [3, 5, 7]:
+        depth_matrix = weight_variable([size, size, in_ch, 1])
+        point_matrix = weight_variable([1, 1, 1*in_ch, out_ch])
+        return tf.nn.separable_conv2d(x_input, depth_matrix, point_matrix, strides=[1, 1, 1, 1], padding='SAME')
+    else:
+        raise Exception("Unknown filter size: %d." % size)
+
+
+def avg_pool(inputs, size=-1):
+    """avg_pool downsamples a feature map."""
+    if not inputs[1]:
+        x_input = inputs[0][0]
+    else:
+        x_input = sum_op(inputs)
+    if size in [3, 5, 7]:
+        return tf.nn.avg_pool(x_input, ksize=[1, size, size, 1], strides=[1, size, size, 1], padding='SAME')
+    else:
+        raise Exception("Unknown filter size: %d." % size)
+
+def max_pool(inputs, size=-1):
+    """max_pool downsamples a feature map."""
+    if not inputs[1]:
+        x_input = inputs[0][0]
+    else:
+        x_input = sum_op(inputs)
+    if size in [3, 5, 7]:
+        return tf.nn.max_pool(x_input, ksize=[1, size, size, 1], strides=[1, size, size, 1], padding='SAME')
+    else:
+        raise Exception("Unknown filter size: %d." % size)
+
+
+def post_process(inputs, ch_size=-1):
+    """post_process"""
+    x_input = inputs[0][0]
+    bias_matrix = bias_variable([ch_size])
+    return tf.nn.relu(x_input + bias_matrix)

examples/trials/NAS/README.md → examples/trials/nas_cifar10/README.md

- **Run Neural Network Architecture Search in NNI**	
- ===	
- 
-Now we have an NAS example [NNI-NAS-Example](https://github.com/Crysple/NNI-NAS-Example) run in NNI using NAS interface from our contributors.	
- 
-Thanks our lovely contributors. 	
- 
-And welcome more and more people to join us!
+ **Run Neural Network Architecture Search in NNI**	
+ ===	
+ 
+Now we have an NAS example [NNI-NAS-Example](https://github.com/Crysple/NNI-NAS-Example) run in NNI using NAS interface from our contributors.	
+ 
+Thanks our lovely contributors. 	
+ 
+And welcome more and more people to join us!
\ No newline at end of file

examples/trials/network_morphism/README_zh_CN.md

@@ -99,10 +99,10 @@ nnictl create --config config.yml
 
 `Fashion-MNIST` 是来自 [Zalando](https://jobs.zalando.com/tech/) 文章的图片 — 有 60,000 个样例的训练集和 10,000 个样例的测试集。 每个样例是 28x28 的灰度图，分为 10 个类别。 由于 MNIST 数据集过于简单，该数据集现在开始被广泛使用，用来替换 MNIST 作为基准数据集。
 
-这里有两个样例，[FashionMNIST-keras.py](../../../../examples/trials/network_morphism/FashionMNIST/FashionMNIST_keras.py) 和 [FashionMNIST-pytorch.py](../../../../examples/trials/network_morphism/FashionMNIST/FashionMNIST_pytorch.py)。 注意，在 `config.yml` 中，需要为此数据集修改 `input_width` 为 28，以及 `input_channel` 为 1。
+这里有两个样例，[FashionMNIST-keras.py](./FashionMNIST/FashionMNIST_keras.py) 和 [FashionMNIST-pytorch.py](./FashionMNIST/FashionMNIST_pytorch.py)。 注意，在 `config.yml` 中，需要为此数据集修改 `input_width` 为 28，以及 `input_channel` 为 1。
 
 ### Cifar10
 
 `CIFAR-10` 数据集 [Canadian Institute For Advanced Research](https://www.cifar.ca/) 是广泛用于机器学习和视觉算法训练的数据集。 它是机器学习领域最广泛使用的数据集之一。 CIFAR-10 数据集包含了 60,000 张 32x32 的彩色图片，分为 10 类。
 
-这里有两个样例，[cifar10-keras.py](../../../../examples/trials/network_morphism/cifar10/cifar10_keras.py) 和 [cifar10-pytorch.py](../../../../examples/trials/network_morphism/cifar10/cifar10_pytorch.py)。 在 `config.yml` 中，该数据集 `input_width` 的值是 32，并且 `input_channel` 是 3。
+这里有两个样例，[cifar10-keras.py](./cifar10/cifar10_keras.py) 和 [cifar10-pytorch.py](./cifar10/cifar10_pytorch.py)。 在 `config.yml` 中，该数据集 `input_width` 的值是 32，并且 `input_channel` 是 3。
\ No newline at end of file

examples/trials/weight_sharing/ga_squad/data.py

@@ -241,9 +241,7 @@ def get_id(word_dict, word):
     '''
     Given word, return word id.
     '''
-    if word in word_dict.keys():
-        return word_dict[word]
-    return word_dict['<unk>']
+    return word_dict.get(word, word_dict['<unk>'])
 
 
 def get_buckets(min_length, max_length, bucket_count):

examples/tuners/random_nas_tuner/random_nas_tuner.py

+import numpy as np
+
+from nni.tuner import Tuner
+
+def random_archi_generator(nas_ss, random_state):
+    '''random
+    '''
+    chosen_archi = {}
+    print("zql: nas search space: ", nas_ss)
+    for block_name, block in nas_ss.items():
+        tmp_block = {}
+        for layer_name, layer in block.items():
+            tmp_layer = {}
+            for key, value in layer.items():
+                if key == 'layer_choice':
+                    index = random_state.randint(len(value))
+                    tmp_layer['chosen_layer'] = value[index]
+                elif key == 'optional_inputs':
+                    tmp_layer['chosen_inputs'] = []
+                    print("zql: optional_inputs", layer['optional_inputs'])
+                    if layer['optional_inputs']:
+                        if isinstance(layer['optional_input_size'], int):
+                            choice_num = layer['optional_input_size']
+                        else:
+                            choice_range = layer['optional_input_size']
+                            choice_num = random_state.randint(choice_range[0], choice_range[1]+1)
+                        for _ in range(choice_num):
+                            index = random_state.randint(len(layer['optional_inputs']))
+                            tmp_layer['chosen_inputs'].append(layer['optional_inputs'][index])
+                elif key == 'optional_input_size':
+                    pass
+                else:
+                    raise ValueError('Unknown field %s in layer %s of block %s' % (key, layer_name, block_name))
+            tmp_block[layer_name] = tmp_layer
+        chosen_archi[block_name] = tmp_block
+    return chosen_archi
+
+class RandomNASTuner(Tuner):
+    '''RandomNASTuner
+    '''
+
+    def __init__(self):
+        self.searchspace_json = None
+        self.random_state = None
+
+    def update_search_space(self, search_space):
+        '''update
+        '''
+        self.searchspace_json = search_space
+        self.random_state = np.random.RandomState()
+
+    def generate_parameters(self, parameter_id):
+        '''generate
+        '''
+        return random_archi_generator(self.searchspace_json, self.random_state)
+
+    def receive_trial_result(self, parameter_id, parameters, value):
+        '''receive
+        '''
+        pass

setup.py

@@ -56,7 +56,8 @@ setup(
         'scipy',
         'schema',
         'PythonWebHDFS',
-        'colorama'
+        'colorama',
+        'sklearn'
     ],
 
     entry_points = {

src/nni_manager/rest_server/restHandler.ts

@@ -203,7 +203,7 @@ class NNIRestHandler {
                 res.send();
             } catch (err) {
                 // setClusterMetata is a step of initialization, so any exception thrown is a fatal
-                this.handle_error(err, res, true);
+                this.handle_error(NNIError.FromError(err), res, true);
             }
         });
     }

src/nni_manager/training_service/remote_machine/remoteMachineData.ts

@@ -29,31 +29,18 @@ import { GPUSummary, GPUInfo } from '../common/gpuData';
  * Metadata of remote machine for configuration and statuc query
  */
 export class RemoteMachineMeta {
-    public readonly ip : string;
-    public readonly port : number;
-    public readonly username : string;
-    public readonly passwd?: string;
+    public readonly ip : string = '';
+    public readonly port : number = 22;
+    public readonly username : string = '';
+    public readonly passwd: string = '';
     public readonly sshKeyPath?: string;
     public readonly passphrase?: string;
     public gpuSummary : GPUSummary | undefined;
     public readonly gpuIndices?: string;
     public readonly maxTrialNumPerGpu?: number;
-    public occupiedGpuIndexMap: Map<number, number>;
+    //TODO: initialize varialbe in constructor
+    public occupiedGpuIndexMap?: Map<number, number>;
     public readonly useActiveGpu?: boolean = false;
-
-    constructor(ip : string, port : number, username : string, passwd : string,
-                sshKeyPath: string, passphrase : string, gpuIndices?: string, maxTrialNumPerGpu?: number, useActiveGpu?: boolean) {
-        this.ip = ip;
-        this.port = port;
-        this.username = username;
-        this.passwd = passwd;
-        this.sshKeyPath = sshKeyPath;
-        this.passphrase = passphrase;
-        this.gpuIndices = gpuIndices;
-        this.maxTrialNumPerGpu = maxTrialNumPerGpu;
-        this.occupiedGpuIndexMap = new Map<number, number>();
-        this.useActiveGpu = useActiveGpu;
-    }
 }
 
 export function parseGpuIndices(gpuIndices?: string): Set<number> | undefined {

src/nni_manager/training_service/remote_machine/remoteMachineTrainingService.ts

@@ -466,6 +466,7 @@ class RemoteMachineTrainingService implements TrainingService {
         let connectedRMNum: number = 0;
 
         rmMetaList.forEach(async (rmMeta: RemoteMachineMeta) => {
+            rmMeta.occupiedGpuIndexMap = new Map<number, number>();
             let sshClientManager: SSHClientManager = new SSHClientManager([], this.MAX_TRIAL_NUMBER_PER_SSHCONNECTION, rmMeta);
             let sshClient: Client = await sshClientManager.getAvailableSSHClient();
             this.machineSSHClientMap.set(rmMeta, sshClientManager);

src/sdk/pynni/nni/smartparam.py

@@ -124,7 +124,7 @@ else:
             funcs_args,
             fixed_inputs,
             optional_inputs,
-            optional_input_size=0):
+            optional_input_size):
         '''execute the chosen function and inputs.
         Below is an example of chosen function and inputs:
         {
@@ -149,7 +149,7 @@ else:
         chosen_layer = mutable_block[mutable_layer_id]["chosen_layer"]
         chosen_inputs = mutable_block[mutable_layer_id]["chosen_inputs"]
         real_chosen_inputs = [optional_inputs[input_name] for input_name in chosen_inputs]
-        layer_out = funcs[chosen_layer]([fixed_inputs, real_chosen_inputs], *funcs_args[chosen_layer])
+        layer_out = funcs[chosen_layer]([fixed_inputs, real_chosen_inputs], **funcs_args[chosen_layer])
         
         return layer_out
 

src/webui/src/components/SlideBar.tsx

@@ -92,7 +92,7 @@ class SlideBar extends React.Component<{}, SliderState> {
           const aTag = document.createElement('a');
           const isEdge = navigator.userAgent.indexOf('Edge') !== -1 ? true : false;
           const file = new Blob([nniLogfile], { type: 'application/json' });
-          aTag.download = 'nnimanagerLog.json';
+          aTag.download = 'nnimanager.log';
           aTag.href = URL.createObjectURL(file);
           aTag.click();
           if (!isEdge) {
@@ -101,7 +101,7 @@ class SlideBar extends React.Component<{}, SliderState> {
           if (navigator.userAgent.indexOf('Firefox') > -1) {
             const downTag = document.createElement('a');
             downTag.addEventListener('click', function () {
-              downTag.download = 'nnimanagerLog.json';
+              downTag.download = 'nnimanager.log';
               downTag.href = URL.createObjectURL(file);
             });
             let eventMouse = document.createEvent('MouseEvents');
@@ -122,7 +122,7 @@ class SlideBar extends React.Component<{}, SliderState> {
           const aTag = document.createElement('a');
           const isEdge = navigator.userAgent.indexOf('Edge') !== -1 ? true : false;
           const file = new Blob([dispatchLogfile], { type: 'application/json' });
-          aTag.download = 'dispatcherLog.json';
+          aTag.download = 'dispatcher.log';
           aTag.href = URL.createObjectURL(file);
           aTag.click();
           if (!isEdge) {
@@ -131,7 +131,7 @@ class SlideBar extends React.Component<{}, SliderState> {
           if (navigator.userAgent.indexOf('Firefox') > -1) {
             const downTag = document.createElement('a');
             downTag.addEventListener('click', function () {
-              downTag.download = 'dispatcherLog.json';
+              downTag.download = 'dispatcher.log';
               downTag.href = URL.createObjectURL(file);
             });
             let eventMouse = document.createEvent('MouseEvents');

src/webui/src/components/TrialsDetail.tsx

@@ -27,7 +27,6 @@ interface TrialDetailState {
     entriesInSelect: string;
     searchSpace: string;
     isMultiPhase: boolean;
-    isTableLoading: boolean;
     whichGraph: string;
     hyperCounts: number; // user click the hyper-parameter counts
     durationCounts: number;
@@ -79,7 +78,6 @@ class TrialsDetail extends React.Component<{}, TrialDetailState> {
             whichGraph: '1',
             isHasSearch: false,
             isMultiPhase: false,
-            isTableLoading: false,
             hyperCounts: 0,
             durationCounts: 0,
             intermediateCounts: 0
@@ -95,9 +93,6 @@ class TrialsDetail extends React.Component<{}, TrialDetailState> {
             ])
             .then(axios.spread((res, res1) => {
                 if (res.status === 200 && res1.status === 200) {
-                    if (this._isMounted === true) {
-                        this.setState(() => ({ isTableLoading: true }));
-                    }
                     const trialJobs = res.data;
                     const metricSource = res1.data;
                     const trialTable: Array<TableObj> = [];
@@ -187,10 +182,7 @@ class TrialsDetail extends React.Component<{}, TrialDetailState> {
                         }
                     }
                     if (this._isMounted) {
-                        this.setState(() => ({
-                            isTableLoading: false,
-                            tableListSource: trialTable
-                        }));
+                        this.setState(() => ({ tableListSource: trialTable }));
                     }
                     if (entriesInSelect === 'all' && this._isMounted) {
                         this.setState(() => ({
@@ -330,7 +322,7 @@ class TrialsDetail extends React.Component<{}, TrialDetailState> {
         const {
             tableListSource, searchResultSource, isHasSearch, isMultiPhase,
             entriesTable, experimentPlatform, searchSpace, experimentLogCollection,
-            whichGraph, isTableLoading
+            whichGraph
         } = this.state;
         const source = isHasSearch ? searchResultSource : tableListSource;
         return (
@@ -407,7 +399,6 @@ class TrialsDetail extends React.Component<{}, TrialDetailState> {
                 <TableList
                     entries={entriesTable}
                     tableSource={source}
-                    isTableLoading={isTableLoading}
                     isMultiPhase={isMultiPhase}
                     platform={experimentPlatform}
                     updateList={this.getDetailSource}