Add ut to ci & Fix some ut & Add some testcase (#491)

* fix bug

* add docs

* add ut

* add ut

* add to ci

* update doc

* update doc

* update ut

* add ut to ci

* add ut to ci

* add ut to ci

* add ut to ci

* add ut to ci

* add ut to ci

* add ut to ci

* add ut to ci

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* revert

* refactor

* refactor

* s

* merge

azure-pipelines.yml

@@ -14,6 +14,10 @@ jobs:
   - script: |
       source install.sh
     displayName: 'Install nni toolkit via source code'
+  - script: |
+      cd test
+      source unittest.sh
+    displayName: 'Unit test'
   - script: |
       cd test
       PATH=$HOME/.local/bin:$PATH python3 naive_test.py
@@ -21,7 +25,7 @@ jobs:
   - script: |
       cd test
       PATH=$HOME/.local/bin:$PATH python3 sdk_test.py
-    displayName: 'Built-in tuner tests'
+    displayName: 'Built-in dispatcher tests'
 
 - job: 'macOS_10_13'
   pool:
@@ -37,6 +41,10 @@ jobs:
   - script: |
       source install.sh
     displayName: 'Install nni toolkit via source code'
+  - script: |
+      cd test
+      PATH=$HOME/Library/Python/3.7/bin:$PATH && source unittest.sh
+    displayName: 'Unit test'
   - script: |
       cd test
       PATH=$HOME/Library/Python/3.7/bin:$PATH python3 naive_test.py
@@ -44,4 +52,4 @@ jobs:
   - script: |
       cd test
       PATH=$HOME/Library/Python/3.7/bin:$PATH python3 sdk_test.py
-    displayName: 'Built-in tuner tests'
+    displayName: 'Built-in dispatcher tests'
\ No newline at end of file

docs/HowToChooseTuner.md

@@ -10,7 +10,7 @@ For now, NNI has supported the following tuner algorithms. Note that NNI install
  - [Batch Tuner](#Batch)
  - [Grid Search](#Grid)
  - [Hyperband](#Hyperband)
- - [Network Morphism](#NetworkMorphism)
+ - [Network Morphism](#NetworkMorphism) (require pyTorch)
 
  ## Supported tuner algorithms
 
@@ -180,6 +180,10 @@ _Usage_:
 
 [Network Morphism](7) provides functions to automatically search for architecture of deep learning models. Every child network inherits the knowledge from its parent network and morphs into diverse types of networks, including changes of depth, width and skip-connection. Next, it estimates the value of child network using the history architecture and metric pairs. Then it selects the most promising one to train. More detail can be referred to [here](../src/sdk/pynni/nni/networkmorphism_tuner/README.md). 
 
+_Installation_: 
+NetworkMorphism requires [pyTorch](https://pytorch.org/get-started/locally), so users should install it first.
+
+
 _Suggested scenario_: It is suggested that you want to apply deep learning methods to your task (your own dataset) but you have no idea of how to choose or design a network. You modify the [example](../examples/trials/network_morphism/cifar10/cifar10_keras.py) to fit your own dataset and your own data augmentation method. Also you can change the batch size, learning rate or optimizer. It is feasible for different tasks to find a good network architecture. Now this tuner only supports the cv domain.
 
 _Usage_:

src/nni_manager/package.json

@@ -5,7 +5,7 @@
   "scripts": {
     "postbuild": "cp -rf scripts ./dist/ && cp -rf config ./dist/",
     "build": "tsc",
-    "test": "mocha -r ts-node/register -t 15000 --recursive **/*.test.ts --exclude node_modules/**/**/*.test.ts --colors",
+    "test": "mocha -r ts-node/register -t 15000 --recursive **/*.test.ts --exclude node_modules/**/**/*.test.ts --exclude core/test/nnimanager.test.ts --colors",
     "start": "node dist/main.js",
     "tslint": "tslint -p ."
   },

src/sdk/pynni/tests/test_protocol.py

@@ -49,7 +49,7 @@ class ProtocolTestCase(TestCase):
         _prepare_send()
         exception = None
         try:
-            send(CommandType.NewTrialJob, ' ' * 1_000_000)
+            send(CommandType.NewTrialJob, ' ' * 1000000)
         except AssertionError as e:
             exception = e
         self.assertIsNotNone(exception)

src/sdk/pynni/tests/test_trial.py

@@ -18,7 +18,6 @@
 # OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 # ==================================================================================================
 
-
 import nni
 import nni.platform.test as test_platform
 import nni.trial
@@ -30,11 +29,18 @@ from unittest import TestCase, main
 class TrialTestCase(TestCase):
     def setUp(self):
         self._trial_params = { 'msg': 'hi', 'x': 123, 'dict': { 'key': 'value', 'y': None } }
-        nni.trial._params = { 'parameter_id': 'test_param', 'parameters': self._trial_params }
+        test_platform._params = { 'parameter_id': 'test_param', 'parameters': self._trial_params }
 
     def test_get_next_parameter(self):
         self.assertEqual(nni.get_next_parameter(), self._trial_params)
 
+    def test_get_current_parameter(self):
+        nni.get_next_parameter()
+        self.assertEqual(nni.get_current_parameter('x'), 123)
+
+    def test_get_sequence_id(self):
+        self.assertEqual(nni.get_sequence_id(), 0)
+        
     def test_report_intermediate_result(self):
         nni.report_intermediate_result(123)
         self.assertEqual(test_platform.get_last_metric(), {

test/unittest.sh

+#!/bin/bash
+CWD=${PWD}
+
+# -------------For python unittest-------------
+
+## ------Run annotation test------
+echo ""
+echo "===========================Testing: nni_annotation==========================="
+cd ${CWD}/../tools/
+python3 -m unittest -v nni_annotation/test_annotation.py
+
+## Export certain environment variables for unittest code to work
+export NNI_TRIAL_JOB_ID=test_trial_job_id
+export NNI_PLATFORM=unittest
+
+## ------Run sdk test------
+echo ""
+echo "===========================Testing: nni_sdk==========================="
+cd ${CWD}/../src/sdk/pynni/
+python3 -m unittest discover -v tests
+
+
+
+# -------------For typescrip unittest-------------
+cd ${CWD}/../src/nni_manager
+echo ""
+echo "===========================Testing: nni_manager==========================="
+npm run test
\ No newline at end of file