Merge pull request #248 from microsoft/master

merge master

docs/en_US/NAS/NasGuide.md

@@ -176,7 +176,7 @@ For example,
 }
 ```
 
-After applying, the model is then fixed and ready for final training. The model works as a single model, although it might contain more parameters than expected. This comes with pros and cons. The good side is, you can directly load the checkpoint dumped from supernet during the search phase and start retraining from there. However, this is also a model with redundant parameters and this may cause problems when trying to count the number of parameters in the model. For deeper reasons and possible workarounds, see [Trainers](./NasReference.md).
+After applying, the model is then fixed and ready for final training. The model works as a single model, and unused parameters and modules are pruned.
 
 Also, refer to [DARTS](./DARTS.md) for code exemplifying retraining.
 

docs/en_US/TrainingService/PaiMode.md

@@ -7,9 +7,9 @@ Step 1. Install NNI, follow the install guide [here](../Tutorial/QuickStart.md).
 
 Step 2. Get PAI token.   
 Click `My profile` button in the top-right side of PAI's webprotal.
-![](../../img/pai_token_button.jpg)
-Find the token management region, copy one of the token as your account token.
-![](../../img/pai_token_profile.jpg)  
+![](../../img/pai_profile.jpg)  
+Click `copy` button in the page to copy a jwt token.
+![](../../img/pai_token.jpg)  
 
 Step 3. Mount NFS storage to local machine.  
   Click `Submit job` button in PAI's webportal.
@@ -19,7 +19,7 @@ Step 3. Mount NFS storage to local machine.
 The `DEFAULT_STORAGE`field is the path to be mounted in PAI's container when a job is started. The `Preview container paths` is the NFS host and path that PAI provided, you need to mount the corresponding host and path to your local machine first, then NNI could use the PAI's NFS storage.  
 For example, use the following command:
 ```
-sudo mount nfs://gcr-openpai-infra02:/pai/data /local/mnt
+sudo mount -t nfs4 gcr-openpai-infra02:/pai/data /local/mnt
 ```
 Then the `/data` folder in container will be mounted to `/local/mnt` folder in your local machine.  
 You could use the following configuration in your NNI's config file:
@@ -66,7 +66,7 @@ trial:
   virtualCluster: default
   nniManagerNFSMountPath: /home/user/mnt
   containerNFSMountPath: /mnt/data/user
-  paiStoragePlugin: team_wise
+  paiStoragePlugin: teamwise_storage
 # Configuration to access OpenPAI Cluster
 paiConfig:
   userName: your_pai_nni_user
@@ -74,7 +74,7 @@ paiConfig:
   host: 10.1.1.1
 ```
 
-Note: You should set `trainingServicePlatform: pai` in NNI config YAML file if you want to start experiment in pai mode.
+Note: You should set `trainingServicePlatform: pai` in NNI config YAML file if you want to start experiment in pai mode. The host field in configuration file is PAI's job submission page uri, like `10.10.5.1`, the default http protocol in NNI is `http`, if your PAI's cluster enabled https, please use the uri in `https://10.10.5.1` format.
 
 Compared with [LocalMode](LocalMode.md) and [RemoteMachineMode](RemoteMachineMode.md), trial configuration in pai mode have these additional keys:
 * cpuNum

docs/en_US/Tutorial/SetupNniDeveloperEnvironment.md

-**Set up NNI developer environment**
+# Setup NNI development environment
 
-===
+NNI development environment supports Ubuntu 1604 (or above), and Windows 10 with Python3 64bit.
 
-## Best practice for debug NNI source code
+## Installation
 
-For debugging NNI source code, your development environment should be under Ubuntu 16.04 (or above) system with python 3 and pip 3 installed, then follow the below steps.
+The installation steps are similar with installing from source code. But the installation links to code directory, so that code changes can be applied to installation as easy as possible.
 
-### 1. Clone the source code
+### 1. Clone source code
 
-Run the command
-
-```
+```bash
 git clone https://github.com/Microsoft/nni.git
 ```
 
-to clone the source code
+Note, if you want to contribute code back, it needs to fork your own NNI repo, and clone from there.
 
-### 2. Prepare the debug environment and install dependencies
+### 2. Install from source code
 
-Change directory to the source code folder, then run the command
+#### Ubuntu
 
+```bash
+make dev-easy-install
 ```
-make install-dependencies
-```
-
-to install the dependent tools for the environment
-
-### 3. Build source code
 
-Run the command
+#### Windows
 
+```bat
+powershell -ExecutionPolicy Bypass -file install.ps1 -Development
 ```
-make build
-```
-
-to build the source code
 
-### 4. Install NNI to development environment
-
-Run the command
-
-```
-make dev-install
-```
-
-to install the distribution content to development environment, and create cli scripts
-
-### 5. Check if the environment is ready
+### 3. Check if the environment is ready
 
 Now, you can try to start an experiment to check if your environment is ready.
 For example, run the command
 
-```
-nnictl create --config ~/nni/examples/trials/mnist-tfv1/config.yml
+```bash
+nnictl create --config examples/trials/mnist-tfv1/config.yml
 ```
 
 And open WebUI to check if everything is OK
 
-### 6. Redeploy
-
-After the code changes, it may need to redeploy. It depends on what kind of code changed. 
+### 4. Reload changes
 
 #### Python
 
-It doesn't need to redeploy, but the nnictl may need to be restarted.
+Nothing to do, the code is already linked to package folders.
 
 #### TypeScript
 
-* If `src/nni_manager` is changed, run `yarn watch` continually under this folder. It will rebuild code instantly. The nnictl may need to be restarted to reload NNI manager.
+* If `src/nni_manager` is changed, run `yarn watch` under this folder. It will watch and build code continually. The `nnictl` need to be restarted to reload NNI manager.
 * If `src/webui` or `src/nasui` are changed, run `yarn start` under the corresponding folder. The web UI will refresh automatically if code is changed.
 
+### 5. Submit Pull Request
+
+All changes are merged to master branch from your forked repo. The description of Pull Request must be meaningful, and useful.
+
+We will review the changes as soon as possible. Once it passes review, we will merge it to master branch.
 
----
-At last, wish you have a wonderful day.
-For more contribution guidelines on making PR's or issues to NNI source code, you can refer to our [Contributing](Contributing.md) document.
+For more contribution guidelines and coding styles, you can refer to the [contributing document](Contributing.md).

install.ps1

+param ([Switch] $Development)
 [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
 
+
 $install_node = $true
 $install_yarn = $true
 
-if([Environment]::Is64BitOperatingSystem){
+if ([Environment]::Is64BitOperatingSystem) {
     $OS_VERSION = 'win64'
 }
-else{
+else {
     $OS_VERSION = 'win32'
 }
 # nodejs
@@ -15,58 +17,58 @@ $yarnUrl = "https://yarnpkg.com/latest.tar.gz"
 $unzipNodeDir = "node-v*"
 $unzipYarnDir = "yarn-v*"
 
-$NNI_DEPENDENCY_FOLDER = [System.IO.Path]::GetTempPath()+$env:USERNAME
+$NNI_DEPENDENCY_FOLDER = [System.IO.Path]::GetTempPath() + $env:USERNAME
 
 $WHICH_PYTHON = where.exe python
-if($WHICH_PYTHON -eq $null){
+if ($WHICH_PYTHON -eq $null) {
     throw "Can not find python"
 }
-else{
+else {
     $pyVersion = & python -V 2>&1
-    $pyVersion =  ([string]$pyVersion).substring(7,3)
-    if([double]$pyVersion -lt 3.5){
+    $pyVersion = ([string]$pyVersion).substring(7, 3)
+    if ([double]$pyVersion -lt 3.5) {
         throw "python version should >= 3.5"
     }
 }
 
 $WHICH_PIP = where.exe pip
-if($WHICH_PIP -eq $null){
+if ($WHICH_PIP -eq $null) {
     throw "Can not find pip"
 }
 
 $env:PYTHONIOENCODING = "UTF-8"
-if($env:VIRTUAL_ENV){
+if ($env:VIRTUAL_ENV) {
     $NNI_PYTHON3 = $env:VIRTUAL_ENV + "\Scripts"
     $NNI_PKG_FOLDER = $env:VIRTUAL_ENV + "\nni"
     $NNI_PYTHON_SCRIPTS = $NNI_PYTHON3
 }
-else{
+else {
     $NNI_PYTHON3 = $(python -c 'import site; from pathlib import Path; print(Path(site.getsitepackages()[0]))')
     $NNI_PKG_FOLDER = $NNI_PYTHON3 + "\nni"
     $NNI_PYTHON_SCRIPTS = $NNI_PYTHON3 + "\Scripts"
 }
 
-$PIP_INSTALL = """$NNI_PYTHON3\python"" -m pip install ."
+$PIP_INSTALL = """$NNI_PYTHON3\python"" -m pip install "
 
-if(!(Test-Path $NNI_DEPENDENCY_FOLDER)){
+if (!(Test-Path $NNI_DEPENDENCY_FOLDER)) {
     New-Item $NNI_DEPENDENCY_FOLDER -ItemType Directory
 }
-$NNI_NODE_ZIP = $NNI_DEPENDENCY_FOLDER+"\nni-node.zip"
-$NNI_NODE_FOLDER = $NNI_DEPENDENCY_FOLDER+"\nni-node"
-$NNI_YARN_TARBALL = $NNI_DEPENDENCY_FOLDER+"\nni-yarn.tar.gz"
-$NNI_YARN_FOLDER = $NNI_DEPENDENCY_FOLDER+"\nni-yarn"
-$NNI_YARN = $NNI_YARN_FOLDER +"\bin\yarn"
+$NNI_NODE_ZIP = $NNI_DEPENDENCY_FOLDER + "\nni-node.zip"
+$NNI_NODE_FOLDER = $NNI_DEPENDENCY_FOLDER + "\nni-node"
+$NNI_YARN_TARBALL = $NNI_DEPENDENCY_FOLDER + "\nni-yarn.tar.gz"
+$NNI_YARN_FOLDER = $NNI_DEPENDENCY_FOLDER + "\nni-yarn"
+$NNI_YARN = $NNI_YARN_FOLDER + "\bin\yarn"
 
 ## Version number
 $NNI_VERSION_VALUE = $(git describe --tags)
 $NNI_VERSION_TEMPLATE = "999.0.0-developing"
 
-if(!(Test-Path $NNI_NODE_ZIP)){
+if (!(Test-Path $NNI_NODE_ZIP)) {
     Write-Host "Downloading Node..."
     (New-Object Net.WebClient).DownloadFile($nodeUrl, $NNI_NODE_ZIP)
 }
 
-if(!(Test-Path $NNI_YARN_TARBALL)){
+if (!(Test-Path $NNI_YARN_TARBALL)) {
     Write-Host "Downloading Yarn..."
     (New-Object Net.WebClient).DownloadFile($yarnUrl, $NNI_YARN_TARBALL)
 }
@@ -81,20 +83,23 @@ $SCRIPT =  "import tarfile",
 [System.IO.File]::WriteAllLines($SCRIPT_PATH, $SCRIPT)
 
 Add-Type -AssemblyName System.IO.Compression.FileSystem
-function Unzip{
+function Unzip {
     param([string]$zipfile, [string]$outpath)
     [System.IO.Compression.ZipFile]::ExtractToDirectory($zipfile, $outpath)
 }
 if ($install_node) {
     ### nodejs install
-    if(!(Test-Path $NNI_NODE_FOLDER)){
+    if (!(Test-Path $NNI_NODE_FOLDER)) {
         Unzip $NNI_NODE_ZIP $NNI_DEPENDENCY_FOLDER
         $unzipNodeDir = Get-ChildItem "$NNI_DEPENDENCY_FOLDER\$unzipNodeDir"
         Rename-Item $unzipNodeDir "nni-node"
     }
     Copy-Item "$NNI_NODE_FOLDER\node.exe" $NNI_PYTHON_SCRIPTS -Recurse -Force
+}
+
+if ($install_yarn) {
     ### yarn install
-    if(!(Test-Path $NNI_YARN_FOLDER)){
+    if (!(Test-Path $NNI_YARN_FOLDER)) {
         cmd /C """$NNI_PYTHON3\python""" $SCRIPT_PATH
         $unzipYarnDir = Get-ChildItem "$NNI_DEPENDENCY_FOLDER\$unzipYarnDir"
         Rename-Item $unzipYarnDir "nni-yarn"
@@ -104,10 +109,40 @@ if ($install_node) {
 ## install-python-modules:
 ### Installing Python SDK
 (Get-Content setup.py).replace($NNI_VERSION_TEMPLATE, $NNI_VERSION_VALUE) | Set-Content setup.py
-cmd /c $PIP_INSTALL
+
+if ($Development) {
+    $PYTHON_BUILD = "build"
+    if (Test-Path $PYTHON_BUILD) { 
+        # To compat with file and links.
+        cmd /c rmdir /s /q $PYTHON_BUILD
+    }
+    New-Item $PYTHON_BUILD -ItemType Directory
+    New-Item -ItemType Junction -Path "$($PYTHON_BUILD)\nni" -Target "src\sdk\pynni\nni"
+    New-Item -ItemType Junction -Path "$($PYTHON_BUILD)\nnicli" -Target "src\sdk\pycli\nnicli"
+    New-Item -ItemType Junction -Path "$($PYTHON_BUILD)\nni_annotation" -Target "tools\nni_annotation"
+    New-Item -ItemType Junction -Path "$($PYTHON_BUILD)\nni_cmd" -Target "tools\nni_cmd"
+    New-Item -ItemType Junction -Path "$($PYTHON_BUILD)\nni_trial_tool" -Target "tools\nni_trial_tool"
+    New-Item -ItemType Junction -Path "$($PYTHON_BUILD)\nni_gpu_tool" -Target "tools\nni_gpu_tool"
+
+    Copy-Item setup.py $PYTHON_BUILD
+    Copy-Item README.md $PYTHON_BUILD
+
+    Push-Location build
+    #update folders in setup file
+    (Get-Content setup.py).replace("src/sdk/pynni/", "") | Set-Content setup.py
+    (Get-Content setup.py).replace("src/sdk/pycli/", "") | Set-Content setup.py
+    (Get-Content setup.py).replace("src/sdk/pynni", ".") | Set-Content setup.py
+    (Get-Content setup.py).replace("tools/", "") | Set-Content setup.py
+    # install current folder.
+    cmd /c $PIP_INSTALL -e .
+    Pop-Location
+}
+else {
+    cmd /c $PIP_INSTALL .
+}
 
 # Building NNI Manager
-$env:PATH=$NNI_PYTHON_SCRIPTS+';'+$env:PATH
+$env:PATH = $NNI_PYTHON_SCRIPTS + ';' + $env:PATH
 cd src\nni_manager
 cmd /c $NNI_YARN
 cmd /c $NNI_YARN build
@@ -124,17 +159,31 @@ cmd /c $NNI_YARN build
 cd ..\..
 
 ## install-node-modules
-if(!(Test-Path $NNI_PKG_FOLDER)){
-    New-Item $NNI_PKG_FOLDER -ItemType Directory
+if (Test-Path $NNI_PKG_FOLDER) {
+    # it needs to remove the whole folder for following copy.
+    cmd /c rmdir /s /q $NNI_PKG_FOLDER
+}
+
+$NNI_PKG_FOLDER_STATIC = $NNI_PKG_FOLDER + "\static"
+$NASUI_PKG_FOLDER = $NNI_PKG_FOLDER + "\nasui"
+
+if ($Development) {
+    New-Item -ItemType Junction -Path $($NNI_PKG_FOLDER) -Target "src\nni_manager\dist"
+    New-Item -ItemType Junction -Path "$($NNI_PKG_FOLDER)\node_modules" -Target "src\nni_manager\node_modules"
+    New-Item -ItemType Junction -Path $($NNI_PKG_FOLDER_STATIC) -Target "src\webui\build"
+    New-Item -ItemType Junction -Path $($NASUI_PKG_FOLDER) -Target "src\nasui\build"
 }
-Remove-Item $NNI_PKG_FOLDER -Recurse -Force
-Copy-Item "src\nni_manager\dist" $NNI_PKG_FOLDER -Recurse
+else {
+    Copy-Item "src\nni_manager\dist" $NNI_PKG_FOLDER -Recurse
+    Copy-Item "src\webui\build" $NNI_PKG_FOLDER_STATIC -Recurse
+    Copy-Item "src\nasui\build" $NASUI_PKG_FOLDER -Recurse
+}
+
 Copy-Item "src\nni_manager\package.json" $NNI_PKG_FOLDER
 $PKG_JSON = $NNI_PKG_FOLDER + "\package.json"
 (Get-Content $PKG_JSON).replace($NNI_VERSION_TEMPLATE, $NNI_VERSION_VALUE) | Set-Content $PKG_JSON
-cmd /c $NNI_YARN --prod --cwd $NNI_PKG_FOLDER
-$NNI_PKG_FOLDER_STATIC = $NNI_PKG_FOLDER + "\static"
-$NASUI_PKG_FOLDER = $NNI_PKG_FOLDER + "\nasui"
-Copy-Item "src\webui\build" $NNI_PKG_FOLDER_STATIC -Recurse
-Copy-Item "src\nasui\build" $NASUI_PKG_FOLDER -Recurse
 Copy-Item "src\nasui\server.js" $NASUI_PKG_FOLDER -Recurse
+
+if (!$Development) {
+    cmd /c $NNI_YARN --prod --cwd $NNI_PKG_FOLDER
+}

src/nni_manager/core/ipcInterface.ts

@@ -23,11 +23,7 @@ const ipcIncomingFd: number = 4;
  */
 function encodeCommand(commandType: string, content: string): Buffer {
     const contentBuffer: Buffer = Buffer.from(content);
-    if (contentBuffer.length >= 1_000_000) {
-        throw new RangeError('Command too long');
-    }
-    const contentLengthBuffer: Buffer = Buffer.from(contentBuffer.length.toString().padStart(6, '0'));
-
+    const contentLengthBuffer: Buffer = Buffer.from(contentBuffer.length.toString().padStart(14, '0'));
     return Buffer.concat([Buffer.from(commandType), contentLengthBuffer, contentBuffer]);
 }
 
@@ -43,12 +39,12 @@ function decodeCommand(data: Buffer): [boolean, string, string, Buffer] {
         return [false, '', '', data];
     }
     const commandType: string = data.slice(0, 2).toString();
-    const contentLength: number = parseInt(data.slice(2, 8).toString(), 10);
-    if (data.length < contentLength + 8) {
+    const contentLength: number = parseInt(data.slice(2, 16).toString(), 10);
+    if (data.length < contentLength + 16) {
         return [false, '', '', data];
     }
-    const content: string = data.slice(8, contentLength + 8).toString();
-    const remain: Buffer = data.slice(contentLength + 8);
+    const content: string = data.slice(16, contentLength + 16).toString();
+    const remain: Buffer = data.slice(contentLength + 16);
 
     return [true, commandType, content, remain];
 }

src/nni_manager/core/test/assessor.py

@@ -8,13 +8,13 @@ _out_file = open(4, 'wb')
 def send(command, data):
     command = command.encode('utf8')
     data = data.encode('utf8')
-    msg = b'%b%06d%b' % (command, len(data), data)
+    msg = b'%b%14d%b' % (command, len(data), data)
     _out_file.write(msg)
     _out_file.flush()
 
 
 def receive():
-    header = _in_file.read(8)
+    header = _in_file.read(16)
     l = int(header[2:])
     command = header[:2].decode('utf8')
     data = _in_file.read(l).decode('utf8')

src/nni_manager/core/test/ipcInterface.test.ts

@@ -14,7 +14,6 @@ import { NNIError } from '../../common/errors';
 let sentCommands: { [key: string]: string }[] = [];
 const receivedCommands: { [key: string]: string }[] = [];
 
-let commandTooLong: Error | undefined;
 let rejectCommandType: Error | undefined;
 
 function runProcess(): Promise<Error | null> {
@@ -54,14 +53,7 @@ function runProcess(): Promise<Error | null> {
     // Command #2: ok
     dispatcher.sendCommand('ME', '123');
 
-    // Command #3: too long
-    try {
-        dispatcher.sendCommand('ME', 'x'.repeat(1_000_000));
-    } catch (error) {
-        commandTooLong = error;
-    }
-
-    // Command #4: FE is not tuner/assessor command, test the exception type of send non-valid command
+    // Command #3: FE is not tuner/assessor command, test the exception type of send non-valid command
     try {
         dispatcher.sendCommand('FE', '1');
     } catch (error) {
@@ -88,21 +80,11 @@ describe('core/protocol', (): void => {
     });
 
     it('sendCommand() should work without content', (): void => {
-        assert.equal(sentCommands[0], '(\'IN\', \'\')');
+        assert.equal(sentCommands[0], "('IN', '')");
     });
 
     it('sendCommand() should work with content', (): void => {
-        assert.equal(sentCommands[1], '(\'ME\', \'123\')');
-    });
-
-    it('sendCommand() should throw on too long command', (): void => {
-        if (commandTooLong === undefined) {
-            assert.fail('Should throw error')
-        } else {
-            const err: Error | undefined = (<NNIError>commandTooLong).cause;
-            assert(err && err.name === 'RangeError');
-            assert(err && err.message === 'Command too long');
-        }
+        assert.equal(sentCommands[1], "('ME', '123')");
     });
 
     it('sendCommand() should throw on wrong command type', (): void => {

src/sdk/pynni/nni/nas/pytorch/fixed.py

@@ -2,12 +2,16 @@
 # Licensed under the MIT license.
 
 import json
+import logging
 
 from .mutables import InputChoice, LayerChoice, MutableScope
 from .mutator import Mutator
 from .utils import to_list
 
 
+_logger = logging.getLogger(__name__)
+
+
 class FixedArchitecture(Mutator):
     """
     Fixed architecture mutator that always selects a certain graph.
@@ -73,6 +77,41 @@ class FixedArchitecture(Mutator):
         """
         return self._fixed_arc
 
+    def replace_layer_choice(self, module=None, prefix=""):
+        """
+        Replace layer choices with selected candidates. It's done with best effort.
+        In case of weighted choices or multiple choices. if some of the choices on weighted with zero, delete them.
+        If single choice, replace the module with a normal module.
+
+        Parameters
+        ----------
+        module : nn.Module
+            Module to be processed.
+        prefix : str
+            Module name under global namespace.
+        """
+        if module is None:
+            module = self.model
+        for name, mutable in module.named_children():
+            global_name = (prefix + "." if prefix else "") + name
+            if isinstance(mutable, LayerChoice):
+                chosen = self._fixed_arc[mutable.key]
+                if sum(chosen) == 1 and max(chosen) == 1 and not mutable.return_mask:
+                    # sum is one, max is one, there has to be an only one
+                    # this is compatible with both integer arrays, boolean arrays and float arrays
+                    _logger.info("Replacing %s with candidate number %d.", global_name, chosen.index(1))
+                    setattr(module, name, mutable[chosen.index(1)])
+                else:
+                    if mutable.return_mask:
+                        _logger.info("`return_mask` flag of %s is true. As it relies on the behavior of LayerChoice, " \
+                                     "LayerChoice will not be replaced.")
+                    # remove unused parameters
+                    for ch, n in zip(chosen, mutable.names):
+                        if ch == 0 and not isinstance(ch, float):
+                            setattr(mutable, n, None)
+            else:
+                self.replace_layer_choice(mutable, global_name)
+
 
 def apply_fixed_architecture(model, fixed_arc):
     """
@@ -96,4 +135,7 @@ def apply_fixed_architecture(model, fixed_arc):
             fixed_arc = json.load(f)
     architecture = FixedArchitecture(model, fixed_arc)
     architecture.reset()
+
+    # for the convenience of parameters counting
+    architecture.replace_layer_choice()
     return architecture

src/sdk/pynni/nni/networkmorphism_tuner/test_networkmorphism_tuner.py

@@ -102,7 +102,7 @@ class NetworkMorphismTestCase(TestCase):
         graph_init = CnnGenerator(10, (32, 32, 3)).generate()
         json_out = graph_to_json(graph_init, "temp.json")
         graph_recover = json_to_graph(json_out)
-        deeper_graph = to_wider_graph(deepcopy(graph_recover))
+        deeper_graph = to_deeper_graph(deepcopy(graph_recover))
         model = deeper_graph.produce_torch_model()
         out = model(torch.ones(1, 3, 32, 32))
         self.assertEqual(out.shape, torch.Size([1, 10]))
@@ -114,7 +114,7 @@ class NetworkMorphismTestCase(TestCase):
         graph_init = CnnGenerator(10, (32, 32, 3)).generate()
         json_out = graph_to_json(graph_init, "temp.json")
         graph_recover = json_to_graph(json_out)
-        skip_connection_graph = to_wider_graph(deepcopy(graph_recover))
+        skip_connection_graph = to_skip_connection_graph(deepcopy(graph_recover))
         model = skip_connection_graph.produce_torch_model()
         out = model(torch.ones(1, 3, 32, 32))
         self.assertEqual(out.shape, torch.Size([1, 10]))

src/sdk/pynni/nni/protocol.py

@@ -43,8 +43,7 @@ def send(command, data):
     try:
         _lock.acquire()
         data = data.encode('utf8')
-        assert len(data) < 1000000, 'Command too long'
-        msg = b'%b%06d%b' % (command.value, len(data), data)
+        msg = b'%b%014d%b' % (command.value, len(data), data)
         logging.getLogger(__name__).debug('Sending command, data: [%s]', msg)
         _out_file.write(msg)
         _out_file.flush()
@@ -56,9 +55,9 @@ def receive():
     """Receive a command from Training Service.
     Returns a tuple of command (CommandType) and payload (str)
     """
-    header = _in_file.read(8)
+    header = _in_file.read(16)
     logging.getLogger(__name__).debug('Received command, header: [%s]', header)
-    if header is None or len(header) < 8:
+    if header is None or len(header) < 16:
         # Pipe EOF encountered
         logging.getLogger(__name__).debug('Pipe EOF encountered')
         return None, None

src/sdk/pynni/tests/test_protocol.py

@@ -20,30 +20,21 @@ class ProtocolTestCase(TestCase):
     def test_send_en(self):
         out_file = _prepare_send()
         send(CommandType.NewTrialJob, 'CONTENT')
-        self.assertEqual(out_file.getvalue(), b'TR000007CONTENT')
+        self.assertEqual(out_file.getvalue(), b'TR00000000000007CONTENT')
 
     def test_send_zh(self):
         out_file = _prepare_send()
         send(CommandType.NewTrialJob, '你好')
-        self.assertEqual(out_file.getvalue(), 'TR000006你好'.encode('utf8'))
-
-    def test_send_too_large(self):
-        _prepare_send()
-        exception = None
-        try:
-            send(CommandType.NewTrialJob, ' ' * 1000000)
-        except AssertionError as e:
-            exception = e
-        self.assertIsNotNone(exception)
+        self.assertEqual(out_file.getvalue(), 'TR00000000000006你好'.encode('utf8'))
 
     def test_receive_en(self):
-        _prepare_receive(b'IN000005hello')
+        _prepare_receive(b'IN00000000000005hello')
         command, data = receive()
         self.assertIs(command, CommandType.Initialize)
         self.assertEqual(data, 'hello')
 
     def test_receive_zh(self):
-        _prepare_receive('IN000006世界'.encode('utf8'))
+        _prepare_receive('IN00000000000006世界'.encode('utf8'))
         command, data = receive()
         self.assertIs(command, CommandType.Initialize)
         self.assertEqual(data, '世界')

src/webui/package.json

@@ -65,7 +65,6 @@
     "@typescript-eslint/eslint-plugin": "^2.11.0",
     "@typescript-eslint/parser": "^2.11.0",
     "@uifabric/fluent-theme": "^0.16.7",
-    "npx": "^10.2.0",
     "eslint": "^5.16.0",
     "eslint-config-react-app": "^4.0.0",
     "eslint-loader": "2.1.2",
@@ -74,6 +73,7 @@
     "eslint-plugin-jsx-a11y": "6.2.1",
     "eslint-plugin-react": "7.12.4",
     "eslint-plugin-react-hooks": "^1.5.0",
+    "npx": "^10.2.0",
     "typescript": "3.4.5"
   },
   "scripts": {

src/webui/src/components/trial-detail/TableList.tsx

@@ -39,6 +39,11 @@ interface TableListProps {
     trialsUpdateBroadcast: number;
 }
 
+interface SortInfo {
+    field: string;
+    isDescend?: boolean;
+}
+
 interface TableListState {
     intermediateOption: object;
     modalVisible: boolean;
@@ -60,9 +65,9 @@ interface TableListState {
     tableColumns: IColumn[];
     allColumnList: string[];
     tableSourceForSort: Array<TableRecord>;
+    sortMessage: SortInfo;
 }
 
-
 class TableList extends React.Component<TableListProps, TableListState> {
 
     public intervalTrialLog = 10;
@@ -91,7 +96,8 @@ class TableList extends React.Component<TableListProps, TableListState> {
             modalIntermediateHeight: window.innerHeight,
             tableColumns: this.initTableColumnList(this.props.columnList),
             allColumnList: this.getAllColumnKeys(),
-            tableSourceForSort: this.props.tableSource
+            tableSourceForSort: this.props.tableSource,
+            sortMessage: { field: '', isDescend: false }
         };
     }
 
@@ -114,19 +120,29 @@ class TableList extends React.Component<TableListProps, TableListState> {
         const newItems = this.copyAndSort(tableSource, currColumn.fieldName!, currColumn.isSortedDescending);
         this.setState({
             tableColumns: newColumns,
-            tableSourceForSort: newItems
+            tableSourceForSort: newItems,
+            sortMessage: { field: getColumn.key, isDescend: currColumn.isSortedDescending }
         });
+
     };
 
-    private copyAndSort<T>(items: T[], columnKey: string, isSortedDescending?: boolean): T[] {
+    private copyAndSort<T>(items: T[], columnKey: string, isSortedDescending?: boolean): any {
         const key = columnKey as keyof T;
-        return items.slice(0).sort((a: T, b: T) => ((isSortedDescending ? a[key] < b[key] : a[key] > b[key]) ? 1 : -1));
+        return items.slice(0).sort(function (a: T, b: T): any {
+            if (a[key] === undefined) {
+                return 1;
+            }
+            if (b[key] === undefined) {
+                return -1;
+            }
+            return (isSortedDescending ? a[key] < b[key] : a[key] > b[key]) ? 1 : -1;
+        });
     }
 
     AccuracyColumnConfig: any = {
         name: 'Default metric',
         className: 'leftTitle',
-        key: 'accuracy',
+        key: 'latestAccuracy',
         fieldName: 'latestAccuracy',
         minWidth: 200,
         maxWidth: 300,
@@ -143,7 +159,7 @@ class TableList extends React.Component<TableListProps, TableListState> {
         minWidth: 80,
         maxWidth: 240,
         className: 'tableHead',
-        data: 'string',
+        data: 'number',
         onColumnClick: this.onColumnClick,
     };
 
@@ -566,10 +582,21 @@ class TableList extends React.Component<TableListProps, TableListState> {
         const { intermediateKey, modalIntermediateWidth, modalIntermediateHeight,
             tableColumns, allColumnList, isShowColumn, modalVisible,
             selectRows, isShowCompareModal, intermediateOtherKeys,
-            isShowCustomizedModal, copyTrialId, intermediateOption
+            isShowCustomizedModal, copyTrialId, intermediateOption, sortMessage
         } = this.state;
         const { columnList } = this.props;
         const tableSource: Array<TableRecord> = JSON.parse(JSON.stringify(this.state.tableSourceForSort));
+        if (sortMessage.field !== '') {
+            tableSource.sort(function (a, b): any {
+                if (a[sortMessage.field] === undefined) {
+                    return 1;
+                }
+                if (b[sortMessage.field] === undefined) {
+                    return -1;
+                }
+                return (sortMessage.isDescend ? a[sortMessage.field] < b[sortMessage.field] : a[sortMessage.field] > b[sortMessage.field]) ? 1 : -1;
+            });
+        }
         return (
             <Stack>
                 <div id="tableList">

src/webui/yarn.lock

@@ -5371,8 +5371,9 @@ kind-of@^5.0.0:
   resolved "https://registry.yarnpkg.com/kind-of/-/kind-of-5.1.0.tgz#729c91e2d857b7a419a1f9aa65685c4c33f5845d"
 
 kind-of@^6.0.0, kind-of@^6.0.2:
-  version "6.0.2"
-  resolved "https://registry.yarnpkg.com/kind-of/-/kind-of-6.0.2.tgz#01146b36a6218e64e58f3a8d66de5d7fc6f6d051"
+  version "6.0.3"
+  resolved "https://registry.yarnpkg.com/kind-of/-/kind-of-6.0.3.tgz#07c05034a6c349fa06e24fa35aa76db4580ce4dd"
+  integrity sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==
 
 last-call-webpack-plugin@^3.0.0:
   version "3.0.0"

test/config/training_service.yml

@@ -78,6 +78,9 @@ paiYarn:
 pai:
   nniManagerIp:
   maxExecDuration: 15m
+  # PAI has job submission limitation, set maxTrialNum=1 to control trial job numbers for PAI 
+  maxTrialNum: 1
+  trialConcurrency: 1
   paiConfig:
     host:
     userName:

uninstall.ps1

@@ -4,12 +4,12 @@ $env:PYTHONIOENCODING = "UTF-8"
 if($env:VIRTUAL_ENV){
     $NNI_PYTHON3 = $env:VIRTUAL_ENV + "\Scripts"
     $NNI_PKG_FOLDER = $env:VIRTUAL_ENV + "\nni"
-    Remove-Item "$NNI_PYTHON3\node.exe" -Force
+    cmd /c del "$NNI_PYTHON3\node.exe"
 }
 else{
     $NNI_PYTHON3 = $(python -c 'import site; from pathlib import Path; print(Path(site.getsitepackages()[0]))')
     $NNI_PKG_FOLDER = $NNI_PYTHON3 + "\nni"
-    Remove-Item "$NNI_PYTHON3\Scripts\node.exe" -Force
+    cmd /c del "$NNI_PYTHON3\Scripts\node.exe"
 }
 
 $PIP_UNINSTALL = """$NNI_PYTHON3\python"" -m pip uninstall -y "
@@ -17,13 +17,16 @@ $NNI_NODE_FOLDER = $NNI_DEPENDENCY_FOLDER+"\nni-node"
 $NNI_YARN_FOLDER = $NNI_DEPENDENCY_FOLDER+"\nni-yarn"
 
 # uninstall
-Remove-Item $NNI_PKG_FOLDER -Recurse -Force
-cmd /C $PIP_UNINSTALL "nni"
+cmd /c rmdir /s /q $NNI_PKG_FOLDER
+cmd /c $PIP_UNINSTALL "nni"
 
-# clean
-Remove-Item "src/nni_manager/dist" -Recurse -Force
-Remove-Item "src/nni_manager/node_modules" -Recurse -Force
-Remove-Item "src/webui/build" -Recurse -Force
-Remove-Item "src/webui/node_modules" -Recurse -Force
-Remove-Item $NNI_YARN_FOLDER -Recurse -Force
-Remove-Item $NNI_NODE_FOLDER -Recurse -Force
+# clean up
+cmd /c rmdir /s /q "build"
+cmd /c rmdir /s /q "src\nni_manager\dist"
+cmd /c rmdir /s /q  "src\nni_manager\node_modules"
+cmd /c rmdir /s /q  "src\webui\build"
+cmd /c rmdir /s /q  "src\webui\node_modules"
+cmd /c rmdir /s /q  "src\nasui\build"
+cmd /c rmdir /s /q  "src\nasui\node_modules"
+cmd /c rmdir /s /q  $NNI_YARN_FOLDER
+cmd /c rmdir /s /q  $NNI_NODE_FOLDER