GitLab

Support intermediate is dict

SSH client has a max number of open channels for a connection, if we set the number of trialCurrency too big, our ssh client will exec command using ssh frequently, then we will meet the error of Error: (SSH) Channel open failure: open failed.
Refactor the code, set one connection has a max trial concurrency, when the number of trial reach the ssh connection restriction, will create a new ssh connection to exec trial commands.

* fix remote bug

* add document

* add document

* fix remote issue

* fix forEach

* update doc according to comments

* remove 'any more'

* set nniManager.log and dispatcher.log time format to local time

* Refactoring test pipelines, move local integrate test from PR pipeline into a sepearate pipeline which is scheduled daily.

* add epsilon

* add epsilon for ceil

In current version of makefile, if user use make dev-install, they will install two packages "nni-sdk" and "nnictl" in their environment.
Now merge the two package to be one package "nni".

fix unittest err exit

* Install dependencies for PAI/k8s IT

* updates

* fix annotation, resolve annotation's key err bug, refactor the design

* fix remote bug

* add document

* add document

* update

* update

* update

* update

* fix remote issue

* fix forEach

* update doc according to comments

* update

* update

* update

* remove 'any more'

* change multi_phast trial concurrency

* Fix tensorflow MNIST dataset download bug in multiple trials

* Modify the maxTrialNum and trialConcurrency in config

* add different tuner config files for config_test

* change MetisTuner config test due to no lightgbm python module in integration test

* install smac package in azure-pipelines

* SMAC need swig to be installed

* Try to install swig from source code

* remove SMAC test because the dependency can not be installed

* use sudo to install the swig

* sleep 10s to make sure the port has been released

* remove tuner test for networkmorphism because it uses more than 30s to release the tcp port

* word "down" to "done"

* add config test for Curvefitting assessor

* change file name

* Fix data type not match bug

* Optimize MetisTunner

* pretty the code

* Follow the review comment

* add exploration probability

* Avoid None type object generating

* 0.5.2 release note

* fix bug in cold start

* remove unused comment

Fix bug in Curvefitting Assessor

* Add build_version input for build pipeline
* Change time stamp to UTC time

1. Change timestamp to UTC time
2. Use configurable docker hub account to upload integration test docker image

add package requirement data in setup.py

* add different tuner config files for config_test

* change MetisTuner config test due to no lightgbm python module in integration test

* install smac package in azure-pipelines

* SMAC need swig to be installed

* Try to install swig from source code

* remove SMAC test because the dependency can not be installed

* use sudo to install the swig

* sleep 10s to make sure the port has been released

* remove tuner test for networkmorphism because it uses more than 30s to release the tcp port

* word "down" to "done"

* add config test for Curvefitting assessor

* change file name

* Fix data type not match bug

* Optimize MetisTunner

* pretty the code

* Follow the review comment

* add exploration probability

Update readme for 0.5.2

* update link

* update readme in ga_squad

* update readme

* fix typo

* Update README.md

* Update README.md

* Update README.md

* update readme

* update

* fix path

* update reference

* fix bug in config file

* update nni_arch_overview.png

* update

* update

* update

* update home page

* quickly fix bug in cold start

by adding extra rst file 

* fix sklearn dependency
* add doc badge to readme
* fix infinite toctree level

fix broken link for curfitting assessor doc

Update nni docker image build process in build pipeline

In trialkeeper, the output stream of trial process may be in cache, and multiphase need the output stream returns immediately. Flush when sdk print content.

Improve the performance of the curve fitting assessor 

Add integration test for remoteTrainingService platform

Set pai/kubeflow integration test job timeout to infinite.

* change version

trial_keeper use 50070 port to connect to webhdfs server, and PAI use a mapping method to map 50070 port to 5070 port to visit restful server, this method has some risk for PAI may not support this kind of mapping in later release.Now use Pylon path(/webhdfs/api/v1) instead of 50070 port in webhdfs client of trial_keeper, the path is transmitted in trainingService.
In this pr, we have these changes:

1. Change to use webhdfs path instead of 50070 port in hdfs client.
2. Change to use new hdfs package "PythonWebHDFS", which is build to support pylon by myself. You could test the new function from "sparksnail/nni:dev-pai" image to test pai trainingService.
3. Update some variables' name according to comments.