GitLab

Designed new interface to support reusable training service, currently only applies to OpenPAI, and default disabled.

Replace trial_keeper.py to trial_runner.py, trial_runner holds an environment, and receives commands from nni manager to run or stop an trial, and return events to nni manager.
Add trial dispatcher, which inherits from original trianing service interface. It uses to share as many as possible code of all training service, and isolate with training services.
Add EnvironmentService interface to manage environment, including start/stop an environment, refresh status of environments.
Add command channel on both nni manager and trial runner parts, it supports different ways to pass messages between them. Current supported channels are file, web sockets. and supported commands from nni manager are start, kill trial, send new parameters; from runner are initialized(support some channel doesn't know which runner connected), trial end, stdout ((new type), including metric like before), version check (new type), gpu info (new type).
Add storage service to wrapper a storage to standard file operations, like NFS, azure storage and so on.
Partial support run multiple trials in parallel on runner side, but not supported by trial dispatcher side.
Other minor changes,

Add log_level to TS UT, so that UT can show debug level log.
Expose platform to start info.
Add RouterTrainingService to keep origianl OpenPAI training service, and support dynamic IOC binding.
Add more GPU info for future usage, including GPU mem total/free/used, gpu type.
Make some license information consistence.
Fix async/await problems on Array.forEach, this method doesn't support async actually.
Fix IT errors on download data, which causes by my #2484 .
Accelerate some run loop pattern by reducing sleep seconds.

* modify docs

* "nni_search_space.json" -> "search_space.json"

* squash commits in v1.0 first round bug bash

* change authFile to local path

* update readme in ga_squad

* update readme

* fix typo

* Update README.md

* Update README.md

* Update README.md

* update readme

* update

* fix path

* update reference

* fix bug in config file

* update nni_arch_overview.png

* update

* update

* update

* update home page

* update default value of metis tuner

* fix broken link in CommunitySharings

* update docs about nested search space

* update docs

* rename cascding to nested

* fix broken link

* update

* update issue link

* fix typo

* update evaluate parameters from GMM

* refine code

* fix optimized mode bug

* update import warning

* update warning

* update optimized mode

* first commit for update doc structure

* mv the localmode and remotemode to traningservice

* update

* update docs

* Fix bug: incorrect number of trial jobs in hyperband

fix typo

* move en docs under en_US

* update gitignore

* fix image link

* add doc guide

* add napoleon ext

Change pai to OpenPAI
reformat for translation
fix some typo.
rewording.

* fix lex format for code

* fix doc links

* hide api reference

* delete orphan files

* fix deadlink in FAQ

* remove email address

* fix deadlinks for deleted orphan files

fix indents of doc,
change nni to NNI
yaml to yml(file) and YAML(doc)

Fix document with formats and typos

* Update nnictl.py

Fix the issue that nnictl --version via pip installation doesn't work

* Update kubeflow training service document (#494)

* Remove kubectl related document, add messages for kubeconfig
* Add design section for kubeflow training service
* Move the image files for PAI training service doc into img folder.

* Update KubeflowMode.md (#498)

Update KubeflowMode.md, small terms change

* [V0.4.1 bug fix] Cannot run kubeflow training service due to trial_keeper change (#503)

* Update kubeflow training service document

* fix bug a that kubeflow trial job cannot run

* upgrade version number (#499)

* [V0.4.1 bug fix] Support read K8S config from KUBECONFIG environment variable (#507)

* Add KUBCONFIG env variable support

* In main.ts, throw cached error to make sure nnictl can show the error in stderr

* Update document for PAI mode port

* Update document for PAI mode to turn on 8081 port

* fix nnictl bug

* fix nnictl create bug

* add experiment status logic

* add more information for nnictl

* fix Evolution Tuner bug

* refactor code

* fix code in updater.py

* fix nnictl --help

* fix classArgs bug

* update check response.status_code logic

* show trial log path

* update document

* fix install.sh

* set default vallue for maxTrialNum and maxExecDuration

* fix nnictl

* fix config path hint

* support multiPhase

* fix bash-completion

* refactor bash-completion

* add sklearn-regression

* add search_space

* fix bug

* fix install.sh

* refactor code

* remove unused code

* support multi experiments

* fix issues

* Support multiple experiments of nnictl (#183)

* fix nnictl bug

* fix nnictl create bug

* add experiment status logic

* add more information for nnictl

* fix Evolution Tuner bug

* refactor code

* fix code in updater.py

* fix nnictl --help

* fix classArgs bug

* update check response.status_code logic

* show trial log path

* update document

* fix install.sh

* set default vallue for maxTrialNum and maxExecDuration

* fix nnictl

* fix config path hint

* support multiPhase

* fix bash-completion

* refactor bash-completion

* add sklearn-regression

* add search_space

* fix bug

* fix install.sh

* refactor code

* remove unused code

* support multi experiments

* fix issues

* Let nni manager web server handle static content

* set nnictl stop require the port

* Support multiple experiments of nnictl (#183)

* fix nnictl bug

* fix nnictl create bug

* add experiment status logic

* add more information for nnictl

* fix Evolution Tuner bug

* refactor code

* fix code in updater.py

* fix nnictl --help

* fix classArgs bug

* update check response.status_code logic

* show trial log path

* update document

* fix install.sh

* set default vallue for maxTrialNum and maxExecDuration

* fix nnictl

* fix config path hint

* support multiPhase

* fix bash-completion

* refactor bash-completion

* add sklearn-regression

* add search_space

* fix bug

* fix install.sh

* refactor code

* remove unused code

* support multi experiments

* fix issues

* Let nni manager web server handle static content

* Dev multiple experiments (#189)

* fix nnictl bug

* fix nnictl create bug

* add experiment status logic

* add more information for nnictl

* fix Evolution Tuner bug

* refactor code

* fix code in updater.py

* fix nnictl --help

* fix classArgs bug

* update check response.status_code logic

* show trial log path

* update document

* fix install.sh

* set default vallue for maxTrialNum and maxExecDuration

* fix nnictl

* fix config path hint

* support multiPhase

* fix bash-completion

* refactor bash-completion

* add sklearn-regression

* add search_space

* fix bug

* fix install.sh

* refactor code

* remove unused code

* support multi experiments

* fix issues

* set nnictl stop require the port

* Update documents for supporting multiple experiments

* create a constant variable for 51188

* Fixed issue that WebUI can not refresh page

* Upgrade Node.js and Yarn to latest version

* Update v0.2 documentation regards to release note and PAI training service

* Update document to describe NNI docker image

* webui logpath and document (#135)

* Add webui document and logpath as a href

* fix tslint

* fix comments by Chengmin

* Pai training service bug fix and enhancement (#136)

* Add NNI installation scripts

* Update pai script, update NNI_out_dir

* Update NNI dir in nni sdk local.py

* Create .nni folder in nni sdk local.py

* Add check before creating .nni folder

* Fix typo for PAI_INSTALL_NNI_SHELL_FORMAT

* Improve annotation (#138)

* Improve annotation

* Minor bugfix

* Selectively install through pip (#139)

Selectively install through pip 
* update setup.py

* fix paiTrainingService bugs (#137)

* fix nnictl bug

* add hdfs host validation

* fix bugs

* fix dockerfile

* fix install.sh

* update install.sh

* fix dockerfile

* Set timeout for HDFSUtility exists function

* remove unused TODO

* fix sdk

* add optional for outputDir and dataDir

* refactor dockerfile.base

* Remove unused import in hdfsclientUtility

* Add documentation for NNI PAI mode experiment (#141)

* Add documentation for NNI PAI mode

* Fix typo based on PR comments

* Exit with subprocess return code of trial keeper

* Remove additional exit code

* Fix typo based on PR comments

* update doc for smac tuner (#140)

* Revert "Selectively install through pip (#139)" due to potential pip install issue (#142)

* Revert "Selectively install through pip (#139)"

This reverts commit 1d174836.

* Add exit code of subprocess for trial_keeper

* Update README, add link to PAImode doc

* Add documentation for NNI PAI mode

* Fix typo based on PR comments

* Exit with subprocess return code of trial keeper

* Remove additional exit code

* Fix typo based on PR comments