GitLab

Designed new interface to support reusable training service, currently only applies to OpenPAI, and default disabled.

Replace trial_keeper.py to trial_runner.py, trial_runner holds an environment, and receives commands from nni manager to run or stop an trial, and return events to nni manager.
Add trial dispatcher, which inherits from original trianing service interface. It uses to share as many as possible code of all training service, and isolate with training services.
Add EnvironmentService interface to manage environment, including start/stop an environment, refresh status of environments.
Add command channel on both nni manager and trial runner parts, it supports different ways to pass messages between them. Current supported channels are file, web sockets. and supported commands from nni manager are start, kill trial, send new parameters; from runner are initialized(support some channel doesn't know which runner connected), trial end, stdout ((new type), including metric like before), version check (new type), gpu info (new type).
Add storage service to wrapper a storage to standard file operations, like NFS, azure storage and so on.
Partial support run multiple trials in parallel on runner side, but not supported by trial dispatcher side.
Other minor changes,

Add log_level to TS UT, so that UT can show debug level log.
Expose platform to start info.
Add RouterTrainingService to keep origianl OpenPAI training service, and support dynamic IOC binding.
Add more GPU info for future usage, including GPU mem total/free/used, gpu type.
Make some license information consistence.
Fix async/await problems on Array.forEach, this method doesn't support async actually.
Fix IT errors on download data, which causes by my #2484 .
Accelerate some run loop pattern by reducing sleep seconds.

others

1. fix failed curve fitting UTs, due to code changes.
1. move all SDK UTs to tests folder, so that they can be run in default tests.
1. fix some deprecated ut assert function calls.

* skeleton of dlts training service (#1844)

* Hello, DLTS!

* Revert version

* Remove fs-extra

* Add some default cluster config

* schema

* fix

* Optional cluster (default to `.default`)

Depends on DLWorkspace#837

* fix

* fix

* optimize gpu type

* No more copy

* Format

* Code clean up

* Issue fix

* Add optional fields in config

* Issue fix

* Lint

* Lint

* Validate email, password and team

* Doc

* Doc fix

* Set TMPDIR

* Use metadata instead of gpu_capacity

* Cancel paused DLTS job

* workaround lint rules

* pylint

* doc
Co-authored-by: QuanluZhang <z.quanluzhang@gmail.com>

Fix pylint errors

* support specifying gpu for tuner and advisor

* PPO tuner for NAS, supports NNI's NAS interface (#1380)

* squash commits in v1.0 first round bug bash

* change authFile to local path

* dev oneshot and darts mode, add mnist example

* update readme in ga_squad

* update readme

* fix typo

* Update README.md

* Update README.md

* Update README.md

* update readme

* update

* fix path

* update reference

* fix bug in config file

* update nni_arch_overview.png

* update

* update

* update

* update home page

* update default value of metis tuner

* fix broken link in CommunitySharings

* update docs about nested search space

* update docs

* rename cascding to nested

* fix broken link

* update

* update issue link

* fix typo

* update evaluate parameters from GMM

* refine code

* fix optimized mode bug

* update import warning

* update warning

* update optimized mode

* fix parallel_tpe bug and refine

* fix bug

* fix kwargs bug

* fix bug

* update the default value of parallel_tpe

* add docs for parallel-tpe

* fix config_schema bug

* update community doc

* fix typo

* update doc for parallel_tpe

* fix bug in receive trial result

* update by comments

* update by comments

* fix self.optimal_y is 0 when mean mode

* update

* update

* update rst

* fix typo | add docstring

* fix includeintermediateresult

* test pr tigger

* add evolution field

* fix link err in docs

* add spaces

* re-organise links for detailed descriptions of the tuners and accessors; fix link err in HpoComparision.md

* add in-page link by change .md to .html

* delete #section from cross-file links to make links work in both readthedocs and github docs

* gp_tuner init from fmfn's repo

* fix params bug by adding float>int transition

* add optimal choices; support randint&quniform type; add doc

* refine doc and code

* change mnist yml comments

* typo fix

* fix val err

* fix minimize mode err

* add config test and Hpo result

* support quniform type; update doc; update test config

* update doc

* un-commit changed in yarn.lock

* fix optimize mode bug

* optimize mode

* optimize mode

* reset pylint, gitignore

* revert .gitignore yarn.lock

* update

* dev-tf-master

* fix bugs

* fix bugs

* remove unnecessary lines

* dev oneshot and darts

* dev nas

* dev enas and oneshot

* dev enas and oneshot

* dev enas and oneshot

* dev oneshot and enas

* dev oneshot

* add ut

* add docstring

* add docstring

* fix

* resolve comments by changing docstring

* resolve comments