GitLab

* fix tests

* fix tests

* fix test comments

* simplify tests

* Apply suggestions from code review

[dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators  (#3911)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* [dask] Add unit tests that signatures are the same between Dask and scikit-learn estimators (fixes microsoft#3907)

* add support for pandas categorical columns

* remove commented code

* quotes

* syntax error

* fix shape for ranker test

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_dask.py

* trying

* fix tests

* remove unnecessary debugging stuff

* skip accuracy checks on categorical

* use category columns as categorical features
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update test_dask.py

* Update dask.py

* Update .vsts-ci.yml

* Revert "Update .vsts-ci.yml"

This reverts commit 98422be5b5095f0585de333b5b5545356776ef88.

* starting on Dask client

* more docs stuff

* fix pickling

* just copy docstrings

* fit docs

* switch test order

* linting

* use client kwarg

* remove inner set_params()

* add type hints

* fix type hints

* remove commented code

* reorder

* fix tests, add client_ property

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix tests

* linting

* simplify
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [dask] reduce teardown erros in Dask tests

* azure

* show logs

* try again

* more

* submodules

* try a bunch of sdist tests

* empty commit

* empty commit

* empty commit

* use sh-ubuntu

* 10 sdist tasks

* stuff

* empty commit

* empty commit

* empty commit

* empty commit

* try bdist

* empty commit

* empty commit

* empty commit

* empty commit

* py37

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* python 3.8

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* cuda config

* comment out cuda again

* setting timeout

* put client close in the right place

* uncomment CI, make timeout 60

* Update test_utilities.py

* Update cuda.yml

* Update test_utilities.py

* Update cuda_tree_learner.cpp

* Update cuda.yml

* [dask] add tests on warnings, fix incorrect variable in log

* Update tests/python_package_test/test_dask.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update test_engine.py

* Update test_sklearn.py

* Update test_engine.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update test_engine.py

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* Update test_engine.py

* Update test_dual.py

* Update test_engine.py

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* [dask] allow parameter aliases for tree_learner and local_listen_port (fixes #3671)

* num_thread too

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* empty commit

* add _choose_param_value

* revert param order change

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/dask.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* just import deepcopy

* remove machines aliases

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* TST Migrates test_sklearn.py to pytest

* STY Fixes linting

* FIX Adds reason

* ENH Address comments

* Merge test_<est>_local_predict and test_<est> tests for Dask module ##3833

* Merge test_<est>_local_predict to test_<est> tests in dask module - refactor #3833

* Update test_classifier and rename variables resolves #3833

* rename variables resolves #3833

* manage precision error #3833
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* [dask] factor dask-ml out of tests (fixes #3796)

* Update tests/python_package_test/test_dask.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update test_dask.py

* Update test_dask.py

* centralize Python-package logging in one place

* continue

* fix test name

* removed unused import

* enhance test

* fix lint

* hotfix test

* workaround for GPU test

* remove custom logger from Dask-package

* replace one log func with flags by multiple funcs

* move make_ranking into utils

* do not cache

* TST Migrates test_plotting.py to pytest

* STY Fixes linting

* [dask] use keyword args for internal function calls

* add missing comma

* Update python-package/lightgbm/dask.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* revert whitespace changes

* test style
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update test_dual.py

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* Update test_dual.py

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* adding pred_contrib support

* add tests

* linting

* remove raw_score

* add pred kwargs

* faster tests

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* changes to tests

* Update tests/python_package_test/test_dask.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* ranker support wip

* fix ranker tests

* fix _make_ranking rnd gen bug, add sleep to help w stoch binding port failed exceptions

* add wait_for_workers to prevent Binding port exception

* another attempt to stabilize test_dask.py

* requested changes: docstrings, dask_ml, tuples for list_of_parts

* fix lint bug, add group param to test_ranker_local_predict

* decorator to skip tests with errors on fixture teardown

* remove gpu ranker tests, reduce make_ranking data complexity

* another attempt to
silence client, decorator does not silence fixture errors

* address requested changes on 1/20/20

* skip test_dask for all GPU tasks

* address changes requested on 1/21/21

* issubclass instead of __qualname__
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* parity in group docstr with sklearn
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* _make_ranking docstr cleanup
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* TST Migrates tset_engine.py to pytest

* ENH Apply suggestions

* ENH Uses temp path

* ENH Fixes typos

[dask] allow parameter aliases for local_listen_port, num_threads, tree_learner (fixes #3671) (#3789)

* [dask] allow parameter aliases for tree_learner and local_listen_port (fixes #3671)

* num_thread too

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* empty commit
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* speed up tests

* [dask] reduce test times

* starting work

* fixed port-binding issue on localhost

* minor cleanup

* updates

* getting closer

* definitely working for LocalCluster

* it works, it works

* docs

* add tests

* removing testing-only files

* linting

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* remove duplicated code

* remove unnecessary listen()
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* TST Migrates test_basic.py to use pytest

* STY Linting

* CI Force CI to run

* [R-package] allow access to params in Booster

* remove unnecessary whitespace

* fix test on resetting params

* remove pytest_cache

* Update R-package/tests/testthat/test_custom_objective.R

* small code and docs refactoring

* Update CMakeLists.txt

* Update .vsts-ci.yml

* Update test.sh

* continue

* continue

* revert stable sort for all-unique values

* Add Eigen library.

* Working for simple test.

* Apply changes to config params.

* Handle nan data.

* Update docs.

* Add test.

* Only load raw data if boosting=gbdt_linear

* Remove unneeded code.

* Minor updates.

* Update to work with sk-learn interface.

* Update to work with chunked datasets.

* Throw error if we try to create a Booster with an already-constructed dataset having incompatible parameters.

* Save raw data in binary dataset file.

* Update docs and fix parameter checking.

* Fix dataset loading.

* Add test for regularization.

* Fix bugs when saving and loading tree.

* Add test for load/save linear model.

* Remove unneeded code.

* Fix case where not enough leaf data for linear model.

* Simplify code.

* Speed up code.

* Speed up code.

* Simplify code.

* Speed up code.

* Fix bugs.

* Working version.

* Store feature data column-wise (not fully working yet).

* Fix bugs.

* Speed up.

* Speed up.

* Remove unneeded code.

* Small speedup.

* Speed up.

* Minor updates.

* Remove unneeded code.

* Fix bug.

* Fix bug.

* Speed up.

* Speed up.

* Simplify code.

* Remove unneeded code.

* Fix bug, add more tests.

* Fix bug and add test.

* Only store numerical features

* Fix bug and speed up using templates.

* Speed up prediction.

* Fix bug with regularisation

* Visual studio files.

* Working version

* Only check nans if necessary

* Store coeff matrix as an array.

* Align cache lines

* Align cache lines

* Preallocation coefficient calculation matrices

* Small speedups

* Small speedup

* Reverse cache alignment changes

* Change to dynamic schedule

* Update docs.

* Refactor so that linear tree learner is not a separate class.

* Add refit capability.

* Speed up

* Small speedups.

* Speed up add prediction to score.

* Fix bug

* Fix bug and speed up.

* Speed up dataload.

* Speed up dataload

* Use vectors instead of pointers

* Fix bug

* Add OMP exception handling.

* Change return type of LGBM_BoosterGetLinear to bool

* Change return type of LGBM_BoosterGetLinear back to int, only parameter type needed to change

* Remove unused internal_parent_ property of tree

* Remove unused parameter to CreateTreeLearner

* Remove reference to LinearTreeLearner

* Minor style issues

* Remove unneeded check

* Reverse temporary testing change

* Fix Visual Studio project files

* Restore LightGBM.vcxproj.filters

* Speed up

* Speed up

* Simplify code

* Update docs

* Simplify code

* Initialise storage space for max num threads

* Move Eigen to include directory and delete unused files

* Remove old files.

* Fix so it compiles with mingw

* Fix gpu tree learner

* Change AddPredictionToScore back to const

* Fix python lint error

* Fix C++ lint errors

* Change eigen to a submodule

* Update comment

* Add the eigen folder

* Try to fix build issues with eigen

* Remove eigen files

* Add eigen as submodule

* Fix include paths

* Exclude eigen files from Python linter

* Ignore eigen folders for pydocstyle

* Fix C++ linting errors

* Fix docs

* Fix docs

* Exclude eigen directories from doxygen

* Update manifest to include eigen

* Update build_r to include eigen files

* Fix compiler warnings

* Store raw feature data as float

* Use float for calculating linear coefficients

* Remove eigen directory from GLOB

* Don't compile linear model code when building R package

* Fix doxygen issue

* Fix lint issue

* Fix lint issue

* Remove uneeded code

* Restore delected lines

* Restore delected lines

* Change return type of has_raw to bool

* Update docs

* Rename some variables and functions for readability

* Make tree_learner parameter const in AddScore

* Fix style issues

* Pass vectors as const reference when setting tree properties

* Make temporary storage of serial_tree_learner mutable so we can make the object's methods const

* Remove get_raw_size, use num_numeric_features instead

* Fix typo

* Make contains_nan_ and any_nan_ properties immutable again

* Remove data_has_nan_ property of tree

* Remove temporary test code

* Make linear_tree a dataset param

* Fix lint error

* Make LinearTreeLearner a separate class

* Fix lint errors

* Fix lint error

* Add linear_tree_learner.o

* Simulate omp_get_max_threads if openmp is not available

* Update PushOneData to also store raw data.

* Cast size to int

* Fix bug in ReshapeRaw

* Speed up code with multithreading

* Use OMP_NUM_THREADS

* Speed up with multithreading

* Update to use ArrayToString

* Fix tests

* Fix test

* Fix bug introduced in merge

* Minor updates

* Update docs

* migrated implementation from dask/dask-lightgbm

* relaxed tests

* tests skipped in case that MPI is used

* fixed python 2.7 import + tests disabled on windows

* python < 3.6 is not supported in tests

* tests enabled only for linux

* tests disabled for mpi interface

* dask version pinned to >= 2.0

* added @jameslamb as code owner

* added missing pandas dependency

* code refactoring, removed code duplication - lightgbm.dask.LGBMClassifier.fit is the same as lightgbm.dask.LGBMRegressor.fit

* fixed refactoring

* code deduplication - fit method moved into mixin class

* fixed CODEOWNERS

* removed unnecessary import

* skip the module execution on python < 3.6 and on platform different than linux.

* removed skip for python < 3.6

* review comments

* removed noqa, renamed API classes, renamed local variables