GitLab

[ci] [python-package] fix missing import, test that lightgbm can be imported with only required dependencies (fixes #5631) (#5632)

* [python-package] add type hints on cv()

* remove inadvertent changes

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

[python-package] Use scikit-learn interpretation of negative `n_jobs` and change default to number of cores (#5105)

* use joblib formula for negative n_jobs

* correction for n_jobs calculation

* use more robust cpu_count from joblib

* change default n_jobs to number of cores

* fix detection of num_threads under parameters

* better handling of n_jobs at prediction time

* fix incorrect usage of list.pop

* correct pop/remove yet again

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* add comments clarifying negative n_jobs

* fix CI (code taken from PR comment)

* change default to n_jobs=None in dask interface

* corrections for handling of n_jobs

* linter

* corrections for predict-time n_jobs

* linter

* add more comments about n_jobs values

* linter

* more corrections

* linter

* linter

* linter

* Update python-package/lightgbm/compat.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* workaround for passing test about outputs with multiple threads

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* map nullable dtypes to regular float dtypes

* cast x3 to float after introducing missing values

* add test for regular dtypes

* use .astype and then values. update nullable_dtypes test and include test for regular numpy dtypes

* more specific allowed dtypes. test no copy when single float dtype df

* use np.find_common_type. set np.float128 to None when it isn't supported

* set default as type(None)

* move tests that use lgb.train to test_engine

* include np.float32 when finding common dtype

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* add linebreak
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Faster categorical column names selection (#1)

* Faster categorical column names selection

Change slow and redundant dataframe query by select_dtypes into a dataframe.dtypes list comprehension

* Update compat with CategoricalDtype

* sort imports

* import CategoricalDtype from pandas.api.types

* add categorical import try/except

* Make dummy classes constructible with any arguments

* Update compat.py

[dask] Make output of feature contribution predictions for sparse matrices match those from sklearn estimators (fixes #3881) (#4378)

* test_classifier working

* adding tests

* docs

* tests

* revert unnecessary changes in tests

* test output type

* linting

* linting

* use from_delayed() instead

* docstring pycodestyle is happy with

* isort

* put pytest skips back

* respect sparse return type

* fix doc

* remove unnecessary dask_array_concatenate()

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* update predict_proba() docstring

* remove unnecessary np.array()

* Update python-package/lightgbm/dask.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix assertion

* fix test use of len()

* restore np.array() in tests

* use np.asarray() instead

* use toarray()

* remove empty functions in compat
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* chanche MRO

* fix MRO resolution

* Some mypy fixes

* address James' comments

* Re-introduce pass in empty classes

* Update compat.py

Remove extra lines

* [dask] allow tight control over ports

* getting there, getting there

* fix params maybe

* fixing params

* remove unnecessary stuff

* fix tests

* fixes

* some minor changes

* fix flaky test

* linting

* more linting

* clarify parameter description

* add warning

* revert docs change

* Update python-package/lightgbm/dask.py

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* trying to fix stuff

* this is working

* update tests

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* indent
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* run isort in CI linting job

* workaround conda compatibility issues

* add type hints in dask module

* starting on asserts

* remove unused code

* add hints for dtypes

* replace accidentally-removed docstrings

* revert unrelated change

* Update python-package/lightgbm/dask.py

* empty commit

* fix hints on group

* capitalize array

* hide hints in signatures

* empty commit

* sphinx version

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix hint for MatrixLike

* Update python-package/lightgbm/dask.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* update docstring

* empty commit
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix Dask docstrings and mimic sklearn importing way

* Update .vsts-ci.yml

* revert CI checks

* use import aliases for Dask classes

* check Dask is installed in _predict() func

* fix lint issues introduced during resolving merge conflicts

* Update dask.py

* initial Dask docs

* fix MRO

* address review comments

* Update setup.py

* Update .appveyor.yml

* Update .travis.yml

* Update .vsts-ci.yml

* Update __init__.py

* Update test.sh

* Update test_windows.ps1

* Update advanced_example.py

* Update requirements_base.txt

* Update conf.py

* Update conf.py

* Update test_engine.py

* Update utils.py

* Update dockerfile-r

* Update README.md

* Update dockerfile.gpu

* Update test_consistency.py

* Update basic.py

* Update compat.py

* Update engine.py

* Update sklearn.py

* Update sklearn.py

* Update callback.py

* Update setup.py

* Update __init__.py

* Update plotting.py

* Update sklearn.py

* Update engine.py

* Update compat.py

* Update callback.py

* Update basic.py

* Update compat.py

* Update basic.py

* Update basic.py

* Update compat.py

* Update compat.py

* Update plotting.py

* Update engine.py

* Update basic.py

* Update sklearn.py

* Update compat.py

* Update engine.py

* Update engine.py

* Update callback.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update sklearn.py

* Update sklearn.py

* Update plotting.py

* Update sklearn.py

* Update compat.py

* Update compat.py

* Update engine.py

* Update plotting.py

* Update sklearn.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update compat.py

* Update compat.py

* Update compat.py

* Update engine.py

* Update basic.py

* Update compat.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update compat.py

* Update compat.py

* Update basic.py

* Update basic.py

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* Update conf.py

* Revert "Update dockerfile-r"

This reverts commit 4ff6ffc7e3eeda24cc6a59a3bb0c973f02d9d71c.

* Revert "specify the last supported version of scikit-learn (#2637)"

This reverts commit d1002776.

* ban scikit-learn 0.22.0 and skip broken test

* fix updated test

* fix lint test

* Revert "fix lint test"

This reverts commit 8b4db0805fe7a9e7f7eb0be3eac231f85026d196.

* reworked pandas dtypes mapper

* added tests

* added sparsity support for new version of pandas

* fixed tests for old pandas

* check pd.Series for bad dtypes as well

* enhanced tests

* fixed pylint

* convert datatable to numpy directly

* fix according to comments

* updated more docstrings

* simplified isinstance check

* Update compat.py

* added docstring style test and fixed errors in existing docstrings

* hotfix

* hotfix

* fix grammar

* hotfix

* fixed FutureWarning about cv default value

* fixed according to new check_estimator API

* fixed joblib warning

* added possibility to predict numpy array with nrow > max_int32

* save memory more efficient

* better variable name

* added possibility to predict scipy.sparse.csr matrix with nrow > max_int32

* added possibility to predict scipy.sparse.csc matrix with nrow > max_int32

* reworked numpy array case

* removed excess import

* added tests for plotting trees in Python

* refined module_INSTALLED mechanism

* added note about that create_tree_digraph is better than plot_tree

* decode error description

* added break line char in log massages

* added class_weight parameter in sklearn wrapper

* added note about alternative parameters in case of binary classification task

* made max_bin parameter deprecated

* fixed accidental docstrings in Sphinx

* concrete version when deprecated stuff will be removed

* added warnings in case of duplicated params to Dataset

* fixed indents in docs

* fixed docs building at Python 2.x

* updated docs building guide

* get back Python 3 at Travis

* test commit for OSX

* be more precise with python version inpylint task

* turned warnings into errors

* removed linkchecker validation from osx

* removed warning about not included into any toctree

* docs config cleanup

* refined deprecation warnings mechanism

* refine docs mock mechanism

* be more precise with python version in check-docs task

* redused the number of code lines

* refined venv deployment

* rollback python version in check-docs task

* revert 'refine docs mock mechanism'. autodoc_mock_imports seems to be not working

* added targets for big images

* rollback to default python version in check-docs task

* break long lines for mobile view

* replaced pip with conda where it's possible

* improved sklearn interface; added sklearns' tests

* moved best_score into the if statement

* improved docstrings; simplified LGBMCheckConsistentLength

* fixed typo

* pylint

* updated example

* fixed Ranker interface

* added missed boosting_type

* fixed more comfortable autocomplete without unused objects

* removed check for None of eval_at

* fixed according to review

* fixed typo

* added description of fit return type

* dictionary->dict for short

* markdown cleanup

* refine plot

* use warnings

* refine  logic

* revert 'move to compat.py'