GitLab

* Add integrated OpenCL build on Linux

* Build integrated OpenCL Linux wheel in CI

* Fix test_dual.py on Linux arm64

* Enable integrated OpenCL Linux wheel arm64 testing in CI

* Update documentation

* Add comment about gpu_use_dp

* add missing fi dropped in merge conflict resolution

* install opencl-headers on bdist task

* use new CI image for x86_64

* update check_dynamic_dependencies script

* use main CI image
Co-authored-by: James Lamb <jaylamb20@gmail.com>

[CUDA] Add Poisson regression objective for cuda_exp and refactor objective functions for cuda_exp (#5486)

* add poisson regression objective for cuda_exp

* enable Poisson regression for cuda_exp

* refactor cuda objective functions

* remove useless changes

* fix linter errors

* remove redundant buffer in cuda poisson regression objective

* fix log of cuda_exp binary objective

* fix threshold of poisson objective result

* remove useless changes

* fix compilation errors

* add cuda quantile regression objective

* remove cuda quantile regression objective
Co-authored-by: James Lamb <jaylamb20@gmail.com>

[ci] prefer CPython in Windows test environment and use safer approach for cleaning up network (fixes #5509) (#5510)

* add feature interaction constraint for cuda_exp

* test feature interaction constraints for cuda_exp

* remove useless check

* update comment

* change percentile in CUDARegressionL1loss::BoostFromScore to 0.5

* add huber regression for cuda_exp

* renew tree output on GPU

add test cases for regression objectives

* remove useless changes

* add white space

* fix test_regression

* add binary objective for cuda_exp

* include <string> and <vector>

* exchange include ordering

* fix length of score to copy in evaluation

* fix EvalOneMetric

* fix cuda binary objective and prediction when boosting on gpu

* Add white space

* fix BoostFromScore for CUDABinaryLogloss

update log in test_register_logger

* include <algorithm>

* simplify shared memory buffer

* fix cuda_exp ci

* fix ci failures introduced by #5279

* cleanup cuda.yml

* fix test.sh

* clean up test.sh

* clean up test.sh

* skip lines by cuda_exp in test_register_logger

* Update tests/python_package_test/test_utilities.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* include parameters from reference dataset on copy

* lint

* set non-default parameters

* ignore training set on early stopping callback

* fixes

* lint

* Apply suggestions from code review
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* trigger ci
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Extract streaming to own PR

* small merge fixes and cleanup

* linting fixes

* fix cast warning

* Fix accidental deletion during branch transfer

* responded to initial triage comments

* Added more tests to use create-from-samples APIs

* added mutex and adjusted nclasses logic

* Fix thread-safety for pushing data to sparse bins through Push APIs

* lint and doc fixes

* Small SWIG fix

* nit fix

* Responded to StrikerRUS comments

* fix breaking change after merge with master

* Extract streaming to own PR

* small merge fixes and cleanup

* Fix accidental deletion during branch transfer

* responded to initial triage comments

* Added more tests to use create-from-samples APIs

* Fix rstcheck call in ci

* remove TODOs

* Extract streaming to own PR

* small merge fixes and cleanup

* Fix accidental deletion during branch transfer

* responded to initial triage comments

* Added more tests to use create-from-samples APIs

* Small SWIG fix

* remove ci change

* responded to shiyu1994 comments

* responded to StrikerRUS comments

* Fixes from StrikerRUS comments

[python-package] highlight the path a sample takes through a tree in `plot_tree` and `create_tree_digraph` (fixes #4784) (#5119)

* highlight path in plot_tree

* lint

* rename x to example_case. support categorical features. add test

* lint

* check for exactly one row. test empty example_case

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* handle missing values in numeric splits

* remove literal. add categorical split function

* make categorical feature more important. lint

* add enum. update categorical split. apply suggestions

* update numeric split decision

* lint

* Update python-package/lightgbm/plotting.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* dump sorted parameter aliases

* update lgb.check.wrapper_param

* update _choose_param_value to look like lgb.check.wrapper_param

* apply suggestions from review

* reduce diff

* move DumpAliases to config

* remove unnecessary check

* restore parameter check

add validate_features to refit

* allow custom weighing in sklearn api

* add suggestions from review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* check feature names and order in predict with dataframe

* slice df in predict to remove the target

* scramble features

* handle int column names

* only change column order when needed

* include validate_features param in booster and sklearn estimators

* document validate_features argument

* use all_close in preds checks and check for assertion error to compare different arrays

* perform remapping and checks in cpp

* remove extra logs

* fixes

* revert cpp

* proposal

* remove extra arg

* lint

* restore _data_from_pandas arguments

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* move data conversion to Predictor.predict

* use Vector2Ptr
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [python] preserve None in _choose_param_value()

* Update python-package/lightgbm/basic.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

[python-package] Use scikit-learn interpretation of negative `n_jobs` and change default to number of cores (#5105)

* use joblib formula for negative n_jobs

* correction for n_jobs calculation

* use more robust cpu_count from joblib

* change default n_jobs to number of cores

* fix detection of num_threads under parameters

* better handling of n_jobs at prediction time

* fix incorrect usage of list.pop

* correct pop/remove yet again

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* add comments clarifying negative n_jobs

* fix CI (code taken from PR comment)

* change default to n_jobs=None in dask interface

* corrections for handling of n_jobs

* linter

* corrections for predict-time n_jobs

* linter

* add more comments about n_jobs values

* linter

* more corrections

* linter

* linter

* linter

* Update python-package/lightgbm/compat.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* workaround for passing test about outputs with multiple threads

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

Clear split info buffer in cost efficient gradient boosting before every iteration (fix partially #3679) (#5164)

* clear split info buffer in cegb_ before every iteration

* check nullable of cegb_ in serial_tree_learner.cpp

* add a test case for checking the split buffer in CEGB

* swith to Threading::For instead of raw OpenMP

* apply review suggestions

* apply review comments

* remove device cpu

* Update test_basic.py

* Address review comment

* dont copy dataframe on rename

* test with feature_name and 'auto'

* Update basic.py

* Update test_engine.py

* Add return type annotation

[python-package] make a shallow copy when replacing categorical features with codes (fixes #4596) (#5225)

* allow using feature names when retrieving number of bins

* unname vector

* use default feature names when not defined

* lint

* apply suggestions

* remove extra comma

* add test with categorical feature

* make feature names sync more transparent

* Update dataset_loader.cpp

* Update gbdt.h

* Update regression_objective.hpp

* Update linker_topo.cpp

* Update xentropy_objective.hpp

* Update regression_objective.hpp

* investigate inf test failure

* avoid overflow in regression objective

* remove `test_inf_handle` test
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>

* check number of features when retrieving number of bins

* check for negative values

* lint