GitLab

* add parameter data_sample_strategy

* abstract GOSS as a sample strategy(GOSS1), togetherwith origial GOSS (Normal Bagging has not been abstracted, so do NOT use it now)

* abstract Bagging as a subclass (BAGGING), but original Bagging members in GBDT are still kept

* fix some variables

* remove GOSS(as boost) and Bagging logic in GBDT

* rename GOSS1 to GOSS(as sample strategy)

* add warning about use GOSS as boosting_type

* a little ; bug

* remove CHECK when "gradients != nullptr"

* rename DataSampleStrategy to avoid confusion

* remove and add some ccomments, followingconvention

* fix bug about GBDT::ResetConfig (ObjectiveFunction inconsistencty bet…

* add std::ignore to avoid compiler warnings (anpotential fails)

* update Makevars and vcxproj

* handle constant hessian

move resize of gradient vectors out of sample strategy

* mark override for IsHessianChange

* fix lint errors

* rerun parameter_generator.py

* update config_auto.cpp

* delete redundant blank line

* update num_data_ when train_data_ is updated

set gradients and hessians when GOSS

* check bagging_freq is not zero

* reset config_ value

merge ResetBaggingConfig and ResetGOSS

* remove useless check

* add ttests in test_engine.py

* remove whitespace in blank line

* remove arguments verbose_eval and evals_result

* Update tests/python_package_test/test_engine.py

reduce num_boost_round
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update tests/python_package_test/test_engine.py

reduce num_boost_round
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update tests/python_package_test/test_engine.py

reduce num_boost_round
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update tests/python_package_test/test_engine.py

reduce num_boost_round
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update tests/python_package_test/test_engine.py

reduce num_boost_round
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update tests/python_package_test/test_engine.py

reduce num_boost_round
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update src/boosting/sample_strategy.cpp

modify warning about setting goss as `boosting_type`
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update tests/python_package_test/test_engine.py

replace load_boston() with make_regression()

remove value checks of mean_squared_error in test_sample_strategy_with_boosting()

* Update tests/python_package_test/test_engine.py

add value checks of mean_squared_error in test_sample_strategy_with_boosting()

* Modify warnning about using goss as boosting type

* Update tests/python_package_test/test_engine.py

add random_state=42 for make_regression()

reduce the threshold of mean_square_error

* Update src/boosting/sample_strategy.cpp
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* remove goss from boosting types in documentation

* Update src/boosting/bagging.hpp
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update src/boosting/bagging.hpp
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update src/boosting/goss.hpp
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update src/boosting/goss.hpp
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* rename GOSS with GOSSStrategy

* update doc

* address comments

* fix table in doc

* Update include/LightGBM/config.h
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* update documentation

* update test case

* revert useless change in test_engine.py

* add tests for evaluation results in test_sample_strategy_with_boosting

* include <string>

* change to assert_allclose in test_goss_boosting_and_strategy_equivalent

* more tolerance in result checking, due to minor difference in results of gpu versions

* change == to np.testing.assert_allclose

* fix test case

* set gpu_use_dp to true

* change --report to --report-level for rstcheck

* use gpu_use_dp=true in test_goss_boosting_and_strategy_equivalent

* revert unexpected changes of non-ascii characters

* revert unexpected changes of non-ascii characters

* remove useless changes

* allocate gradients_pointer_ and hessians_pointer when necessary

* add spaces

* remove redundant virtual

* include <LightGBM/utils/log.h> for USE_CUDA

* check for  in test_goss_boosting_and_strategy_equivalent

* check for identity in test_sample_strategy_with_boosting

* remove cuda  option in test_sample_strategy_with_boosting

* Update tests/python_package_test/test_engine.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_engine.py
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* ResetGradientBuffers after ResetSampleConfig

* ResetGradientBuffers after ResetSampleConfig

* ResetGradientBuffers after bagging

* remove useless code

* check objective_function_ instead of gradients

* enable rf with goss

simplify params in test cases

* remove useless changes

* allow rf with feature subsampling alone

* change position of ResetGradientBuffers

* check for dask

* add parameter types for data_sample_strategy
Co-authored-by: Guangda Liu <v-guangdaliu@microsoft.com>
Co-authored-by: Yu Shi <shiyu_k1994@qq.com>
Co-authored-by: GuangdaLiu <90019144+GuangdaLiu@users.noreply.github.com>
Co-authored-by: James Lamb <jaylamb20@gmail.com>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Add integrated OpenCL build on Linux

* Build integrated OpenCL Linux wheel in CI

* Fix test_dual.py on Linux arm64

* Enable integrated OpenCL Linux wheel arm64 testing in CI

* Update documentation

* Add comment about gpu_use_dp

* add missing fi dropped in merge conflict resolution

* install opencl-headers on bdist task

* use new CI image for x86_64

* update check_dynamic_dependencies script

* use main CI image
Co-authored-by: James Lamb <jaylamb20@gmail.com>

[CUDA] Add Poisson regression objective for cuda_exp and refactor objective functions for cuda_exp (#5486)

* add poisson regression objective for cuda_exp

* enable Poisson regression for cuda_exp

* refactor cuda objective functions

* remove useless changes

* fix linter errors

* remove redundant buffer in cuda poisson regression objective

* fix log of cuda_exp binary objective

* fix threshold of poisson objective result

* remove useless changes

* fix compilation errors

* add cuda quantile regression objective

* remove cuda quantile regression objective
Co-authored-by: James Lamb <jaylamb20@gmail.com>

[ci] prefer CPython in Windows test environment and use safer approach for cleaning up network (fixes #5509) (#5510)

* add feature interaction constraint for cuda_exp

* test feature interaction constraints for cuda_exp

* remove useless check

* update comment

* change percentile in CUDARegressionL1loss::BoostFromScore to 0.5

* add huber regression for cuda_exp

* renew tree output on GPU

add test cases for regression objectives

* remove useless changes

* add white space

* fix test_regression

* add binary objective for cuda_exp

* include <string> and <vector>

* exchange include ordering

* fix length of score to copy in evaluation

* fix EvalOneMetric

* fix cuda binary objective and prediction when boosting on gpu

* Add white space

* fix BoostFromScore for CUDABinaryLogloss

update log in test_register_logger

* include <algorithm>

* simplify shared memory buffer

* fix cuda_exp ci

* fix ci failures introduced by #5279

* cleanup cuda.yml

* fix test.sh

* clean up test.sh

* clean up test.sh

* skip lines by cuda_exp in test_register_logger

* Update tests/python_package_test/test_utilities.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* include parameters from reference dataset on copy

* lint

* set non-default parameters

* ignore training set on early stopping callback

* fixes

* lint

* Apply suggestions from code review
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* trigger ci
Co-authored-by: James Lamb <jaylamb20@gmail.com>

[python-package] highlight the path a sample takes through a tree in `plot_tree` and `create_tree_digraph` (fixes #4784) (#5119)

* highlight path in plot_tree

* lint

* rename x to example_case. support categorical features. add test

* lint

* check for exactly one row. test empty example_case

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* handle missing values in numeric splits

* remove literal. add categorical split function

* make categorical feature more important. lint

* add enum. update categorical split. apply suggestions

* update numeric split decision

* lint

* Update python-package/lightgbm/plotting.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* dump sorted parameter aliases

* update lgb.check.wrapper_param

* update _choose_param_value to look like lgb.check.wrapper_param

* apply suggestions from review

* reduce diff

* move DumpAliases to config

* remove unnecessary check

* restore parameter check

add validate_features to refit

* allow custom weighing in sklearn api

* add suggestions from review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* check feature names and order in predict with dataframe

* slice df in predict to remove the target

* scramble features

* handle int column names

* only change column order when needed

* include validate_features param in booster and sklearn estimators

* document validate_features argument

* use all_close in preds checks and check for assertion error to compare different arrays

* perform remapping and checks in cpp

* remove extra logs

* fixes

* revert cpp

* proposal

* remove extra arg

* lint

* restore _data_from_pandas arguments

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* move data conversion to Predictor.predict

* use Vector2Ptr
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [python] preserve None in _choose_param_value()

* Update python-package/lightgbm/basic.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

[python-package] Use scikit-learn interpretation of negative `n_jobs` and change default to number of cores (#5105)

* use joblib formula for negative n_jobs

* correction for n_jobs calculation

* use more robust cpu_count from joblib

* change default n_jobs to number of cores

* fix detection of num_threads under parameters

* better handling of n_jobs at prediction time

* fix incorrect usage of list.pop

* correct pop/remove yet again

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* add comments clarifying negative n_jobs

* fix CI (code taken from PR comment)

* change default to n_jobs=None in dask interface

* corrections for handling of n_jobs

* linter

* corrections for predict-time n_jobs

* linter

* add more comments about n_jobs values

* linter

* more corrections

* linter

* linter

* linter

* Update python-package/lightgbm/compat.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update python-package/lightgbm/sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* workaround for passing test about outputs with multiple threads

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update tests/python_package_test/test_sklearn.py
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

Clear split info buffer in cost efficient gradient boosting before every iteration (fix partially #3679) (#5164)

* clear split info buffer in cegb_ before every iteration

* check nullable of cegb_ in serial_tree_learner.cpp

* add a test case for checking the split buffer in CEGB

* swith to Threading::For instead of raw OpenMP

* apply review suggestions

* apply review comments

* remove device cpu

* Update test_basic.py

* Address review comment

* dont copy dataframe on rename

* test with feature_name and 'auto'

* Update basic.py

* Update test_engine.py

* Add return type annotation

[python-package] make a shallow copy when replacing categorical features with codes (fixes #4596) (#5225)

* allow using feature names when retrieving number of bins

* unname vector

* use default feature names when not defined

* lint

* apply suggestions

* remove extra comma

* add test with categorical feature

* make feature names sync more transparent

* Update dataset_loader.cpp

* Update gbdt.h

* Update regression_objective.hpp

* Update linker_topo.cpp

* Update xentropy_objective.hpp

* Update regression_objective.hpp

* investigate inf test failure

* avoid overflow in regression objective

* remove `test_inf_handle` test
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>

* check number of features when retrieving number of bins

* check for negative values

* lint

[python-package] remove 'fobj' in favor of passing custom objective function in params (fixes #3244) (#5052)

* feat: support custom metrics in params

* feat: support objective in params

* test: custom objective and metric

* fix: imports are incorrectly sorted

* feat: convert eval metrics str and set to list

* feat: convert single callable eval_metric to list

* test: single callable objective in params
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* feat: callable fobj in basic cv function
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* test: cv support objective callable
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* fix: assert in cv_res
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* docs: objective callable in params
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* recover test_boost_from_average_with_single_leaf_trees
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* linters fail
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* remove metrics helper functions
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* feat: choose objective through _choose_param_values
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* test: test objective through _choose_param_values
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* test: test objective is callabe in train
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* test: parametrize choose_param_value with objective aliases
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* test: cv booster metric is none
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* fix: if string and callable choose callable
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* test train uses custom objective metrics
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* test: cv uses custom objective metrics
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* refactor: remove fobj parameter in train and cv
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* refactor: objective through params in sklearn API
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* custom objective function in advanced_example
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* fix whitespackes lint

* objective is none not a particular case for predict method
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* replace scipy.expit with custom implementation
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* test: set num_boost_round value to 20
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* fix: custom objective default_value is none
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* refactor: remove self._fobj
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* custom_objective default value is None
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* refactor: variables name reference dummy_obj
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* linter errors

* fix: process objective parameter when calling predict
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* linter errors

* fix: objective is None during predict call
Signed-off-by: Miguel Trejo <armando.trejo.marrufo@gmail.com>

* make `log_evaluation` callback pickleable

* make callback tests stricter

* make `record_evaluation` callback picklable