GitLab

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* Update Installation-Guide.rst

* Update install.libs.R

* Update setup.py

* Update r_package.yml

* Update install.libs.R

[R-package] enable saving Booster with saveRDS() and loading it with readRDS() (fixes #4296) (#4685)

* idiomatic serialization

* linter

* linter, namespace

* comments, linter, fix failing test

* standardize error messages for null handles

* auto-restore handle in more functions

* linter

* missing declaration

* correct wrong signature

* fix docs

* Update R-package/R/lgb.train.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update R-package/R/lgb.drop_serialized.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update R-package/R/lgb.restore_handle.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update R-package/R/lgb.restore_handle.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update R-package/R/lgb.make_serializable.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* move 'restore_handle' from feature importance to dump method

* missing header

* move arguments order, update docs

* linter

* avoid leaving files in working directory

* add test for save_model=NULL

* missing comma

* Update R-package/R/lgb.restore_handle.R
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update R-package/src/lightgbm_R.cpp
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* change name of error function

* update comment

* restore old serialization functions but set as deprecated

* Update R-package/R/readRDS.lgb.Booster.R
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update R-package/R/saveRDS.lgb.Booster.R
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* update docs

* Update R-package/R/readRDS.lgb.Booster.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update R-package/R/saveRDS.lgb.Booster.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update R-package/tests/testthat/test_basic.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update R-package/R/readRDS.lgb.Booster.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* comments

* fix variable name

* restore serialization test for linear models

* Update R-package/R/lightgbm.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* update docs

* fix issues with null terminator
Co-authored-by: James Lamb <jaylamb20@gmail.com>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* add C API function that returns all param names with aliases

* add C API function that returns all param names with aliases

* add R code

* test R code

* remove debug CI

* fix R lint

* refactor

* run CI

* fix R

* fix

* revert CI checks

* revert changes in docs

* Try to make function `const`
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* add `const` in cpp file

* address review comments and sync with `master`
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* add print and summary S3 method

* correct wrong signature

* attempt at bypassing linter

* Update R-package/R/lgb.Booster.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update R-package/src/lightgbm_R.h
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Update include/LightGBM/c_api.h
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* add more tests

* linter

* don't pluralize single tree

* remove duplicated function

* update changed function name

* missing declaration

* Update lightgbm_R.h

* Update R-package/tests/testthat/test_lgb.Booster.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* accommodate custom objectives in print

* linter

* linter
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* [R-package] parallelize compilation in CMake-based builds

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* working on adding -j

* pass -j through to install.libs.R

* add docs on -j

* use -j4

* Update R-package/README.md
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [R-package] fix segfaults caused by missing Booster and Dataset handles (fixes #4208)

* fix test errors

* fixes for cpplint

* Update R-package/tests/testthat/test_dataset.R
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix tests

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* move asserts inside try-catch
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix R memory leaks

* attempt at solving linter complaints

* fix compilation on windows

* move R_API_BEGIN to correct place

* make sure exception objects reach out of scope

* better way to solve rchk complaints

* remove goto statement

* [R-package] fix protection stack imbalance and unprotected objects issues

* [R-package] fix minor linting issues

* [ci][R-package] change timeout-minutes in valgrind test

* [R-package] remove extra space
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* [R-package] remove counter for number of protected objects

* Update .github/workflows/r_valgrind.yml
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* [R-package] move more finalizer logic intoo C++ side

* add C finalizers

* use gc()

* put skip() back

* Update R-package/tests/testthat/test_lgb.Booster.R
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* Update lightgbm_R.cpp

* fix typos

* started converting handles

* more changes

* sort of working for Dataset

* yay all the tests are passing for Dataset handle changes

* working for other handle types

* remove debugging logging

* remove unnecessary spaces

* fix null logic

* more NULL

* updates

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* consolidate steps
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [R-package] move creation of character vectors in some methods to C++ side

* convert LGBM_BoosterGetEvalNames_R

* convert LGBM_BoosterDumpModel_R and LGBM_BoosterSaveModelToString_R

* remove debugging code

* update docs

* remove comment

* add handling for larger model strings

* handle large strings in feature and eval names

* got long feature names working

* more fixes

* linting

* resize

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* stricter test
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [R-package] update docs on C++ interface

* Update R-package/src/lightgbm_R.h
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update R-package/src/lightgbm_R.h
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* add missing start_iteration doc
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* converted LGBM_BoosterSaveModel_R

* switch all other non-params cases

* handle params

* real pointer for matrix

* remove R_REAL_PTR

* remove R_INT_PTR

* add test

* [R-package] replace R_AS_INT with R built-in

* update header

* more changes

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [R-package] move some functions over to SEXP objects

* clarify comment

* remove paste0()

* fix registration table

* fix cmake builds

* [R-package] prevent symbol lookup conflicts

* add unit tests

* only run test on Windows

* move to .Call() calls

* fix references

* testing registration for CMake builds

* revert NAMESPACE changes

* revert testing changes

* [R-package] raise errors from C++ side

* working but a lot of warnings

* more changes

* simplify

* cleanup

* linting

* fix valgrind issues

* revert unnecessary change

* [R-package] Add GPU install options (fixes #3765)

* whitespace

* linting

* Update build_r.R
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Add Eigen library.

* Working for simple test.

* Apply changes to config params.

* Handle nan data.

* Update docs.

* Add test.

* Only load raw data if boosting=gbdt_linear

* Remove unneeded code.

* Minor updates.

* Update to work with sk-learn interface.

* Update to work with chunked datasets.

* Throw error if we try to create a Booster with an already-constructed dataset having incompatible parameters.

* Save raw data in binary dataset file.

* Update docs and fix parameter checking.

* Fix dataset loading.

* Add test for regularization.

* Fix bugs when saving and loading tree.

* Add test for load/save linear model.

* Remove unneeded code.

* Fix case where not enough leaf data for linear model.

* Simplify code.

* Speed up code.

* Speed up code.

* Simplify code.

* Speed up code.

* Fix bugs.

* Working version.

* Store feature data column-wise (not fully working yet).

* Fix bugs.

* Speed up.

* Speed up.

* Remove unneeded code.

* Small speedup.

* Speed up.

* Minor updates.

* Remove unneeded code.

* Fix bug.

* Fix bug.

* Speed up.

* Speed up.

* Simplify code.

* Remove unneeded code.

* Fix bug, add more tests.

* Fix bug and add test.

* Only store numerical features

* Fix bug and speed up using templates.

* Speed up prediction.

* Fix bug with regularisation

* Visual studio files.

* Working version

* Only check nans if necessary

* Store coeff matrix as an array.

* Align cache lines

* Align cache lines

* Preallocation coefficient calculation matrices

* Small speedups

* Small speedup

* Reverse cache alignment changes

* Change to dynamic schedule

* Update docs.

* Refactor so that linear tree learner is not a separate class.

* Add refit capability.

* Speed up

* Small speedups.

* Speed up add prediction to score.

* Fix bug

* Fix bug and speed up.

* Speed up dataload.

* Speed up dataload

* Use vectors instead of pointers

* Fix bug

* Add OMP exception handling.

* Change return type of LGBM_BoosterGetLinear to bool

* Change return type of LGBM_BoosterGetLinear back to int, only parameter type needed to change

* Remove unused internal_parent_ property of tree

* Remove unused parameter to CreateTreeLearner

* Remove reference to LinearTreeLearner

* Minor style issues

* Remove unneeded check

* Reverse temporary testing change

* Fix Visual Studio project files

* Restore LightGBM.vcxproj.filters

* Speed up

* Speed up

* Simplify code

* Update docs

* Simplify code

* Initialise storage space for max num threads

* Move Eigen to include directory and delete unused files

* Remove old files.

* Fix so it compiles with mingw

* Fix gpu tree learner

* Change AddPredictionToScore back to const

* Fix python lint error

* Fix C++ lint errors

* Change eigen to a submodule

* Update comment

* Add the eigen folder

* Try to fix build issues with eigen

* Remove eigen files

* Add eigen as submodule

* Fix include paths

* Exclude eigen files from Python linter

* Ignore eigen folders for pydocstyle

* Fix C++ linting errors

* Fix docs

* Fix docs

* Exclude eigen directories from doxygen

* Update manifest to include eigen

* Update build_r to include eigen files

* Fix compiler warnings

* Store raw feature data as float

* Use float for calculating linear coefficients

* Remove eigen directory from GLOB

* Don't compile linear model code when building R package

* Fix doxygen issue

* Fix lint issue

* Fix lint issue

* Remove uneeded code

* Restore delected lines

* Restore delected lines

* Change return type of has_raw to bool

* Update docs

* Rename some variables and functions for readability

* Make tree_learner parameter const in AddScore

* Fix style issues

* Pass vectors as const reference when setting tree properties

* Make temporary storage of serial_tree_learner mutable so we can make the object's methods const

* Remove get_raw_size, use num_numeric_features instead

* Fix typo

* Make contains_nan_ and any_nan_ properties immutable again

* Remove data_has_nan_ property of tree

* Remove temporary test code

* Make linear_tree a dataset param

* Fix lint error

* Make LinearTreeLearner a separate class

* Fix lint errors

* Fix lint error

* Add linear_tree_learner.o

* Simulate omp_get_max_threads if openmp is not available

* Update PushOneData to also store raw data.

* Cast size to int

* Fix bug in ReshapeRaw

* Speed up code with multithreading

* Use OMP_NUM_THREADS

* Speed up with multithreading

* Update to use ArrayToString

* Fix tests

* Fix test

* Fix bug introduced in merge

* Minor updates

* Update docs

* [R-package] Remove CLI-only objects

* more guards

* more guards

* variable not string

* simplify fix

* revert build_r.R changes

* move define of global_timer
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* store without offset in multi_val_dense_bin

* fix offset bug

* add comment for offset

* add comment for bin type selection

* faster operations for offset

* keep most freq bin in histogram for multi val dense

* use original feature iterators

* consider 9 cases (3 x 3) for multi val bin construction

* fix dense bin setting

* fix bin data in multi val group

* fix offset of the first feature histogram

* use float hist buf

* avx in histogram construction

* use avx for hist construction without prefetch

* vectorize bin extraction

* use only 128 vec

* use avx2

* use vectorization for sparse row wise

* add bit size for multi val dense bin

* float with no vectorization

* change multithreading strategy to dynamic

* remove intrinsic header

* fix dense multi val col copy

* remove bit size

* use large enough block size when the bin number is large

* calc min block size by sparsity

* rescale gradients

* rollback gradients scaling

* single precision histogram buffer as an option

* add float hist buffer with thread buffer

* fix setting zero in hist data

* fix hist begin pointer in tree learners

* remove debug logs

* remove omp simd

* update Makevars of R-package

* fix feature group binary storing

* two row wise for double hist buffer

* add subfeature for two row wise

* remove useless code and fix two row wise

* refactor code

* grouping the dense feature groups can get sparse multi val bin

* clean format problems

* one thread for two blocks in sep row wise

* use ordered gradients for sep row wise

* fix grad ptr

* ordered grad with combined block for sep row wise

* fix block threading

* use the same min block size

* rollback share min block size

* remove logs

* Update src/io/dataset.cpp
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>

* fix parameter description

* remove sep_row_wise

* remove check codes

* add check for empty multi val bin

* fix lint error

* rollback changes in config.h

* Apply suggestions from code review
Co-authored-by: Ubuntu <shiyu@gbdt-04.ren3kv4wanvufliwrpy4k03lsf.xx.internal.cloudapp.net>
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>

* fix int64 write error

* attempt

* [WIP] [ci] [R-package] Add CI job that runs valgrind tests

* update all-successful

* install

* executable

* fix redirect stuff

* Apply suggestions from code review
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>

* more flags

* add mc to msvc proj

* fix memory leak in mc

* Update monotone_constraints.hpp

* Update r_package.yml

* remove R_INT64_PTR

* disable openmp

* Update gbdt_model_text.cpp

* Update gbdt_model_text.cpp

* Apply suggestions from code review

* try to free vector

* free more memories.

* Update src/boosting/gbdt_model_text.cpp

* fix using

* try the UNPROTECT(1);

* fix a const pointer

* fix Common

* reduce UNPROTECT

* remove UNPROTECT(1);

* fix null handle

* fix predictor

* use NULL after free

* fix a leaking in test

* try more fixes

* test the effect of tests

* throw exception in Fatal

* add test back

* Apply suggestions from code review

* commet some tests

* Apply suggestions from code review

* Apply suggestions from code review

* trying to comment out tests

* Update openmp_wrapper.h

* Apply suggestions from code review

* Update configure

* Update configure.ac

* trying to uncomment

* more comments

* more uncommenting

* more uncommenting

* fix comment

* more uncommenting

* uncomment fully-commented out stuff

* try uncommenting more dataset tests

* uncommenting more tests

* ok getting closer

* more uncommenting

* free dataset

* skipping a test, more uncommenting

* more skipping

* re-enable OpenMP

* allow on OpenMP thing

* move valgrind to comment-only job

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* changes from code review

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* linting

* issue comments too

* remove issue_comment
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [R-package] [docs] Reorganize installation instructions (fixes #3320)

* more changes

* remove ability to use precompiled lib_lightgbm

* remove language about installing from the CRAN section

* move installation stuff

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix anchor
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [R-package] update DESCRIPTION per CRAN comments

* newlines

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* more fixes

* update Rbuildignore

* more changes

* more changes per CRAN response

* add email

* run examples in CI

* add newest CRAN response

* add Solaris patch

* update patch

* another attempt at ifaddrs patch

* fix unnecessary comment

* update configure

* comments

* bump version

* tabs

* fix address alignment, required by cran (#3415)

* fix dataset binary file alignment

* many fixes

* fix warnings

* fix bug

* Update file_io.cpp

* Update file_io.cpp

* simplify code

* Apply suggestions from code review

* general

* remove unneeded alignment

* Update file_io.h

* int32 to byte8 alignment

* Apply suggestions from code review

* Apply suggestions from code review

* [R-package] add new copyright holder in DESCRIPTION (#3409)

* [R-package] add new copyright holder in DESCRIPTION

* fix role

* fixing conflicts

* [R-package] add new copyright holder in DESCRIPTION (#3409)

* [R-package] add new copyright holder in DESCRIPTION

* fix role

* trying to fix conflicts

* more fixes

* this will work

* update cran-comments

* simplify solaris, add more testing docs

* stuff

* remove rchck docs

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* remove extra use of cat()

* change solaris check

* update docs

* remove testing code

* fix warning about cleanup not having execute permissions

* fix cmake builds

* remove blank line
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>

* Update install.libs.R

* Update CMakeLists.txt

* [python] add start_iteration to python predict interface (#3058)

* Apply suggestions from code review

* Update lightgbm_R.h

* Apply suggestions from code review

* Apply suggestions from code review

* fix R interface

* update R documentation
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>

Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [R-package] make package installable with CRAN toolchain (fixes #2960)

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* remove GPU stuff

* use wildcard to find objects to build

* use -lomp

* build configure before moving files

* using wildcard for objects

* Update .github/workflows/main.yml
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* add explicit objects back

* reduce allowed R CMD check NOTEs and catch stderr from build-cran-package on Windows

* fixing things

* pin autoconf version

* show diff

* add automake back

* run less checks

* command was in the wrong place

* fix autoconf version

* change strategy for handling configure

* fix Rbuildignore

* fix NOTEs

* fix notes about unrecognized files

* fixing extra files

* remove USE_R35

* add OpenMP check for Mac CRAN build

* run all checks

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* suggestions from code review

* undo indenting

* remove 03 from Makevars.win.in

* update language about OpenMP in configure script

* checking if configure.ac check works

* add autoconf back

* remove testing code in configure.ac

* more fixes for CI on configure script

* print git diff

* add VERSION.txt when checking configure

* fix relative paths

* remove git diff
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* feature importance type in saved model file

* fix nullptr

* fixed formatting

* fix python/R

* Update src/c_api.cpp

* Apply suggestions from code review
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* fix c_api test

* fix swig

* minor docs improvements and added defines for importance types
Co-authored-by: StrikerRUS <nekit94-12@hotmail.com>
Co-authored-by: James Lamb <jaylamb20@gmail.com>