GitLab

[tests][python] move tests that use `train()` function defined in `engine.py` from `test_basic.py` to `test_engine.py` (#5034)

* Update test_basic.py

* Update test_engine.py

* Update test_engine.py

[python] fixes for supporting 2d numpy arrays for predictions, grads and hess in multiclass custom objective and eval (#5030)

* fixes for supporting 2d numpy arrays for predictions, grads and hess in multiclass custom objective

* Apply suggestions from code review
Co-authored-by: José Morales <jmoralz92@gmail.com>
Co-authored-by: José Morales <jmoralz92@gmail.com>

* Correct documentation for sparse predictions

The documentation says that the parameter `nindptr` for `LGBM_BoosterPredictSparseOutput` should be the number of rows plus one, but this is incorrect when the input type is CSC. This PR fixes it.

* Update c_api.h

* Update c_api.h

* Update c_api.h

* map nullable dtypes to regular float dtypes

* cast x3 to float after introducing missing values

* add test for regular dtypes

* use .astype and then values. update nullable_dtypes test and include test for regular numpy dtypes

* more specific allowed dtypes. test no copy when single float dtype df

* use np.find_common_type. set np.float128 to None when it isn't supported

* set default as type(None)

* move tests that use lgb.train to test_engine

* include np.float32 when finding common dtype

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* add linebreak
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update lightgbm.R

* Update lightgbm.Rd

* promote objective and init_score to top-level arguments

* follow comments

* Update R-package/R/lightgbm.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* update docs

* linter

* comments

* comments

* comments

* extend test for default objective

* Update R-package/tests/testthat/test_basic.R
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* docs: weight parameter non-negative

* docs: weights non negative only for train data

* docs: weights should be non negative for validation data

* typo in html render

* docs: brief weights non-negative description

[python-package] use 2d collections for predictions, grads and hess in multiclass custom objective (#4925)

* reshape predictions, grad and hess in multiclass custom objective

* add sklearn test. move custom obj to utils. docs for numpy

* use num_model_per_iteration to get num_classes

* update docs and dask multiclass custom objective test

* move reshaping to __inner_predict. add test for feval

* add missing note. remove extra line

* improve docs for sklearn wrapper

* empty commit

* install add scikit-learn to conda environment for building docs

* factor out lgb.check.obj()

* remove lgb.check.obj()

* add test on lgb.cv()

* clarify that categoricals will be converted to ints and not that they should be ints in the input data

* update remaining sections

* update config.h

* add suggestions

* CUDATreeLearner: free GPU memory in destruuctor if any allocated

* Minor changes: checking for num_gpu_feature_groups is not needed

* Trigger CI again

* [ci] [docs] use mamba for readthedocs builds (fixes #4954)

* update docs

* simplify build script and add docs flag to gitignore

* exit with non-0 if build fails

* update CI job

* add doxygen

* remove outdated requirement_base.txt reference

* use conda create instead of conda env create

* fix conda create flags

* add nodefaults to env.yml

* Update docs/README.rst
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* try to fix check-docs CI job

* additional changes

* switch from mamba to miniforge

* simplify docker command and fix issues in local build script

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* update docs and conda

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update .appveyor.yml

* Update .vsts-ci.yml

* [R-package] prefer params to keyword argument in lgb.train()

* make test stricter

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update setup.py

* Update cuda.yml

* Update static_analysis.yml

* Update linkchecker.yml

* Update .appveyor.yml

* Update .vsts-ci.yml

* Update python_package.yml

* Update .vsts-ci.yml

* Update python_package.yml

* Update cuda.yml

* Update static_analysis.yml

* Update linkchecker.yml

* Update .vsts-ci.yml

* Update .appveyor.yml

* re-arange Python version

* do not produce model files on disk by default

* remove 'save_name' argument from 'lightgbm()'

* make record_evaluation compatible with cv

* test multiple metrics in cv

* lint

* fix cv with train metric. save stdv as well

* always add dataset prefix to cv_agg

* remove unused function

* update deps

* pin eigen

* add hidden bazel files to `.Rbuildignore`

* more verbose test logs

* more verbosity in logs for other tasks

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* revert fast_double_parser update

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* empty commit

* revert tests verbosity

* test: swap compilers

* empty commit 1

* empty commit 2

* empty commit 3

* empty commit 4

* Revert "test: swap compilers"

This reverts commit e77b0013e9725cc82389f76f004e0367ed9295d8.

* Update test_dask.py

* Update test_engine.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update test_engine.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update test_sklearn.py

* Update dockerfile-python

* Update README.md

* Update dockerfile.gpu

* Update dockerfile.gpu

* Update .vsts-ci.yml

* Update .appveyor.yml

* Update test_windows.ps1

[docs] document `conda-forge` channel preference over `default` one and describe possible workaround for OpenMP conflicts in FAQ (#4994)

* Update README.rst

* Update FAQ.rst

* Python FAQ entry about conda-forge

* fix syntax

* [ci] use conda-forge in CI jobs (fixes #4948)

* comment out more jobs

* try reverting graphviz patch, running more cuda jobs

* get graphviz from PyPI and try removing some patches for r-lintr

* start running appveyor again

* use conda-forge if using conda

* fix commands

* conda install graphviz

* try newer openmp

* pin below openmp 11.x

* focus on gpu task

* trying to narrow down error

* maybe gcc11 is the issue

* start adding other tests back

* pin openmp too

* maybe need to pin to gcc less than 10.x

* pin libgfortran and libstdcxx as well

* pin to gcc 9.3.0

* move constraints up to initial environment

* add all CI jobs back

* try installing python-graphviz separately

* try new lightgbm/vsts-agent image

* fix typo

* test if pinning gcc for linux gpu_source build is still necessary

* ok yes, pinning gcc is necessary

* test if Linux gpu_source works with Python 3.9.6

* no special exception for Linux gpu_source job

* pin to Python 3.9.6 in Linux gpu_source

* try explicitly asking for libstdcxx-ng for every linux build

* swap compilers

* switch compilers back

* revert accidental whitespace change

* comment out CI

* try Linux gpu_source with different Python versions

* Revert "try Linux gpu_source with different Python versions"

This reverts commit f6f63cbb9b4a9cf138f3580ae4223a8acdd0e94a.

* Revert "comment out CI"

This reverts commit ece191f01e3650c2f325e80ff86bfc8c485fb7bc.

* remove libxml2 install, change CONDA path

* avoid installing conda in rchk job

* empty commit 1

* empty commit 2

* empty commit 3

* empty commit 4

* add more verbose logging around installation of python-graphviz

* empty commit 1

* get mamba info

* get more conda info

* add another mamba info call

* allow for other macOS environments in GHA configuration

* Revert "allow for other macOS environments in GHA configuration"

This reverts commit a3c7a19926be94e3719f5ae9100fbe30e87b35da.

* get more logs from mamba

* get Build.ArtifactsStagingDirectory

* get more logs and try to force re-installing everything

* clean cache after every step

* remove --update-all and make logs less verbose

* remove more print statements and uncomment jobs

* test if conda-clean issue fixes segfaults for gpu_source

* pin python version for gpu_source

* empty commit 1

* use miniforge instead

* empty commit 1

* Apply suggestions from code review

* bring workarounds back

* remove duplicated graphviz system-wide installation (reverts #4095, #4097, #4238)

* empty commit 1

* empty commit 2

* empty commit 3

* empty commit 4

* empty commit 5

* empty commit 6

* empty commit 7

* empty commit 8

* empty commit 9

* empty commit 10

* empty commit 10

* empty commit 10

* empty commit 10

* empty commit 11

* one more try

* try to downgrade Python version for Linux GPU job

* swap compilers

* Revert "swap compilers"

This reverts commit f04dc27b17920a69cbcba1254a8e109ce9791154.
Co-authored-by: Nikita Titov <nekit94-12@hotmail.com>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* [ci] manually create symlinks to R entrypoints on macOS (fixes #4988)

* exclude non-R CI jobs

* upgrade to R 4.1.2

* get logs for R install

* pin R 4.1.x jobs to macOS-10.15

* update to R 4.1.2 on Windows

* allow for non-latest macOS builds in GHA configuration

* fix prefix check

* fix config check

* more direct check for mac version

* uncomment other CIs

* update R version in CI job names

* Revert "[ci] ignore certificates for kitware apt channel in CUDA jobs (fixes #4646) (#4648)"

This reverts commit 10e0edc4.

* update cuda at CI

* feat: refit additional kwargs for dataset and predict

* test: kwargs for refit method

* fix: __init__ got multiple values for argument

* fix: pycodestyle E302 error

* refactor: dataset_params to avoid breaking change

* refactor: expose all Dataset params in refit

* feat: dataset_params updates new_params

* fix: remove unnecessary params to test

* test: parameters input are the same

* docs: address StrikeRUS changes

* test: refit test changes in train dataset

* test: set init_score and decay_rate to zero

* ENH save computations of exp in objectives

* CLN missing declaration