GitLab

* ignore training set on early stopping callback

* fixes

* lint

* Apply suggestions from code review
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* trigger ci
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* [python-package] add type hints on Booster eval methods

* remove unnecessary changes

* fix hints

* make `log_evaluation` callback pickleable

* make callback tests stricter

* make `record_evaluation` callback picklable

* make `log_evaluation` callback pickleable

* make callback tests stricter

* Turn `early_stopping` into a Callable class

* Fix

* Lint

* Remove print

* Fix order

* Revert "Lint"

This reverts commit 7ca8b557572446888cf793c0082d9a7efd1e29a7.

* Apply suggestion from code review

* Nit

* Lint

* Move callable class outside the func for pickling

* Move _pickle and _unpickle to tests utils

* Add early stopping callback picklability test

* Nit

* Fix

* Lint

* Improve type hint

* Lint

* Lint

* Add cloudpickle to test_windows

* Update tests/python_package_test/test_engine.py

* Fix

* Apply suggestions from code review

* make record_evaluation compatible with cv

* test multiple metrics in cv

* lint

* fix cv with train metric. save stdv as well

* always add dataset prefix to cv_agg

* remove unused function

* Update test_sklearn.py

* Update python_package.yml

* Update python_package.yml

* Update callback.py

* Update callback.py

* [python-package] fix mypy errors in engine.py

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* allow for stdv

* whitespace
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* Update __init__.py

* Update callback.py

* initial changes

* initial version

* better handling of cases

* warn only with positive threshold

* remove early_stopping_threshold from high-level functions

* remove remaining early_stopping_threshold

* update test to use callback

* better handling of cases

* rename threshold to min_delta

enhance parameter description

update tests

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* reduce num_boost_round in tests

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* trigger ci
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: Nikita Titov <nekit94-12@hotmail.com>

* Update __init__.py

* Update Python-API.rst

* Update engine.py

* Update test_utilities.py

* Update sklearn.py

* Update callback.py

* Update callback.py

* Update callback.py

* deprecate advanced args of `train()` and `cv()`

* update Dask test

* improve deducing

* address review comments

* Update callback.py

* Update engine.py

* Update test_sklearn.py

* Update test_basic.py

* Update dask.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update callback.py

* added f-string

* fixed indent issues

* fixed some linting issues

* trying to fix E225 for 233:65

* Update python-package/lightgbm/callback.py

* first_metric_only log message

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>
Co-authored-by: James Lamb <jaylamb20@gmail.com>
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* added type hints; implemented one workaround

* resolving some linting errors

* Added doc strings

* fixed more linting errors

* Made documentation more imperative.

* removed one type hint

* more specific type hinting
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* added import

* Apply suggestions from code review
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* made a class and function private

* Apply suggestions from code review

Make the documentation clearer.
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* linting error fix

* more linting errors

* removing the decorator

* ignore mypy function attribute errors

* fix lints
Co-authored-by: James Lamb <jaylamb20@gmail.com>

* Some mypy fixes

* address James' comments

* Re-introduce pass in empty classes

* Update compat.py

Remove extra lines

* centralize Python-package logging in one place

* continue

* fix test name

* removed unused import

* enhance test

* fix lint

* hotfix test

* workaround for GPU test

* remove custom logger from Dask-package

* replace one log func with flags by multiple funcs

* Update setup.py

* Update .appveyor.yml

* Update .travis.yml

* Update .vsts-ci.yml

* Update __init__.py

* Update test.sh

* Update test_windows.ps1

* Update advanced_example.py

* Update requirements_base.txt

* Update conf.py

* Update conf.py

* Update test_engine.py

* Update utils.py

* Update dockerfile-r

* Update README.md

* Update dockerfile.gpu

* Update test_consistency.py

* Update basic.py

* Update compat.py

* Update engine.py

* Update sklearn.py

* Update sklearn.py

* Update callback.py

* Update setup.py

* Update __init__.py

* Update plotting.py

* Update sklearn.py

* Update engine.py

* Update compat.py

* Update callback.py

* Update basic.py

* Update compat.py

* Update basic.py

* Update basic.py

* Update compat.py

* Update compat.py

* Update plotting.py

* Update engine.py

* Update basic.py

* Update sklearn.py

* Update compat.py

* Update engine.py

* Update engine.py

* Update callback.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update sklearn.py

* Update sklearn.py

* Update plotting.py

* Update sklearn.py

* Update compat.py

* Update compat.py

* Update engine.py

* Update plotting.py

* Update sklearn.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update compat.py

* Update compat.py

* Update compat.py

* Update engine.py

* Update basic.py

* Update compat.py

* Update basic.py

* Update basic.py

* Update basic.py

* Update compat.py

* Update compat.py

* Update basic.py

* Update basic.py

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* Update conf.py

* Revert "Update dockerfile-r"

This reverts commit 4ff6ffc7e3eeda24cc6a59a3bb0c973f02d9d71c.

* removed duplicated code from language wrappers

* removed check for resetting metric

* handle aliases centralized

* convert aliases dict to class

* Bug fix for first_metric_only if the first metric is train metric.

* Update bug fix for feval issue.

* Disable feval for first_metric_only.

* Additional test items.

* Fix wrong assertEqual settings & formating.

* Change dataset of test.

* Fix random seed for test.

* Modiry assumed test result due to different sklearn verion between CI and local.

* Remove f-string

* Applying variable  assumed test result for test.

* Fix flake8 error.

* Modifying  in accordance with review comments.

* Modifying for pylint.

* simplified tests

* Deleting error criteria `if eval_metric is None`.

* Delete test items of classification.

* Simplifying if condition.

* Applying first_metric_only for sklearn wrapper.

* Modifying test_sklearn for comforming to python 2.x

* Fix flake8 error.

* Additional fix for sklearn and add tests.

* Bug fix and add test cases.

* some refactor

* fixed lint

* fixed lint

* Fix duplicated metrics scores to pass the test.

* Fix the case first_metric_only not in params.

* Converting metrics aliases.

* Add comment.

* Modify comment for pylint.

* Modify comment for pydocstyle.

* Using split test set for two eval_set.

* added test case for metric aliases and length checks

* minor style fixes

* fixed rmse name and alias position

* Fix the case metric=[]

* Fix using env.model._train_data_name

* Fix wrong test condition.

* Move initial process to _init() func.

* Modify test setting for test_sklearn & training data matching on callback.py

* test_sklearn.py
-> A test case for training is wrong, so fixed.

* callback.py
-> A condition of if statement for detecting test dataset is wrong, so fixed.

* Support composite name metrics.

* Remove metric check process & reduce redundant test cases.

For #2273 fixed not only the order of metrics in cpp, removing metric check process at callback.py

* Revised according to the matters pointed out on a review.

* increased code readability

* Fix the issue of order of validation set.

* Changing to OrderdDict from default dict for score result.

* added missed check in cv function for first_metric_only and feval co-occurrence

* keep order only for metrics but not for datasets in best_score

* move OrderedDict initialization to init phase

* fixed minor printing issues

* move first metric detection to init phase and split can be performed without checks

* split only once during callback

* removed excess code

* fixed typo in variable name and squashed ifs

* use setdefault

* hotfix

* fixed failing test

* refined tests

* refined sklearn test

* Making "feval" effective on early stopping.

* allow feval and first_metric_only for cv

* removed unused code

* added tests for feval

* fixed printing

* add note about whitespaces in feval name

* Modifying final iteration process in case valid set is  training data.

* Use first_metric_only flag for early_stopping function.

In order to apply early stopping with only first metric, applying first_metric_only flag for early_stopping function.

* upcate comment

* Revert "upcate comment"

This reverts commit 1e75a1a415cc16cfbe795181e148ebfe91469be4.

* added test

* fixed docstring

* cut comment and save one line

* document new feature

* always save the score of the first round in early stopping

fix #1971

* avoid using std::log on non-positive numbers

* remove unnecessary changes

* add tests

* Update test_sklearn.py

* enhanced tests

* disabled early stopping in dart mode

* check dart mode during the init phase of early_stop callback

* added docstring style test and fixed errors in existing docstrings

* hotfix

* hotfix

* fix grammar

* hotfix

* break extremely large lines in basic.py

* break extremely large lines in callback.py

* break extremely large lines in engine.py

* break extremely large lines in sklearn.py

* hotfixes

* bring consistency and clearness into early_stopping_rounds desc, metric desc and implementation

* hotfix

* hotfix

* used NDCG as default metric for lambdarank task

* fixed missed methods at ReadTheDocs and changed default eval_metric

* leaved only unique metrics

* fixed comment

The document of `early_stopping_rounds` says it will check all of
eval_set. But, this is not true. It doesn't check the dataset
specified as the training data.

This change appends an extra phrase "except the training data" to all
of the sentences "If there's more than one, will check all of them" in
documents.