GitLab

* Add configuration parameters for CEGB.

* Add skeleton CEGB tree learner

Like the original CEGB version, this inherits from SerialTreeLearner.
Currently, it changes nothing from the original.

* Track features used in CEGB tree learner.

* Pull CEGB tradeoff and coupled feature penalty from config.

* Implement finding best splits for CEGB

This is heavily based on the serial version, but just adds using the coupled penalties.

* Set proper defaults for cegb parameters.

* Ensure sanity checks don't switch off CEGB.

* Implement per-data-point feature penalties in CEGB.

* Implement split penalty and remove unused parameters.

* Merge changes from CEGB tree learner into serial tree learner

* Represent features_used_in_data by a bitset, to reduce the memory overhead of CEGB, and add sanity checks for the lengths of the penalty vectors.

* Fix bug where CEGB would incorrectly penalise a previously used feature

The tree learner did not update the gains of previously computed leaf splits when splitting a leaf elsewhere in the tree.
This caused it to prefer new features due to incorrectly penalising splitting on previously used features.

* Document CEGB parameters and add them to the appropriate section.

* Remove leftover reference to cegb tree learner.

* Remove outdated diff.

* Fix warnings

* Fix minor issues identified by @StrikerRUS.

* Add docs section on CEGB, including citation.

* Fix link.

* Fix CI failure.

* Add some unit tests

* Fix pylint issues.

* Fix remaining pylint issue

* Use first_metric_only flag for early_stopping function.

In order to apply early stopping with only first metric, applying first_metric_only flag for early_stopping function.

* upcate comment

* Revert "upcate comment"

This reverts commit 1e75a1a415cc16cfbe795181e148ebfe91469be4.

* added test

* fixed docstring

* cut comment and save one line

* document new feature

* disabled split value histogram for categorical features

* updated test for cat. feature

* updated docs

* added get_split_value_histogram method

* added param for ordinary return value

* fixed number of tests in pytest

* fixed data shape and removed unused code

* refactored tests

* hotfix

* hotfix

* Initial attempt to implement appending features in-memory to another data set

The intent is for this to enable munging files together easily, without needing to round-trip via numpy or write multiple copies to disk.
In turn, that enables working more efficiently with data sets that were written separately.

* Implement Dataset.dump_text, and fix small bug in appending of group bin boundaries.

Dumping to text enables us to compare results, without having to worry about issues like features being reordered.

* Add basic tests for validation logic for add_features_from.

* Remove various internal mapping items from dataset text dumps

These are too sensitive to the exact feature order chosen, which is not visible to the user.
Including them in tests appears unnecessary, as the data dumping code should provide enough coverage.

* Add test that add_features_from results in identical data sets according to dump_text.

* Add test that booster behaviour after using add_features_from matches that of training on the full data

This checks:
- That training after add_features_from works at all
- That add_features_from does not cause training to misbehave

* Expose feature_penalty and monotone_types/constraints via get_field

These getters allow us to check that add_features_from does the right thing with these vectors.

* Add tests that add_features correctly handles feature_penalty and monotone_constraints.

* Ensure add_features_from properly frees the added dataset and add unit test for this

Since add_features_from moves the feature group pointers from the added dataset to the dataset being added to, the added dataset is invalid after the call.
We must ensure we do not try and access this handle.

* Remove some obsolete TODOs

* Tidy up DumpTextFile by using a single iterator for each feature

This iterators were also passed around as raw pointers without being freed, which is now fixed.

* Factor out offsetting logic in AddFeaturesFrom

* Remove obsolete TODO

* Remove another TODO

This one is debatable, test code can be a bit messy and duplicate-heavy, factoring it out tends to end badly.
Leaving this for now, will revisit if adding more tests later on becomes a mess.

* Add documentation for newly-added methods.

* Fix whitespace issues identified by pylint.

* Fix a few more whitespace issues.

* Fix doc comments

* Implement deep copying for feature groups.

* Replace awkward std::move usage by emplace_back, and reduce vector size to num_features rather than num_total_features.

* Copy feature groups in addFeaturesFrom, rather than moving them.

* Fix bugs in FeatureGroup copy constructor and ensure source dataset remains usable

* Add reserve to PushVector and PushOffset

* Move definition of Clone into class body

* Fix PR review issues

* Fix for loop increment style.

* Fix test failure

* Some more docstring fixes.

* Remove blank line

* always save the score of the first round in early stopping

fix #1971

* avoid using std::log on non-positive numbers

* remove unnecessary changes

* add tests

* Update test_sklearn.py

* enhanced tests

* added test for huge string model

* fixed tree sizes field

* simplified model structure

* fixed test and added try/except

[tests][python] added tests for metrics' behavior and fixed case for multiclass task with custom objective (#1954)

* added metrics test for standard interface

* simplified code

* less trees

* less trees

* use dummy custom objective and metric

* added tests for multiclass metrics aliases

* fixed bug in case of custom obj and num_class > 1

* added metric test for sklearn wrapper

* added get_data method

* hotfix

* added warning for other data types

* reworked according to review comments

* minor addition to FAQ

* added test

* [python] fix creating train_set in fit

https://github.com/Microsoft/LightGBM/blob/cc99f0d36ae929eb02b22a072823ab7c6d3155ab/python-package/lightgbm/sklearn.py#L519
may False even if valid_data[0] is X and valid_data[1] is y actually, because `check_X_y` might return copy of X and y.
https://scikit-learn.org/0.20/modules/generated/sklearn.utils.check_X_y.html

cf. https://github.com/Microsoft/LightGBM/pull/451

* use assertIn

* fix RF's bugs

* fix tests

* rollback num_iterations

* fix a bug and reduce memory costs

* reduce memory cost

* add pandas_categorical to returned json

* save pandas_categorical to model file

* added regression test

* removed excess conversion to list

* removed deprecated line

* hotfix

* added docstring style test and fixed errors in existing docstrings

* hotfix

* hotfix

* fix grammar

* hotfix

* break huge lines in sklearn tests

* break huge line in plotting tests

* break huge lines in basic tests

* multiple enhancements in engine tests

* multiple enhancements in sklearn tests

* hotfixes

* break huge lines and use with statement in C API test

* make NDCG test more strict

* fix ndcg consistency.

* more stable sorts

* Update gbdt_model_text.cpp

* Update dataset.cpp

* Update gbdt_model_text.cpp

* average predictions for constant features

* fix possible numerical issues in std::log.

* fix pylint

* fix bugs in c_api

* fix styles

* clean code for multi class

* rewrite test

* fix pylint

* skip test_constant_features

* refine test

* fix tests

* fix tests

* update FAQ

* fix test

* Update FAQ.rst

* fixed FutureWarning about cv default value

* fixed according to new check_estimator API

* fixed joblib warning

* added sklearn splitter classes in cv function

* added tests

* fix RF's bug withMAPE

* simplify rf's code & support multi-class rf

* fix bug & add test

* add more tests

* Update test_engine.py

* Update test_engine.py

* renamed functions

* updated find_lib function

* added test for pandas label of Dataset

* fix when label type is pandas DataFrame; document possible pandas Series type

* added NumberOfTotalModel and NumModelPerIteration to C_API and python-package

* fixed tests

* added tests for current_iteration, num_trees, num_model_per_iteration methods

* break huge line in test

* hotfix

* add support of refit-decay

* add refit into c_api

* add test

* update document

* Update basic.py

* Update test_engine.py

* Update basic.py

* Update test_engine.py

* fix comments

* update test

* fix the comments

* Update test_engine.py

* add start_iteration in model saving

* fix test

* shuffle models ability

* fix bug

* update document

* refine

* Update engine.py

* Update basic.py

* fix comments

* fix comment

* added new aliases for params

* run helper/parameter_generator.py

* removed useless test

* ignore vim temporary files

* add importance_type arg to sklearn API

* update documentation info

* remote a trailing space

* remove trailing space (again :))

* add instructions on importance choices to sklearn API

* drop mention of constructor in the feature type setting

* adding a test for different feture types

* remove trailing spaces, make shorter assert in feature importance type handling test

* fixing style issue introduced with the new test

* removed excess import

* added tests for plotting trees in Python

* refined module_INSTALLED mechanism

* added note about that create_tree_digraph is better than plot_tree

* fixed grammar

* fixed params description in graph plotting functions

* clarified types of attributes in their descriptions

* increased readability of graphs by adding spaces

* added precision parameter to plot tree functions

* added missed description of plot_example in python_guide folder and fixed consistency for packages naming

* more reliable OS detection

* fixed grammar

* made pylint happy

* fixed docs

* reworker predict method of sklearn wrapper

* fixed encapsulation

* added test

* fixed consistency between docstring and params docs

* fixed verbose

* replaced predict_proba with predict in test

* fixed verbose again

* fixed fraction params descriptions

* added description of skip_drop and drop_rate constraints

* fixed subsample_freq consistency with C++ default value

* fixed nice look of params list

* made force splits json file example clickable

* fixed nice look of metrics list and added comma

* reduced warning in test about same param specified twice

* replaced pred_parameter with **kwargs in predict method

* added test for **kwargs in predict method

* fixed warnings

* fixed pylint

* updated pep8 to pycodestyle

* fixed E722 do not use bare 'except'

* fixed W605 invalid escape sequence '\*'

* fixed W504 line break after binary operator

* ignore W605 invalid escape sequence '\*' in nuget builder

* made pycodestyle happy

* Read and write datsets from hdfs.
* Only enabled when cmake is run with -DUSE_HDFS:BOOL=TRUE
* Introduces VirtualFile(Reader|Writer) to asbtract VFS differences