"src/vscode:/vscode.git/clone" did not exist on "7b8bb4f223af081fafb1e5e54fa9df1da428482d"
  • Frank Fineis's avatar
    [dask] add support for eval sets and custom eval functions (#4101) · b5502d19
    Frank Fineis authored
    
    
    * es WiP, need to add eval_sample_weight and eval_group
    
    * add weight, group to dask es. WiP.
    
    * dask es reorg
    
    * Update python-package/lightgbm/dask.py
    
    _train_part model.fit args to lines
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update tests/python_package_test/test_dask.py
    
    _train_part model.fit args to lines, pt2
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update python-package/lightgbm/dask.py
    
    _train_part model.fit args to lines pt3
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update tests/python_package_test/test_dask.py
    
    dask_model.fit args to lines
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update tests/python_package_test/test_dask.py
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update python-package/lightgbm/dask.py
    
    use is instead of id()
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update tests/python_package_test/test_dask.py
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update tests/python_package_test/test_dask.py
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    
    * applying changes to eval_set PR WiP
    
    * dask support for eval_names, eval_metric, eval_stopping_rounds
    
    * add evals_result checks and other eval_set attribute-related test checks. need to merge master - WiP
    
    * fix lint errors in test_dask.py
    
    * drop group_shape from _lgbmmodel_doc_fit.format for non-rankers, add support for eval_at for dask ranker
    
    * add eval_at to test_dask eval_set ranker tests
    
    * add back group_shape to lgbmmmodel docs, tighten tests
    
    * drop random eval weights from early stopping, probably causing training to terminate too early
    
    * add eval data templates to sklearn fit docs, add eval data docs to dask
    
    * add n_features to _create_data, eval_set tests stop w/ desirable tree counts
    
    * import alphabetically
    
    * add back get_worker for eval_set error handling
    
    * test_dask argmin typo
    
    * push forgotten eval_names bugfix
    
    * eval_stopping_rounds -> early_stopping_rounds, fix failing non-es test
    
    * change default eval_at to tuple 1-5
    
    * re-drop get_worker
    
    * drop early stopping support from eval_set commits, move eval_set worker check prior to client.submit
    
    * add eval_class_weight and eval_init_score to lightgbm/dask, WiP
    
    * clean up eval_set tests, allow user to specify fewer eval_names, clswghts than eval_sets
    
    * remove redundant backslash
    
    * lint fixes
    
    * fix eval_at, eval_metric duplication, let eval_at be Iterable not just Tuple
    
    * use all data_outputs for test_eval_set tests
    
    * undo newlines from first pr
    
    * add custom_eval_metric test, correct issue with eval_at and metric names
    
    * move _constant_metric outside of test
    
    * dataset reference names instead of __strings__
    
    * add padding to eval_set parts makes each part has same len(eval_set)
    
    * eval set code clean up
    
    * revert n_evals to be max len eval_set across all parts on worker
    
    * pylint errors in _DatasetNames
    
    * more pylint fixes
    
    * pylinting...
    
    * add by pytest.mark, mistakenly deleted during merge conflict resolution
    
    * address code review comments
    
    * add _pad_eval_names to handle nondeterministic evals_result_ valid set names
    
    * change not evaluated evals_result_ test criteria
    
    * address fit eval docs issues, switch _DatasetNames to Enum
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * Update python-package/lightgbm/dask.py
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * update eval_metrics, eval_at dask fit docstr to match sklearn, make tests reflect that l2 (rmse), logloss in evals_result_ by default
    
    * address eval_set dict keys naming in docstr and training eval_set naming issue
    
    * in test_dask check for obj-default metric names in eval_results, remove check for training key
    
    * lint fixes for _pad_eval_names
    
    * remove unnecessary breaklinen in _pad_eval_names docstr
    
    * use Enum.member syntax not Enum.member.name
    
    * remove str from supported eval_at types
    
    * add whitespace and remove DaskDataframes mention from eval_ param docstrs in _train
    
    * remove "of shape = [n_samples]" from group_shape docs
    
    * add eval_at base_doc in DaskLGBMRanker.fit
    
    * remove excess paren from eval_names docs in _train
    
    * make requested changes to test_dask.py
    
    * remove Optional() wrapper on eval_at
    
    * add _lgbmmodel_doc_custom_eval_note to dask.py fit.__doc__
    
    * fix ordering of .sklearn imports to attempt lint fix
    
    * dask custom eval note to f-string pt1
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * dask custom eval note to f-string pt 2
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    
    * dask custom eval note to f-string pt 3
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
    Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
    b5502d19
sklearn.py 49.5 KB