1. 28 Oct, 2021 1 commit
  2. 27 Oct, 2021 1 commit
  3. 25 Oct, 2021 1 commit
  4. 20 Oct, 2021 1 commit
  5. 13 Oct, 2021 1 commit
  6. 08 Oct, 2021 1 commit
  7. 05 Oct, 2021 3 commits
  8. 23 Sep, 2021 2 commits
  9. 25 Aug, 2021 1 commit
  10. 22 Aug, 2021 1 commit
  11. 23 Jul, 2021 1 commit
  12. 02 Jul, 2021 1 commit
    • Chen Yufei's avatar
      [python-package] Create Dataset from multiple data files (#4089) · c359896e
      Chen Yufei authored
      * [python-package] create Dataset from sampled data.
      
      * [python-package] create Dataset from List[Sequence].
      
      1. Use random access for data sampling
      2. Support read data from multiple input files
      3. Read data in batch so no need to hold all data in memory
      
      * [python-package] example: create Dataset from multiple HDF5 file.
      
      * fix: revert is_class implementation for seq
      
      * fix: unwanted memory view reference for seq
      
      * fix: seq is_class accepts sklearn matrices
      
      * fix: requirements for example
      
      * fix: pycode
      
      * feat: print static code linting stage
      
      * fix: linting: avoid shell str regex conversion
      
      * code style: doc style
      
      * code style: isort
      
      * fix ci dependency: h5py on windows
      
      * [py] remove rm files in test seq
      https://github.com/microsoft/LightGBM/pull/4089#discussion_r612929623
      
      * docs(python): init_from_sample summary
      
      https://github.com/microsoft/LightGBM/pull/4089#discussion_r612903389
      
      
      
      * remove dataset dump sample data debugging code.
      
      * remove typo fix.
      
      Create separate PR for this.
      
      * fix typo in src/c_api.cpp
      Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
      
      * style(linting): py3 type hint for seq
      
      * test(basic): os.path style path handling
      
      * Revert "feat: print static code linting stage"
      
      This reverts commit 10bd79f7f8258bea8e61c3abb8c9c7e4456a916d.
      
      * feat(python): sequence on validation set
      
      * minor(python): comment
      
      * minor(python): test option hint
      
      * style(python): fix code linting
      
      * style(python): add pydoc for ref_dataset
      
      * doc(python): sequence
      Co-authored-by: default avatarshiyu1994 <shiyu_k1994@qq.com>
      
      * revert(python): sequence class abc
      
      * chore(python): remove rm_files
      
      * Remove useless static_assert.
      
      * refactor: test_basic test for sequence.
      
      * fix lint complaint.
      
      * remove dataset._dump_text in sequence test.
      
      * Fix reverting typo fix.
      
      * Apply suggestions from code review
      Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
      
      * Fix type hint, code and doc style.
      
      * fix failing test_basic.
      
      * Remove TODO about keep constant in sync with cpp.
      
      * Install h5py only when running python-examples.
      
      * Fix lint complaint.
      
      * Apply suggestions from code review
      Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
      
      * Doc fixes, remove unused params_str in __init_from_seqs.
      
      * Apply suggestions from code review
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      
      * Remove unnecessary conda install in windows ci script.
      
      * Keep param as example in dataset_from_multi_hdf5.py
      
      * Add _get_sample_count function to remove code duplication.
      
      * Use batch_size parameter in generate_hdf.
      
      * Apply suggestions from code review
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      
      * Fix after applying suggestions.
      
      * Fix test, check idx is instance of numbers.Integral.
      
      * Update python-package/lightgbm/basic.py
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      
      * Expose Sequence class in Python-API doc.
      
      * Handle Sequence object not having batch_size.
      
      * Fix isort lint complaint.
      
      * Apply suggestions from code review
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      
      * Update docstring to mention Sequence as data input.
      
      * Remove get_one_line in test_basic.py
      
      * Make Sequence an abstract class.
      
      * Reduce number of tests for test_sequence.
      
      * Add c_api: LGBM_SampleCount, fix potential bug in LGBMSampleIndices.
      
      * empty commit to trigger ci
      
      * Apply suggestions from code review
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      
      * Rename to LGBM_GetSampleCount, change LGBM_SampleIndices out_len to int32_t.
      
      Also rename total_nrow to num_total_row in c_api.h for consistency.
      
      * Doc about Sequence in docs/Python-Intro.rst.
      
      * Fix: basic.py change LGBM_SampleIndices out_len to int32.
      
      * Add create_valid test case with Dataset from Sequence.
      
      * Apply suggestions from code review
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      
      * Apply suggestions from code review
      Co-authored-by: default avatarshiyu1994 <shiyu_k1994@qq.com>
      
      * Remove no longer used DEFAULT_BIN_CONSTRUCT_SAMPLE_CNT.
      
      * Update python-package/lightgbm/basic.py
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      Co-authored-by: default avatarWillian Zhang <willian@willian.email>
      Co-authored-by: default avatarWillian Z <Willian@Willian-Zhang.com>
      Co-authored-by: default avatarJames Lamb <jaylamb20@gmail.com>
      Co-authored-by: default avatarshiyu1994 <shiyu_k1994@qq.com>
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      c359896e
  13. 28 Jun, 2021 1 commit
  14. 26 Jun, 2021 1 commit
  15. 25 Jun, 2021 1 commit
  16. 03 Jun, 2021 2 commits
  17. 26 May, 2021 1 commit
  18. 21 May, 2021 1 commit
  19. 18 May, 2021 1 commit
  20. 10 May, 2021 1 commit
  21. 07 May, 2021 1 commit
    • Chen Yufei's avatar
      Precise text file parsing (#4081) · f8318088
      Chen Yufei authored
      
      
      * New build option: USE_PRECISE_TEXT_PARSER.
      
      Use fast_double_parser for text file parsing. For each number, fallback
      to strtod in case of parse failure.
      
      * Add benchmark for CSVParser with Atof and AtofPrecise.
      
      * Fix lint complaint.
      
      * Fix typo in open result error message.
      
      * Revert "Fix lint complaint."
      
      This reverts commit 92ab0b6bce9f17d7be9eaeb20f19d4a0a36f0387.
      
      * Revert "Add benchmark for CSVParser with Atof and AtofPrecise."
      
      This reverts commit 4f8639abd06c679d4382eb715a1793afd94df3d2.
      
      * Use AtofPrecise in Common::__StringToTHelper.
      
      * [option] precise_float_parser: precise float number parsing for text input.
      
      * Remove USE_PRECISE_TEXT_PARSER compile option.
      
      * test: add test for Common::AtofPrecise.
      
      * test: remove ChunkedArrayTest with 0 length.
      
      This triggers Log::Fatal which aborts the test program.
      
      * fix lint, add copyright.
      
      * Revert "test: remove ChunkedArrayTest with 0 length."
      
      This reverts commit 346c76affe9e78b6ca2738c4a56dbb9c00f31102.
      
      * Use LightGBM::Common::Sign
      
      * save precise_float_parser in model file.
      
      * Fix error checking in AtofPrecise. Add more test cases.
      
      * Remove test case that can't pass under macOS.
      
      * Apply suggestions from code review
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      Co-authored-by: default avatarNikita Titov <nekit94-08@mail.ru>
      f8318088
  22. 04 May, 2021 2 commits
  23. 29 Apr, 2021 1 commit
  24. 27 Apr, 2021 1 commit
  25. 23 Apr, 2021 1 commit
  26. 22 Apr, 2021 1 commit
  27. 15 Apr, 2021 1 commit
  28. 11 Apr, 2021 1 commit
  29. 05 Apr, 2021 1 commit
  30. 24 Mar, 2021 1 commit
  31. 17 Mar, 2021 1 commit
    • ashok-ponnuswami-msft's avatar
      Range check for DCG position discount lookup (#4069) · 4580393f
      ashok-ponnuswami-msft authored
      * Add check to prevent out of index lookup in the position discount table. Add debug logging to report number of queries found in the data.
      
      * Change debug logging location so that we can print the data file name as well.
      
      * Revert "Change debug logging location so that we can print the data file name as well."
      
      This reverts commit 3981b34bd6e0530f89c4733e78e6b6603bf50d48.
      
      * Add data file name to debug logging.
      
      * Move log line to a place where it is output even when query IDs are read from a separate file.
      
      * Also add the out-of-range check to rank metrics.
      
      * Perform check after number of queries is initialized.
      
      * Update
      4580393f
  32. 12 Mar, 2021 1 commit
  33. 23 Feb, 2021 1 commit
  34. 21 Feb, 2021 1 commit
    • mjmckp's avatar
      Fix evalution of linear trees with a single leaf. (#3987) · 605c97b5
      mjmckp authored
      
      
      * Fix index out-of-range exception generated by BaggingHelper on small datasets.
      
      Prior to this change, the line "score_t threshold = tmp_gradients[top_k - 1];" would generate an exception, since tmp_gradients would be empty when the cnt input value to the function is zero.
      
      * Update goss.hpp
      
      * Update goss.hpp
      
      * Add API method LGBM_BoosterPredictForMats which runs prediction on a data set given as of array of pointers to rows (as opposed to existing method LGBM_BoosterPredictForMat which requires data given as contiguous array)
      
      * Fix incorrect upstream merge
      
      * Add link to LightGBM.NET
      
      * Fix indenting to 2 spaces
      
      * Dummy edit to trigger CI
      
      * Dummy edit to trigger CI
      
      * remove duplicate functions from merge
      
      * Fix evalution of linear trees with a single leaf.
      
      Note that trees without linear models at the leaf always handle num_leaves = 1 as a special case and directly output the leaf value.  Linear trees were missing this special case handling, and hence would have the following issues:
       * Calling Tree::Predict or Tree::PredictByMap would cause an access violation exception attempting to access the first value of the empty split_feature_ array in GetLeaf.
       * PredictionFunLinear would either cause an access violation or go into an infinite loop when attempting to do the equivalent of GetLeaf.
      
      Note also that PredictionFun does not need the same changes as PredictionFunLinear, since both are only called by Tree::AddPredictionToScore, which has a special case for (!is_linear_ && num_leaves_ <= 1) that precludes calling PredictionFun.
      Co-authored-by: default avatarmatthew-peacock <matthew.peacock@whiteoakam.com>
      Co-authored-by: default avatarGuolin Ke <guolin.ke@outlook.com>
      605c97b5
  35. 19 Feb, 2021 1 commit
    • mjmckp's avatar
      Use high precision conversion from double to string in Tree::ToString() for... · 7f91dc66
      mjmckp authored
      
      Use high precision conversion from double to string in Tree::ToString() for new linear tree members (#3938)
      
      * Fix index out-of-range exception generated by BaggingHelper on small datasets.
      
      Prior to this change, the line "score_t threshold = tmp_gradients[top_k - 1];" would generate an exception, since tmp_gradients would be empty when the cnt input value to the function is zero.
      
      * Update goss.hpp
      
      * Update goss.hpp
      
      * Add API method LGBM_BoosterPredictForMats which runs prediction on a data set given as of array of pointers to rows (as opposed to existing method LGBM_BoosterPredictForMat which requires data given as contiguous array)
      
      * Fix incorrect upstream merge
      
      * Add link to LightGBM.NET
      
      * Fix indenting to 2 spaces
      
      * Dummy edit to trigger CI
      
      * Dummy edit to trigger CI
      
      * remove duplicate functions from merge
      
      * In Tree::ToString() method, print double values for linear tree models with high precision, so that the tree may be accurately reproduced elsewhere (LightGBM.Net in particular)
      
      * Need to use more precise StringToArray instead of StringToArrayFast when parsing double valued arrays for linear trees, to ensure models round-trip via string or file correctly.
      Co-authored-by: default avatarmatthew-peacock <matthew.peacock@whiteoakam.com>
      Co-authored-by: default avatarGuolin Ke <guolin.ke@outlook.com>
      7f91dc66