Parameters.rst 65.4 KB
Newer Older
1
..  List of parameters is auto generated by LightGBM\helpers\parameter_generator.py from LightGBM\include\LightGBM\config.h file.
2

3
4
5
.. role:: raw-html(raw)
    :format: html

6
7
8
Parameters
==========

9
This page contains descriptions of all parameters in LightGBM.
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

**List of other helpful links**

- `Python API <./Python-API.rst>`__

- `Parameters Tuning <./Parameters-Tuning.rst>`__

**External Links**

- `Laurae++ Interactive Documentation`_

Parameters Format
-----------------

The parameters format is ``key1=value1 key2=value2 ...``.
25
Parameters can be set both in config file and command line.
26
27
28
By using command line, parameters should not have spaces before and after ``=``.
By using config files, one line can only contain one parameter. You can use ``#`` to comment.

29
If one parameter appears in both command line and config file, LightGBM will use the parameter from the command line.
30

31
32
.. start params list

33
34
35
Core Parameters
---------------

36
-  ``config`` :raw-html:`<a id="config" title="Permalink to this parameter" href="#config">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string, aliases: ``config_file``
37
38
39

   -  path of config file

40
   -  **Note**: can be used only in CLI version
41

42
-  ``task`` :raw-html:`<a id="task" title="Permalink to this parameter" href="#task">&#x1F517;&#xFE0E;</a>`, default = ``train``, type = enum, options: ``train``, ``predict``, ``convert_model``, ``refit``, aliases: ``task_type``
43

44
   -  ``train``, for training, aliases: ``training``
45

46
   -  ``predict``, for prediction, aliases: ``prediction``, ``test``
47

48
   -  ``convert_model``, for converting model file into if-else format, see more information in `IO Parameters <#io-parameters>`__
49

50
   -  ``refit``, for refitting existing models with new data, aliases: ``refit_tree``
51

Guolin Ke's avatar
Guolin Ke committed
52
   -  **Note**: can be used only in CLI version; for language-specific packages you can use the correspondent functions
53

54
-  ``objective`` :raw-html:`<a id="objective" title="Permalink to this parameter" href="#objective">&#x1F517;&#xFE0E;</a>`, default = ``regression``, type = enum, options: ``regression``, ``regression_l1``, ``huber``, ``fair``, ``poisson``, ``quantile``, ``mape``, ``gamma``, ``tweedie``, ``binary``, ``multiclass``, ``multiclassova``, ``cross_entropy``, ``cross_entropy_lambda``, ``lambdarank``, ``rank_xendcg``, aliases: ``objective_type``, ``app``, ``application``
55

56
   -  regression application
57

Guolin Ke's avatar
Guolin Ke committed
58
      -  ``regression``, L2 loss, aliases: ``regression_l2``, ``l2``, ``mean_squared_error``, ``mse``, ``l2_root``, ``root_mean_squared_error``, ``rmse``
59

Guolin Ke's avatar
Guolin Ke committed
60
      -  ``regression_l1``, L1 loss, aliases: ``l1``, ``mean_absolute_error``, ``mae``
61

62
      -  ``huber``, `Huber loss <https://en.wikipedia.org/wiki/Huber_loss>`__
63

64
      -  ``fair``, `Fair loss <https://www.kaggle.com/c/allstate-claims-severity/discussion/24520>`__
65

66
      -  ``poisson``, `Poisson regression <https://en.wikipedia.org/wiki/Poisson_regression>`__
67

68
      -  ``quantile``, `Quantile regression <https://en.wikipedia.org/wiki/Quantile_regression>`__
69

70
      -  ``mape``, `MAPE loss <https://en.wikipedia.org/wiki/Mean_absolute_percentage_error>`__, aliases: ``mean_absolute_percentage_error``
71

72
      -  ``gamma``, Gamma regression with log-link. It might be useful, e.g., for modeling insurance claims severity, or for any target that might be `gamma-distributed <https://en.wikipedia.org/wiki/Gamma_distribution#Occurrence_and_applications>`__
Guolin Ke's avatar
Guolin Ke committed
73

74
      -  ``tweedie``, Tweedie regression with log-link. It might be useful, e.g., for modeling total loss in insurance, or for any target that might be `tweedie-distributed <https://en.wikipedia.org/wiki/Tweedie_distribution#Occurrence_and_applications>`__
Guolin Ke's avatar
Guolin Ke committed
75

76
77
78
79
80
   -  binary classification application

      -  ``binary``, binary `log loss <https://en.wikipedia.org/wiki/Cross_entropy>`__ classification (or logistic regression)

      -  requires labels in {0, 1}; see ``cross-entropy`` application for general probability labels in [0, 1]
81
82
83

   -  multi-class classification application

84
      -  ``multiclass``, `softmax <https://en.wikipedia.org/wiki/Softmax_function>`__ objective function, aliases: ``softmax``
85

86
      -  ``multiclassova``, `One-vs-All <https://en.wikipedia.org/wiki/Multiclass_classification#One-vs.-rest>`__ binary objective function, aliases: ``multiclass_ova``, ``ova``, ``ovr``
Nikita Titov's avatar
Nikita Titov committed
87
88

      -  ``num_class`` should be set as well
89
90
91

   -  cross-entropy application

Guolin Ke's avatar
Guolin Ke committed
92
      -  ``cross_entropy``, objective function for cross-entropy (with optional linear weights), aliases: ``xentropy``
93

Guolin Ke's avatar
Guolin Ke committed
94
      -  ``cross_entropy_lambda``, alternative parameterization of cross-entropy, aliases: ``xentlambda``
95

96
      -  label is anything in interval [0, 1]
97

98
   -  ranking application
99

100
      -  ``lambdarank``, `lambdarank <https://papers.nips.cc/paper/2971-learning-to-rank-with-nonsmooth-cost-functions.pdf>`__ objective. `label_gain <#label_gain>`__ can be used to set the gain (weight) of ``int`` label and all values in ``label`` must be smaller than number of elements in ``label_gain``
101

102
      -  ``rank_xendcg``, `XE_NDCG_MART <https://arxiv.org/abs/1911.09798>`__ ranking objective function, aliases: ``xendcg``, ``xe_ndcg``, ``xe_ndcg_mart``, ``xendcg_mart``
103

104
      -  ``rank_xendcg`` is faster than and achieves the similar performance as ``lambdarank``
105

106
      -  label should be ``int`` type, and larger number represents the higher relevance (e.g. 0:bad, 1:fair, 2:good, 3:perfect)
107

108
-  ``boosting`` :raw-html:`<a id="boosting" title="Permalink to this parameter" href="#boosting">&#x1F517;&#xFE0E;</a>`, default = ``gbdt``, type = enum, options: ``gbdt``, ``rf``, ``dart``, ``goss``, aliases: ``boosting_type``, ``boost``
109

110
   -  ``gbdt``, traditional Gradient Boosting Decision Tree, aliases: ``gbrt``
111

112
   -  ``rf``, Random Forest, aliases: ``random_forest``
113

114
   -  ``dart``, `Dropouts meet Multiple Additive Regression Trees <https://arxiv.org/abs/1505.01866>`__
115
116
117

   -  ``goss``, Gradient-based One-Side Sampling

118
-  ``data`` :raw-html:`<a id="data" title="Permalink to this parameter" href="#data">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string, aliases: ``train``, ``train_data``, ``train_data_file``, ``data_filename``
119

120
   -  path of training data, LightGBM will train from this data
121

122
123
   -  **Note**: can be used only in CLI version

124
-  ``valid`` :raw-html:`<a id="valid" title="Permalink to this parameter" href="#valid">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string, aliases: ``test``, ``valid_data``, ``valid_data_file``, ``test_data``, ``test_data_file``, ``valid_filenames``
125

126
   -  path(s) of validation/test data, LightGBM will output metrics for these data
127

128
   -  support multiple validation data, separated by ``,``
129

130
131
   -  **Note**: can be used only in CLI version

132
-  ``num_iterations`` :raw-html:`<a id="num_iterations" title="Permalink to this parameter" href="#num_iterations">&#x1F517;&#xFE0E;</a>`, default = ``100``, type = int, aliases: ``num_iteration``, ``n_iter``, ``num_tree``, ``num_trees``, ``num_round``, ``num_rounds``, ``num_boost_round``, ``n_estimators``, constraints: ``num_iterations >= 0``
133
134

   -  number of boosting iterations
135

136
   -  **Note**: internally, LightGBM constructs ``num_class * num_iterations`` trees for multi-class classification problems
137

138
-  ``learning_rate`` :raw-html:`<a id="learning_rate" title="Permalink to this parameter" href="#learning_rate">&#x1F517;&#xFE0E;</a>`, default = ``0.1``, type = double, aliases: ``shrinkage_rate``, ``eta``, constraints: ``learning_rate > 0.0``
139
140
141
142
143

   -  shrinkage rate

   -  in ``dart``, it also affects on normalization weights of dropped trees

144
-  ``num_leaves`` :raw-html:`<a id="num_leaves" title="Permalink to this parameter" href="#num_leaves">&#x1F517;&#xFE0E;</a>`, default = ``31``, type = int, aliases: ``num_leaf``, ``max_leaves``, ``max_leaf``, constraints: ``1 < num_leaves <= 131072``
145

146
   -  max number of leaves in one tree
147

148
-  ``tree_learner`` :raw-html:`<a id="tree_learner" title="Permalink to this parameter" href="#tree_learner">&#x1F517;&#xFE0E;</a>`, default = ``serial``, type = enum, options: ``serial``, ``feature``, ``data``, ``voting``, aliases: ``tree``, ``tree_type``, ``tree_learner_type``
149
150
151

   -  ``serial``, single machine tree learner

152
   -  ``feature``, feature parallel tree learner, aliases: ``feature_parallel``
153

154
   -  ``data``, data parallel tree learner, aliases: ``data_parallel``
155

156
   -  ``voting``, voting parallel tree learner, aliases: ``voting_parallel``
157
158
159

   -  refer to `Parallel Learning Guide <./Parallel-Learning-Guide.rst>`__ to get more details

160
-  ``num_threads`` :raw-html:`<a id="num_threads" title="Permalink to this parameter" href="#num_threads">&#x1F517;&#xFE0E;</a>`, default = ``0``, type = int, aliases: ``num_thread``, ``nthread``, ``nthreads``, ``n_jobs``
161
162
163

   -  number of threads for LightGBM

164
   -  ``0`` means default number of threads in OpenMP
165

166
   -  for the best speed, set this to the number of **real CPU cores**, not the number of threads (most CPUs use `hyper-threading <https://en.wikipedia.org/wiki/Hyper-threading>`__ to generate 2 threads per CPU core)
167

168
   -  do not set it too large if your dataset is small (for instance, do not use 64 threads for a dataset with 10,000 rows)
169

170
   -  be aware a task manager or any similar CPU monitoring tool might report that cores not being fully utilized. **This is normal**
171

172
   -  for parallel learning, do not use all CPU cores because this will cause poor performance for the network communication
173

174
175
   -  **Note**: please **don't** change this during training, especially when running multiple jobs simultaneously by external packages, otherwise it may cause undesirable errors

176
-  ``device_type`` :raw-html:`<a id="device_type" title="Permalink to this parameter" href="#device_type">&#x1F517;&#xFE0E;</a>`, default = ``cpu``, type = enum, options: ``cpu``, ``gpu``, aliases: ``device``
177
178

   -  device for the tree learning, you can use GPU to achieve the faster learning
179
180
181

   -  **Note**: it is recommended to use the smaller ``max_bin`` (e.g. 63) to get the better speed up

182
183
184
185
   -  **Note**: for the faster speed, GPU uses 32-bit float point to sum up by default, so this may affect the accuracy for some tasks. You can set ``gpu_use_dp=true`` to enable 64-bit float point, but it will slow down the training

   -  **Note**: refer to `Installation Guide <./Installation-Guide.rst#build-gpu-version>`__ to build LightGBM with GPU support

186
-  ``seed`` :raw-html:`<a id="seed" title="Permalink to this parameter" href="#seed">&#x1F517;&#xFE0E;</a>`, default = ``None``, type = int, aliases: ``random_seed``, ``random_state``
187

188
   -  this seed is used to generate other seeds, e.g. ``data_random_seed``, ``feature_fraction_seed``, etc.
189

190
191
192
   -  by default, this seed is unused in favor of default values of other seeds

   -  this seed has lower priority in comparison with other seeds, which means that it will be overridden, if you set other seeds explicitly
193
194
195
196

Learning Control Parameters
---------------------------

197
198
-  ``force_col_wise`` :raw-html:`<a id="force_col_wise" title="Permalink to this parameter" href="#force_col_wise">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool

199
200
201
   -  used only with ``cpu`` device type

   -  set this to ``true`` to force col-wise histogram building
202

203
   -  enabling this is recommended when:
204

205
      -  the number of columns is large, or the total number of bins is large
206

207
      -  ``num_threads`` is large, e.g. ``>20``
208

209
      -  you want to reduce memory cost
210

211
212
213
   -  **Note**: when both ``force_col_wise`` and ``force_row_wise`` are ``false``, LightGBM will firstly try them both, and then use the faster one. To remove the overhead of testing set the faster one to ``true`` manually

   -  **Note**: this parameter cannot be used at the same time with ``force_row_wise``, choose only one of them
214
215
216

-  ``force_row_wise`` :raw-html:`<a id="force_row_wise" title="Permalink to this parameter" href="#force_row_wise">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool

217
218
219
220
221
   -  used only with ``cpu`` device type

   -  set this to ``true`` to force row-wise histogram building

   -  enabling this is recommended when:
222

223
      -  the number of data points is large, and the total number of bins is relatively small
224

225
      -  ``num_threads`` is relatively small, e.g. ``<=16``
226

227
      -  you want to use small ``bagging_fraction`` or ``goss`` boosting to speed up
228

229
   -  **Note**: setting this to ``true`` will double the memory cost for Dataset object. If you have not enough memory, you can try setting ``force_col_wise=true``
230

231
   -  **Note**: when both ``force_col_wise`` and ``force_row_wise`` are ``false``, LightGBM will firstly try them both, and then use the faster one. To remove the overhead of testing set the faster one to ``true`` manually
232

233
   -  **Note**: this parameter cannot be used at the same time with ``force_col_wise``, choose only one of them
234

235
236
237
238
239
240
-  ``histogram_pool_size`` :raw-html:`<a id="histogram_pool_size" title="Permalink to this parameter" href="#histogram_pool_size">&#x1F517;&#xFE0E;</a>`, default = ``-1.0``, type = double, aliases: ``hist_pool_size``

   -  max cache size in MB for historical histogram

   -  ``< 0`` means no limit

241
-  ``max_depth`` :raw-html:`<a id="max_depth" title="Permalink to this parameter" href="#max_depth">&#x1F517;&#xFE0E;</a>`, default = ``-1``, type = int
242

243
   -  limit the max depth for tree model. This is used to deal with over-fitting when ``#data`` is small. Tree still grows leaf-wise
244

245
   -  ``<= 0`` means no limit
246

247
-  ``min_data_in_leaf`` :raw-html:`<a id="min_data_in_leaf" title="Permalink to this parameter" href="#min_data_in_leaf">&#x1F517;&#xFE0E;</a>`, default = ``20``, type = int, aliases: ``min_data_per_leaf``, ``min_data``, ``min_child_samples``, constraints: ``min_data_in_leaf >= 0``
248
249
250

   -  minimal number of data in one leaf. Can be used to deal with over-fitting

251
-  ``min_sum_hessian_in_leaf`` :raw-html:`<a id="min_sum_hessian_in_leaf" title="Permalink to this parameter" href="#min_sum_hessian_in_leaf">&#x1F517;&#xFE0E;</a>`, default = ``1e-3``, type = double, aliases: ``min_sum_hessian_per_leaf``, ``min_sum_hessian``, ``min_hessian``, ``min_child_weight``, constraints: ``min_sum_hessian_in_leaf >= 0.0``
252
253
254

   -  minimal sum hessian in one leaf. Like ``min_data_in_leaf``, it can be used to deal with over-fitting

255
-  ``bagging_fraction`` :raw-html:`<a id="bagging_fraction" title="Permalink to this parameter" href="#bagging_fraction">&#x1F517;&#xFE0E;</a>`, default = ``1.0``, type = double, aliases: ``sub_row``, ``subsample``, ``bagging``, constraints: ``0.0 < bagging_fraction <= 1.0``
256

257
   -  like ``feature_fraction``, but this will randomly select part of data without resampling
258
259
260
261
262

   -  can be used to speed up training

   -  can be used to deal with over-fitting

263
   -  **Note**: to enable bagging, ``bagging_freq`` should be set to a non zero value as well
264

Guolin Ke's avatar
Guolin Ke committed
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
-  ``pos_bagging_fraction`` :raw-html:`<a id="pos_bagging_fraction" title="Permalink to this parameter" href="#pos_bagging_fraction">&#x1F517;&#xFE0E;</a>`, default = ``1.0``, type = double, aliases: ``pos_sub_row``, ``pos_subsample``, ``pos_bagging``, constraints: ``0.0 < pos_bagging_fraction <= 1.0``

   -  used only in ``binary`` application

   -  used for imbalanced binary classification problem, will randomly sample ``#pos_samples * pos_bagging_fraction`` positive samples in bagging

   -  should be used together with ``neg_bagging_fraction``

   -  set this to ``1.0`` to disable

   -  **Note**: to enable this, you need to set ``bagging_freq`` and ``neg_bagging_fraction`` as well

   -  **Note**: if both ``pos_bagging_fraction`` and ``neg_bagging_fraction`` are set to ``1.0``,  balanced bagging is disabled

   -  **Note**: if balanced bagging is enabled, ``bagging_fraction`` will be ignored

-  ``neg_bagging_fraction`` :raw-html:`<a id="neg_bagging_fraction" title="Permalink to this parameter" href="#neg_bagging_fraction">&#x1F517;&#xFE0E;</a>`, default = ``1.0``, type = double, aliases: ``neg_sub_row``, ``neg_subsample``, ``neg_bagging``, constraints: ``0.0 < neg_bagging_fraction <= 1.0``

   -  used only in ``binary`` application

   -  used for imbalanced binary classification problem, will randomly sample ``#neg_samples * neg_bagging_fraction`` negative samples in bagging

   -  should be used together with ``pos_bagging_fraction``

   -  set this to ``1.0`` to disable

   -  **Note**: to enable this, you need to set ``bagging_freq`` and ``pos_bagging_fraction`` as well

   -  **Note**: if both ``pos_bagging_fraction`` and ``neg_bagging_fraction`` are set to ``1.0``,  balanced bagging is disabled

   -  **Note**: if balanced bagging is enabled, ``bagging_fraction`` will be ignored

297
-  ``bagging_freq`` :raw-html:`<a id="bagging_freq" title="Permalink to this parameter" href="#bagging_freq">&#x1F517;&#xFE0E;</a>`, default = ``0``, type = int, aliases: ``subsample_freq``
298

299
   -  frequency for bagging
300

301
302
303
304
   -  ``0`` means disable bagging; ``k`` means perform bagging at every ``k`` iteration

   -  **Note**: to enable bagging, ``bagging_fraction`` should be set to value smaller than ``1.0`` as well

305
-  ``bagging_seed`` :raw-html:`<a id="bagging_seed" title="Permalink to this parameter" href="#bagging_seed">&#x1F517;&#xFE0E;</a>`, default = ``3``, type = int, aliases: ``bagging_fraction_seed``
306
307
308

   -  random seed for bagging

309
-  ``feature_fraction`` :raw-html:`<a id="feature_fraction" title="Permalink to this parameter" href="#feature_fraction">&#x1F517;&#xFE0E;</a>`, default = ``1.0``, type = double, aliases: ``sub_feature``, ``colsample_bytree``, constraints: ``0.0 < feature_fraction <= 1.0``
310

311
   -  LightGBM will randomly select part of features on each iteration (tree) if ``feature_fraction`` smaller than ``1.0``. For example, if you set it to ``0.8``, LightGBM will select 80% of features before training each tree
312

313
   -  can be used to speed up training
314

315
   -  can be used to deal with over-fitting
316

317
-  ``feature_fraction_bynode`` :raw-html:`<a id="feature_fraction_bynode" title="Permalink to this parameter" href="#feature_fraction_bynode">&#x1F517;&#xFE0E;</a>`, default = ``1.0``, type = double, aliases: ``sub_feature_bynode``, ``colsample_bynode``, constraints: ``0.0 < feature_fraction_bynode <= 1.0``
318

Nikita Titov's avatar
Nikita Titov committed
319
   -  LightGBM will randomly select part of features on each tree node if ``feature_fraction_bynode`` smaller than ``1.0``. For example, if you set it to ``0.8``, LightGBM will select 80% of features at each tree node
320
321
322

   -  can be used to deal with over-fitting

323
324
325
326
   -  **Note**: unlike ``feature_fraction``, this cannot speed up training

   -  **Note**: if both ``feature_fraction`` and ``feature_fraction_bynode`` are smaller than ``1.0``, the final fraction of each node is ``feature_fraction * feature_fraction_bynode``

327
-  ``feature_fraction_seed`` :raw-html:`<a id="feature_fraction_seed" title="Permalink to this parameter" href="#feature_fraction_seed">&#x1F517;&#xFE0E;</a>`, default = ``2``, type = int
328
329

   -  random seed for ``feature_fraction``
330

331
332
333
334
335
336
337
338
339
340
341
342
-  ``extra_trees`` :raw-html:`<a id="extra_trees" title="Permalink to this parameter" href="#extra_trees">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool

   -  use extremely randomized trees

   -  if set to ``true``, when evaluating node splits LightGBM will check only one randomly-chosen threshold for each feature

   -  can be used to deal with over-fitting

-  ``extra_seed`` :raw-html:`<a id="extra_seed" title="Permalink to this parameter" href="#extra_seed">&#x1F517;&#xFE0E;</a>`, default = ``6``, type = int

   -  random seed for selecting thresholds when ``extra_trees`` is true

343
-  ``early_stopping_round`` :raw-html:`<a id="early_stopping_round" title="Permalink to this parameter" href="#early_stopping_round">&#x1F517;&#xFE0E;</a>`, default = ``0``, type = int, aliases: ``early_stopping_rounds``, ``early_stopping``, ``n_iter_no_change``
344

345
   -  will stop training if one metric of one validation data doesn't improve in last ``early_stopping_round`` rounds
346

347
   -  ``<= 0`` means disable
348

349
350
351
352
-  ``first_metric_only`` :raw-html:`<a id="first_metric_only" title="Permalink to this parameter" href="#first_metric_only">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool

   -  set this to ``true``, if you want to use only the first metric for early stopping

353
-  ``max_delta_step`` :raw-html:`<a id="max_delta_step" title="Permalink to this parameter" href="#max_delta_step">&#x1F517;&#xFE0E;</a>`, default = ``0.0``, type = double, aliases: ``max_tree_output``, ``max_leaf_output``
354

355
   -  used to limit the max output of tree leaves
356

357
   -  ``<= 0`` means no constraint
358

359
   -  the final max output of leaves is ``learning_rate * max_delta_step``
360

361
-  ``lambda_l1`` :raw-html:`<a id="lambda_l1" title="Permalink to this parameter" href="#lambda_l1">&#x1F517;&#xFE0E;</a>`, default = ``0.0``, type = double, aliases: ``reg_alpha``, constraints: ``lambda_l1 >= 0.0``
362
363
364

   -  L1 regularization

365
-  ``lambda_l2`` :raw-html:`<a id="lambda_l2" title="Permalink to this parameter" href="#lambda_l2">&#x1F517;&#xFE0E;</a>`, default = ``0.0``, type = double, aliases: ``reg_lambda``, ``lambda``, constraints: ``lambda_l2 >= 0.0``
366
367
368

   -  L2 regularization

369
-  ``min_gain_to_split`` :raw-html:`<a id="min_gain_to_split" title="Permalink to this parameter" href="#min_gain_to_split">&#x1F517;&#xFE0E;</a>`, default = ``0.0``, type = double, aliases: ``min_split_gain``, constraints: ``min_gain_to_split >= 0.0``
370

371
   -  the minimal gain to perform split
372

373
-  ``drop_rate`` :raw-html:`<a id="drop_rate" title="Permalink to this parameter" href="#drop_rate">&#x1F517;&#xFE0E;</a>`, default = ``0.1``, type = double, aliases: ``rate_drop``, constraints: ``0.0 <= drop_rate <= 1.0``
374

375
   -  used only in ``dart``
376

377
   -  dropout rate: a fraction of previous trees to drop during the dropout
378

379
-  ``max_drop`` :raw-html:`<a id="max_drop" title="Permalink to this parameter" href="#max_drop">&#x1F517;&#xFE0E;</a>`, default = ``50``, type = int
380

381
   -  used only in ``dart``
382

383
   -  max number of dropped trees during one boosting iteration
384

385
   -  ``<=0`` means no limit
386

387
-  ``skip_drop`` :raw-html:`<a id="skip_drop" title="Permalink to this parameter" href="#skip_drop">&#x1F517;&#xFE0E;</a>`, default = ``0.5``, type = double, constraints: ``0.0 <= skip_drop <= 1.0``
388

389
   -  used only in ``dart``
390

391
   -  probability of skipping the dropout procedure during a boosting iteration
392

393
-  ``xgboost_dart_mode`` :raw-html:`<a id="xgboost_dart_mode" title="Permalink to this parameter" href="#xgboost_dart_mode">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool
394

395
   -  used only in ``dart``
396

397
   -  set this to ``true``, if you want to use xgboost dart mode
398

399
-  ``uniform_drop`` :raw-html:`<a id="uniform_drop" title="Permalink to this parameter" href="#uniform_drop">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool
400

401
   -  used only in ``dart``
402

403
   -  set this to ``true``, if you want to use uniform drop
404

405
-  ``drop_seed`` :raw-html:`<a id="drop_seed" title="Permalink to this parameter" href="#drop_seed">&#x1F517;&#xFE0E;</a>`, default = ``4``, type = int
406

407
   -  used only in ``dart``
408

409
   -  random seed to choose dropping models
410

411
-  ``top_rate`` :raw-html:`<a id="top_rate" title="Permalink to this parameter" href="#top_rate">&#x1F517;&#xFE0E;</a>`, default = ``0.2``, type = double, constraints: ``0.0 <= top_rate <= 1.0``
412

413
   -  used only in ``goss``
414

415
   -  the retain ratio of large gradient data
416

417
-  ``other_rate`` :raw-html:`<a id="other_rate" title="Permalink to this parameter" href="#other_rate">&#x1F517;&#xFE0E;</a>`, default = ``0.1``, type = double, constraints: ``0.0 <= other_rate <= 1.0``
418

419
   -  used only in ``goss``
420

421
422
   -  the retain ratio of small gradient data

423
-  ``min_data_per_group`` :raw-html:`<a id="min_data_per_group" title="Permalink to this parameter" href="#min_data_per_group">&#x1F517;&#xFE0E;</a>`, default = ``100``, type = int, constraints: ``min_data_per_group > 0``
424
425

   -  minimal number of data per categorical group
426

427
-  ``max_cat_threshold`` :raw-html:`<a id="max_cat_threshold" title="Permalink to this parameter" href="#max_cat_threshold">&#x1F517;&#xFE0E;</a>`, default = ``32``, type = int, constraints: ``max_cat_threshold > 0``
428

429
   -  used for the categorical features
430

431
   -  limit the max threshold points in categorical features
432

433
-  ``cat_l2`` :raw-html:`<a id="cat_l2" title="Permalink to this parameter" href="#cat_l2">&#x1F517;&#xFE0E;</a>`, default = ``10.0``, type = double, constraints: ``cat_l2 >= 0.0``
434
435

   -  used for the categorical features
Guolin Ke's avatar
Guolin Ke committed
436

437
   -  L2 regularization in categorical split
438

439
-  ``cat_smooth`` :raw-html:`<a id="cat_smooth" title="Permalink to this parameter" href="#cat_smooth">&#x1F517;&#xFE0E;</a>`, default = ``10.0``, type = double, constraints: ``cat_smooth >= 0.0``
440
441
442
443
444

   -  used for the categorical features

   -  this can reduce the effect of noises in categorical features, especially for categories with few data

445
-  ``max_cat_to_onehot`` :raw-html:`<a id="max_cat_to_onehot" title="Permalink to this parameter" href="#max_cat_to_onehot">&#x1F517;&#xFE0E;</a>`, default = ``4``, type = int, constraints: ``max_cat_to_onehot > 0``
446

447
448
   -  when number of categories of one feature smaller than or equal to ``max_cat_to_onehot``, one-vs-other split algorithm will be used

449
-  ``top_k`` :raw-html:`<a id="top_k" title="Permalink to this parameter" href="#top_k">&#x1F517;&#xFE0E;</a>`, default = ``20``, type = int, aliases: ``topk``, constraints: ``top_k > 0``
450

451
   -  used only in ``voting`` tree learner, refer to `Voting parallel <./Parallel-Learning-Guide.rst#choose-appropriate-parallel-algorithm>`__
452
453

   -  set this to larger value for more accurate result, but it will slow down the training speed
454

455
-  ``monotone_constraints`` :raw-html:`<a id="monotone_constraints" title="Permalink to this parameter" href="#monotone_constraints">&#x1F517;&#xFE0E;</a>`, default = ``None``, type = multi-int, aliases: ``mc``, ``monotone_constraint``
Guolin Ke's avatar
Guolin Ke committed
456

457
   -  used for constraints of monotonic features
Guolin Ke's avatar
Guolin Ke committed
458

459
   -  ``1`` means increasing, ``-1`` means decreasing, ``0`` means non-constraint
Guolin Ke's avatar
Guolin Ke committed
460

461
462
   -  you need to specify all features in order. For example, ``mc=-1,0,1`` means decreasing for 1st feature, non-constraint for 2nd feature and increasing for the 3rd feature

463
464
465
466
467
468
469
470
471
472
-  ``monotone_constraints_method`` :raw-html:`<a id="monotone_constraints_method" title="Permalink to this parameter" href="#monotone_constraints_method">&#x1F517;&#xFE0E;</a>`, default = ``basic``, type = string, aliases: ``monotone_constraining_method``, ``mc_method``

   -  used only if ``monotone_constraints`` is set

   -  monotone constraints method

      -  ``basic``, the most basic monotone constraints method. It does not slow the library at all, but over-constrains the predictions

      -  ``intermediate``, a `more advanced method <https://github.com/microsoft/LightGBM/files/3457826/PR-monotone-constraints-report.pdf>`__, which may slow the library very slightly. However, this method is much less constraining than the basic method and should significantly improve the results

473
474
475
476
477
478
479
480
-  ``monotone_penalty`` :raw-html:`<a id="monotone_penalty" title="Permalink to this parameter" href="#monotone_penalty">&#x1F517;&#xFE0E;</a>`, default = ``0.0``, type = double, aliases: ``monotone_splits_penalty``, ``ms_penalty``, ``mc_penalty``, constraints: ``monotone_penalty >= 0.0``

   -  used only if ``monotone_constraints`` is set

   -  `monotone penalty <https://github.com/microsoft/LightGBM/files/3457826/PR-monotone-constraints-report.pdf>`__: a penalization parameter X forbids any monotone splits on the first X (rounded down) level(s) of the tree. The penalty applied to monotone splits on a given depth is a continuous, increasing function the penalization parameter

   -  if ``0.0`` (the default), no penalization is applied

481
-  ``feature_contri`` :raw-html:`<a id="feature_contri" title="Permalink to this parameter" href="#feature_contri">&#x1F517;&#xFE0E;</a>`, default = ``None``, type = multi-double, aliases: ``feature_contrib``, ``fc``, ``fp``, ``feature_penalty``
Guolin Ke's avatar
Guolin Ke committed
482
483
484
485
486

   -  used to control feature's split gain, will use ``gain[i] = max(0, feature_contri[i]) * gain[i]`` to replace the split gain of i-th feature

   -  you need to specify all features in order

487
-  ``forcedsplits_filename`` :raw-html:`<a id="forcedsplits_filename" title="Permalink to this parameter" href="#forcedsplits_filename">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string, aliases: ``fs``, ``forced_splits_filename``, ``forced_splits_file``, ``forced_splits``
488
489
490
491
492
493
494

   -  path to a ``.json`` file that specifies splits to force at the top of every decision tree before best-first learning commences

   -  ``.json`` file can be arbitrarily nested, and each split contains ``feature``, ``threshold`` fields, as well as ``left`` and ``right`` fields representing subsplits

   -  categorical splits are forced in a one-hot fashion, with ``left`` representing the split containing the feature value and ``right`` representing other values

495
496
   -  **Note**: the forced split logic will be ignored, if the split makes gain worse

497
   -  see `this file <https://github.com/microsoft/LightGBM/tree/master/examples/binary_classification/forced_splits.json>`__ as an example
Guolin Ke's avatar
Guolin Ke committed
498

Guolin Ke's avatar
Guolin Ke committed
499
500
501
502
503
504
-  ``refit_decay_rate`` :raw-html:`<a id="refit_decay_rate" title="Permalink to this parameter" href="#refit_decay_rate">&#x1F517;&#xFE0E;</a>`, default = ``0.9``, type = double, constraints: ``0.0 <= refit_decay_rate <= 1.0``

   -  decay rate of ``refit`` task, will use ``leaf_output = refit_decay_rate * old_leaf_output + (1.0 - refit_decay_rate) * new_leaf_output`` to refit trees

   -  used only in ``refit`` task in CLI version or as argument in ``refit`` function in language-specific package

505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
-  ``cegb_tradeoff`` :raw-html:`<a id="cegb_tradeoff" title="Permalink to this parameter" href="#cegb_tradeoff">&#x1F517;&#xFE0E;</a>`, default = ``1.0``, type = double, constraints: ``cegb_tradeoff >= 0.0``

   -  cost-effective gradient boosting multiplier for all penalties

-  ``cegb_penalty_split`` :raw-html:`<a id="cegb_penalty_split" title="Permalink to this parameter" href="#cegb_penalty_split">&#x1F517;&#xFE0E;</a>`, default = ``0.0``, type = double, constraints: ``cegb_penalty_split >= 0.0``

   -  cost-effective gradient-boosting penalty for splitting a node

-  ``cegb_penalty_feature_lazy`` :raw-html:`<a id="cegb_penalty_feature_lazy" title="Permalink to this parameter" href="#cegb_penalty_feature_lazy">&#x1F517;&#xFE0E;</a>`, default = ``0,0,...,0``, type = multi-double

   -  cost-effective gradient boosting penalty for using a feature

   -  applied per data point

-  ``cegb_penalty_feature_coupled`` :raw-html:`<a id="cegb_penalty_feature_coupled" title="Permalink to this parameter" href="#cegb_penalty_feature_coupled">&#x1F517;&#xFE0E;</a>`, default = ``0,0,...,0``, type = multi-double

   -  cost-effective gradient boosting penalty for using a feature

   -  applied once per forest

Belinda Trotta's avatar
Belinda Trotta committed
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
-  ``path_smooth`` :raw-html:`<a id="path_smooth" title="Permalink to this parameter" href="#path_smooth">&#x1F517;&#xFE0E;</a>`, default = ``0``, type = double, constraints: ``path_smooth >=  0.0``

   -  controls smoothing applied to tree nodes

   -  helps prevent overfitting on leaves with few samples

   -  if set to zero, no smoothing is applied

   -  if ``path_smooth > 0`` then ``min_data_in_leaf`` must be at least ``2``

   -  larger values give stronger regularisation

      -  the weight of each node is ``(n / path_smooth) * w + w_p / (n / path_smooth + 1)``, where ``n`` is the number of samples in the node, ``w`` is the optimal node weight to minimise the loss (approximately ``-sum_gradients / sum_hessians``), and ``w_p`` is the weight of the parent node

      -  note that the parent output ``w_p`` itself has smoothing applied, unless it is the root node, so that the smoothing effect accumulates with the tree depth

541
542
543
544
545
546
547
548
549
550
-  ``interaction_constraints`` :raw-html:`<a id="interaction_constraints" title="Permalink to this parameter" href="#interaction_constraints">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string

   -  controls which features can appear in the same branch

   -  by default interaction constraints are disabled, to enable them you can specify

      -  for CLI, lists separated by commas, e.g. ``[0,1,2],[2,3]``

      -  for Python-package, list of lists, e.g. ``[[0, 1, 2], [2, 3]]``

551
      -  for R-package, list of character or numeric vectors, e.g. ``list(c("var1", "var2", "var3"), c("var3", "var4"))`` or ``list(c(1L, 2L, 3L), c(3L, 4L))``. Numeric vectors should use 1-based indexing, where ``1L`` is the first feature, ``2L`` is the second feature, etc
552
553
554

   -  any two features can only appear in the same branch only if there exists a constraint containing both features

555
-  ``verbosity`` :raw-html:`<a id="verbosity" title="Permalink to this parameter" href="#verbosity">&#x1F517;&#xFE0E;</a>`, default = ``1``, type = int, aliases: ``verbose``
556
557
558

   -  controls the level of LightGBM's verbosity

559
   -  ``< 0``: Fatal, ``= 0``: Error (Warning), ``= 1``: Info, ``> 1``: Debug
560

561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
-  ``input_model`` :raw-html:`<a id="input_model" title="Permalink to this parameter" href="#input_model">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string, aliases: ``model_input``, ``model_in``

   -  filename of input model

   -  for ``prediction`` task, this model will be applied to prediction data

   -  for ``train`` task, training will be continued from this model

   -  **Note**: can be used only in CLI version

-  ``output_model`` :raw-html:`<a id="output_model" title="Permalink to this parameter" href="#output_model">&#x1F517;&#xFE0E;</a>`, default = ``LightGBM_model.txt``, type = string, aliases: ``model_output``, ``model_out``

   -  filename of output model in training

   -  **Note**: can be used only in CLI version
576
577
578
579
580
581
582
583

-  ``saved_feature_importance_type`` :raw-html:`<a id="saved_feature_importance_type" title="Permalink to this parameter" href="#saved_feature_importance_type">&#x1F517;&#xFE0E;</a>`, default = ``0``, type = int

   -  the feature importance type in the saved model file

   -  ``0``: count-based feature importance (numbers of splits are counted); ``1``: gain-based feature importance (values of gain are counted)

   -  **Note**: can be used only in CLI version
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598

-  ``snapshot_freq`` :raw-html:`<a id="snapshot_freq" title="Permalink to this parameter" href="#snapshot_freq">&#x1F517;&#xFE0E;</a>`, default = ``-1``, type = int, aliases: ``save_period``

   -  frequency of saving model file snapshot

   -  set this to positive value to enable this function. For example, the model file will be snapshotted at each iteration if ``snapshot_freq=1``

   -  **Note**: can be used only in CLI version

IO Parameters
-------------

Dataset Parameters
~~~~~~~~~~~~~~~~~~

599
-  ``max_bin`` :raw-html:`<a id="max_bin" title="Permalink to this parameter" href="#max_bin">&#x1F517;&#xFE0E;</a>`, default = ``255``, type = int, constraints: ``max_bin > 1``
600
601
602
603
604
605
606

   -  max number of bins that feature values will be bucketed in

   -  small number of bins may reduce training accuracy but may increase general power (deal with over-fitting)

   -  LightGBM will auto compress memory according to ``max_bin``. For example, LightGBM will use ``uint8_t`` for feature value if ``max_bin=255``

Belinda Trotta's avatar
Belinda Trotta committed
607
608
609
610
611
612
-  ``max_bin_by_feature`` :raw-html:`<a id="max_bin_by_feature" title="Permalink to this parameter" href="#max_bin_by_feature">&#x1F517;&#xFE0E;</a>`, default = ``None``, type = multi-int

   -  max number of bins for each feature

   -  if not specified, will use ``max_bin`` for all features

613
-  ``min_data_in_bin`` :raw-html:`<a id="min_data_in_bin" title="Permalink to this parameter" href="#min_data_in_bin">&#x1F517;&#xFE0E;</a>`, default = ``3``, type = int, constraints: ``min_data_in_bin > 0``
614
615
616
617

   -  minimal number of data inside one bin

   -  use this to avoid one-data-one-bin (potential over-fitting)
618

619
-  ``bin_construct_sample_cnt`` :raw-html:`<a id="bin_construct_sample_cnt" title="Permalink to this parameter" href="#bin_construct_sample_cnt">&#x1F517;&#xFE0E;</a>`, default = ``200000``, type = int, aliases: ``subsample_for_bin``, constraints: ``bin_construct_sample_cnt > 0``
620

621
622
623
624
625
626
   -  number of data that sampled to construct histogram bins

   -  setting this to larger value will give better training result, but will increase data loading time

   -  set this to larger value if data is very sparse

627
-  ``data_random_seed`` :raw-html:`<a id="data_random_seed" title="Permalink to this parameter" href="#data_random_seed">&#x1F517;&#xFE0E;</a>`, default = ``1``, type = int, aliases: ``data_seed``
628

629
   -  random seed for sampling data to construct histogram bins
630

631
-  ``is_enable_sparse`` :raw-html:`<a id="is_enable_sparse" title="Permalink to this parameter" href="#is_enable_sparse">&#x1F517;&#xFE0E;</a>`, default = ``true``, type = bool, aliases: ``is_sparse``, ``enable_sparse``, ``sparse``
632

633
   -  used to enable/disable sparse optimization
634

635
-  ``enable_bundle`` :raw-html:`<a id="enable_bundle" title="Permalink to this parameter" href="#enable_bundle">&#x1F517;&#xFE0E;</a>`, default = ``true``, type = bool, aliases: ``is_enable_bundle``, ``bundle``
636

637
   -  set this to ``false`` to disable Exclusive Feature Bundling (EFB), which is described in `LightGBM: A Highly Efficient Gradient Boosting Decision Tree <https://papers.nips.cc/paper/6907-lightgbm-a-highly-efficient-gradient-boosting-decision-tree>`__
638

639
   -  **Note**: disabling this may cause the slow training speed for sparse datasets
640

641
-  ``use_missing`` :raw-html:`<a id="use_missing" title="Permalink to this parameter" href="#use_missing">&#x1F517;&#xFE0E;</a>`, default = ``true``, type = bool
642

643
   -  set this to ``false`` to disable the special handle of missing value
644

645
-  ``zero_as_missing`` :raw-html:`<a id="zero_as_missing" title="Permalink to this parameter" href="#zero_as_missing">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool
646

647
   -  set this to ``true`` to treat all zero as missing values (including the unshown values in LibSVM / sparse matrices)
648

649
   -  set this to ``false`` to use ``na`` for representing missing values
650

651
-  ``feature_pre_filter`` :raw-html:`<a id="feature_pre_filter" title="Permalink to this parameter" href="#feature_pre_filter">&#x1F517;&#xFE0E;</a>`, default = ``true``, type = bool
652

653
   -  set this to ``true`` to pre-filter the unsplittable features by ``min_data_in_leaf``
654

655
   -  as dataset object is initialized only once and cannot be changed after that, you may need to set this to ``false`` when searching parameters with ``min_data_in_leaf``, otherwise features are filtered by ``min_data_in_leaf`` firstly if you don't reconstruct dataset object
656

657
   -  **Note**: setting this to ``false`` may slow down the training
658

659
-  ``pre_partition`` :raw-html:`<a id="pre_partition" title="Permalink to this parameter" href="#pre_partition">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool, aliases: ``is_pre_partition``
660
661

   -  used for parallel learning (excluding the ``feature_parallel`` mode)
662
663
664

   -  ``true`` if training data are pre-partitioned, and different machines use different partitions

665
-  ``two_round`` :raw-html:`<a id="two_round" title="Permalink to this parameter" href="#two_round">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool, aliases: ``two_round_loading``, ``use_two_round_loading``
666
667
668

   -  set this to ``true`` if data file is too big to fit in memory

669
670
   -  by default, LightGBM will map data file to memory and load features from memory. This will provide faster data loading speed, but may cause run out of memory error when the data file is very big

671
672
   -  **Note**: works only in case of loading data directly from file

673
-  ``header`` :raw-html:`<a id="header" title="Permalink to this parameter" href="#header">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool, aliases: ``has_header``
674
675
676

   -  set this to ``true`` if input data has header

677
678
   -  **Note**: works only in case of loading data directly from file

679
-  ``label_column`` :raw-html:`<a id="label_column" title="Permalink to this parameter" href="#label_column">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = int or string, aliases: ``label``
680

681
   -  used to specify the label column
682
683
684
685
686

   -  use number for index, e.g. ``label=0`` means column\_0 is the label

   -  add a prefix ``name:`` for column name, e.g. ``label=name:is_click``

687
688
   -  **Note**: works only in case of loading data directly from file

689
-  ``weight_column`` :raw-html:`<a id="weight_column" title="Permalink to this parameter" href="#weight_column">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = int or string, aliases: ``weight``
690

691
   -  used to specify the weight column
692
693
694
695
696

   -  use number for index, e.g. ``weight=0`` means column\_0 is the weight

   -  add a prefix ``name:`` for column name, e.g. ``weight=name:weight``

697
698
   -  **Note**: works only in case of loading data directly from file

699
   -  **Note**: index starts from ``0`` and it doesn't count the label column when passing type is ``int``, e.g. when label is column\_0, and weight is column\_1, the correct parameter is ``weight=0``
700

701
-  ``group_column`` :raw-html:`<a id="group_column" title="Permalink to this parameter" href="#group_column">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = int or string, aliases: ``group``, ``group_id``, ``query_column``, ``query``, ``query_id``
702

703
   -  used to specify the query/group id column
704
705
706
707
708

   -  use number for index, e.g. ``query=0`` means column\_0 is the query id

   -  add a prefix ``name:`` for column name, e.g. ``query=name:query_id``

709
710
   -  **Note**: works only in case of loading data directly from file

711
   -  **Note**: data should be grouped by query\_id
712

713
   -  **Note**: index starts from ``0`` and it doesn't count the label column when passing type is ``int``, e.g. when label is column\_0 and query\_id is column\_1, the correct parameter is ``query=0``
714

715
-  ``ignore_column`` :raw-html:`<a id="ignore_column" title="Permalink to this parameter" href="#ignore_column">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = multi-int or string, aliases: ``ignore_feature``, ``blacklist``
716
717

   -  used to specify some ignoring columns in training
718
719
720
721
722

   -  use number for index, e.g. ``ignore_column=0,1,2`` means column\_0, column\_1 and column\_2 will be ignored

   -  add a prefix ``name:`` for column name, e.g. ``ignore_column=name:c1,c2,c3`` means c1, c2 and c3 will be ignored

723
   -  **Note**: works only in case of loading data directly from file
724

725
   -  **Note**: index starts from ``0`` and it doesn't count the label column when passing type is ``int``
726

727
728
   -  **Note**: despite the fact that specified columns will be completely ignored during the training, they still should have a valid format allowing LightGBM to load file successfully

729
-  ``categorical_feature`` :raw-html:`<a id="categorical_feature" title="Permalink to this parameter" href="#categorical_feature">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = multi-int or string, aliases: ``cat_feature``, ``categorical_column``, ``cat_column``
730

731
   -  used to specify categorical features
732
733
734
735
736

   -  use number for index, e.g. ``categorical_feature=0,1,2`` means column\_0, column\_1 and column\_2 are categorical features

   -  add a prefix ``name:`` for column name, e.g. ``categorical_feature=name:c1,c2,c3`` means c1, c2 and c3 are categorical features

737
   -  **Note**: only supports categorical with ``int`` type (not applicable for data represented as pandas DataFrame in Python-package)
738
739

   -  **Note**: index starts from ``0`` and it doesn't count the label column when passing type is ``int``
740

741
742
   -  **Note**: all values should be less than ``Int32.MaxValue`` (2147483647)

743
   -  **Note**: using large values could be memory consuming. Tree decision rule works best when categorical features are presented by consecutive integers starting from zero
744

745
   -  **Note**: all negative values will be treated as **missing values**
746

747
748
   -  **Note**: the output cannot be monotonically constrained with respect to a categorical feature

749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
-  ``forcedbins_filename`` :raw-html:`<a id="forcedbins_filename" title="Permalink to this parameter" href="#forcedbins_filename">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string

   -  path to a ``.json`` file that specifies bin upper bounds for some or all features

   -  ``.json`` file should contain an array of objects, each containing the word ``feature`` (integer feature index) and ``bin_upper_bound`` (array of thresholds for binning)

   -  see `this file <https://github.com/microsoft/LightGBM/tree/master/examples/regression/forced_bins.json>`__ as an example

-  ``save_binary`` :raw-html:`<a id="save_binary" title="Permalink to this parameter" href="#save_binary">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool, aliases: ``is_save_binary``, ``is_save_binary_file``

   -  if ``true``, LightGBM will save the dataset (including validation data) to a binary file. This speed ups the data loading for the next time

   -  **Note**: ``init_score`` is not saved in binary file

   -  **Note**: can be used only in CLI version; for language-specific packages you can use the correspondent function

Predict Parameters
~~~~~~~~~~~~~~~~~~

-  ``num_iteration_predict`` :raw-html:`<a id="num_iteration_predict" title="Permalink to this parameter" href="#num_iteration_predict">&#x1F517;&#xFE0E;</a>`, default = ``-1``, type = int

   -  used only in ``prediction`` task

   -  used to specify how many trained iterations will be used in prediction

   -  ``<= 0`` means no limit

776
-  ``predict_raw_score`` :raw-html:`<a id="predict_raw_score" title="Permalink to this parameter" href="#predict_raw_score">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool, aliases: ``is_predict_raw_score``, ``predict_rawscore``, ``raw_score``
777

778
   -  used only in ``prediction`` task
779

780
   -  set this to ``true`` to predict only the raw scores
781

782
   -  set this to ``false`` to predict transformed scores
783

784
-  ``predict_leaf_index`` :raw-html:`<a id="predict_leaf_index" title="Permalink to this parameter" href="#predict_leaf_index">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool, aliases: ``is_predict_leaf_index``, ``leaf_index``
785

786
   -  used only in ``prediction`` task
787

788
   -  set this to ``true`` to predict with leaf index of all trees
789

790
-  ``predict_contrib`` :raw-html:`<a id="predict_contrib" title="Permalink to this parameter" href="#predict_contrib">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool, aliases: ``is_predict_contrib``, ``contrib``
791

792
   -  used only in ``prediction`` task
793

794
   -  set this to ``true`` to estimate `SHAP values <https://arxiv.org/abs/1706.06060>`__, which represent how each feature contributes to each prediction
795

796
   -  produces ``#features + 1`` values where the last value is the expected value of the model output over the training data
797

798
799
   -  **Note**: if you want to get more explanation for your model's predictions using SHAP values like SHAP interaction values, you can install `shap package <https://github.com/slundberg/shap>`__

Nikita Titov's avatar
Nikita Titov committed
800
   -  **Note**: unlike the shap package, with ``predict_contrib`` we return a matrix with an extra column, where the last column is the expected value
801

802
-  ``predict_disable_shape_check`` :raw-html:`<a id="predict_disable_shape_check" title="Permalink to this parameter" href="#predict_disable_shape_check">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool
803

804
   -  used only in ``prediction`` task
805

806
   -  control whether or not LightGBM raises an error when you try to predict on data with a different number of features than the training data
807

808
809
810
811
812
   -  if ``false`` (the default), a fatal error will be raised if the number of features in the dataset you predict on differs from the number seen during training

   -  if ``true``, LightGBM will attempt to predict on whatever data you provide. This is dangerous because you might get incorrect predictions, but you could use it in situations where it is difficult or expensive to generate some features and you are very confident that they were never chosen for splits in the model

   -  **Note**: be very careful setting this parameter to ``true``
813

814
-  ``pred_early_stop`` :raw-html:`<a id="pred_early_stop" title="Permalink to this parameter" href="#pred_early_stop">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool
815

816
   -  used only in ``prediction`` task
817

818
   -  if ``true``, will use early-stopping to speed up the prediction. May affect the accuracy
819

820
-  ``pred_early_stop_freq`` :raw-html:`<a id="pred_early_stop_freq" title="Permalink to this parameter" href="#pred_early_stop_freq">&#x1F517;&#xFE0E;</a>`, default = ``10``, type = int
821

822
   -  used only in ``prediction`` task
823
824
825

   -  the frequency of checking early-stopping prediction

826
-  ``pred_early_stop_margin`` :raw-html:`<a id="pred_early_stop_margin" title="Permalink to this parameter" href="#pred_early_stop_margin">&#x1F517;&#xFE0E;</a>`, default = ``10.0``, type = double
827
828

   -  used only in ``prediction`` task
829
830
831

   -  the threshold of margin in early-stopping prediction

832
-  ``output_result`` :raw-html:`<a id="output_result" title="Permalink to this parameter" href="#output_result">&#x1F517;&#xFE0E;</a>`, default = ``LightGBM_predict_result.txt``, type = string, aliases: ``predict_result``, ``prediction_result``, ``predict_name``, ``prediction_name``, ``pred_name``, ``name_pred``
833
834
835

   -  used only in ``prediction`` task

836
   -  filename of prediction result
837

838
   -  **Note**: can be used only in CLI version
839

840
841
Convert Parameters
~~~~~~~~~~~~~~~~~~
842

843
-  ``convert_model_language`` :raw-html:`<a id="convert_model_language" title="Permalink to this parameter" href="#convert_model_language">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string
844

845
   -  used only in ``convert_model`` task
846

847
   -  only ``cpp`` is supported yet; for conversion model to other languages consider using `m2cgen <https://github.com/BayesWitnesses/m2cgen>`__ utility
848

849
   -  if ``convert_model_language`` is set and ``task=train``, the model will be also converted
850

851
852
   -  **Note**: can be used only in CLI version

853
-  ``convert_model`` :raw-html:`<a id="convert_model" title="Permalink to this parameter" href="#convert_model">&#x1F517;&#xFE0E;</a>`, default = ``gbdt_prediction.cpp``, type = string, aliases: ``convert_model_file``
854

855
   -  used only in ``convert_model`` task
856

857
   -  output filename of converted model
858

859
860
   -  **Note**: can be used only in CLI version

861
862
Objective Parameters
--------------------
863

864
865
-  ``objective_seed`` :raw-html:`<a id="objective_seed" title="Permalink to this parameter" href="#objective_seed">&#x1F517;&#xFE0E;</a>`, default = ``5``, type = int

866
   -  used only in ``rank_xendcg`` objective
867

868
   -  random seed for objectives, if random process is needed
869

870
-  ``num_class`` :raw-html:`<a id="num_class" title="Permalink to this parameter" href="#num_class">&#x1F517;&#xFE0E;</a>`, default = ``1``, type = int, aliases: ``num_classes``, constraints: ``num_class > 0``
871

872
   -  used only in ``multi-class`` classification application
873

874
-  ``is_unbalance`` :raw-html:`<a id="is_unbalance" title="Permalink to this parameter" href="#is_unbalance">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool, aliases: ``unbalance``, ``unbalanced_sets``
875

876
   -  used only in ``binary`` and ``multiclassova`` applications
877

878
   -  set this to ``true`` if training data are unbalanced
879

880
881
   -  **Note**: while enabling this should increase the overall performance metric of your model, it will also result in poor estimates of the individual class probabilities

882
   -  **Note**: this parameter cannot be used at the same time with ``scale_pos_weight``, choose only **one** of them
883

884
-  ``scale_pos_weight`` :raw-html:`<a id="scale_pos_weight" title="Permalink to this parameter" href="#scale_pos_weight">&#x1F517;&#xFE0E;</a>`, default = ``1.0``, type = double, constraints: ``scale_pos_weight > 0.0``
885

886
   -  used only in ``binary`` and ``multiclassova`` applications
887

888
   -  weight of labels with positive class
889

890
891
   -  **Note**: while enabling this should increase the overall performance metric of your model, it will also result in poor estimates of the individual class probabilities

892
   -  **Note**: this parameter cannot be used at the same time with ``is_unbalance``, choose only **one** of them
893

894
-  ``sigmoid`` :raw-html:`<a id="sigmoid" title="Permalink to this parameter" href="#sigmoid">&#x1F517;&#xFE0E;</a>`, default = ``1.0``, type = double, constraints: ``sigmoid > 0.0``
895

896
   -  used only in ``binary`` and ``multiclassova`` classification and in ``lambdarank`` applications
897

898
   -  parameter for the sigmoid function
899

900
-  ``boost_from_average`` :raw-html:`<a id="boost_from_average" title="Permalink to this parameter" href="#boost_from_average">&#x1F517;&#xFE0E;</a>`, default = ``true``, type = bool
901

902
   -  used only in ``regression``, ``binary``, ``multiclassova`` and ``cross-entropy`` applications
903

904
   -  adjusts initial score to the mean of labels for faster convergence
905

906
-  ``reg_sqrt`` :raw-html:`<a id="reg_sqrt" title="Permalink to this parameter" href="#reg_sqrt">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool
907

908
   -  used only in ``regression`` application
909

910
   -  used to fit ``sqrt(label)`` instead of original values and prediction result will be also automatically converted to ``prediction^2``
911

912
   -  might be useful in case of large-range labels
913

914
-  ``alpha`` :raw-html:`<a id="alpha" title="Permalink to this parameter" href="#alpha">&#x1F517;&#xFE0E;</a>`, default = ``0.9``, type = double, constraints: ``alpha > 0.0``
915

916
   -  used only in ``huber`` and ``quantile`` ``regression`` applications
917

918
   -  parameter for `Huber loss <https://en.wikipedia.org/wiki/Huber_loss>`__ and `Quantile regression <https://en.wikipedia.org/wiki/Quantile_regression>`__
919

920
-  ``fair_c`` :raw-html:`<a id="fair_c" title="Permalink to this parameter" href="#fair_c">&#x1F517;&#xFE0E;</a>`, default = ``1.0``, type = double, constraints: ``fair_c > 0.0``
921

922
   -  used only in ``fair`` ``regression`` application
923

924
   -  parameter for `Fair loss <https://www.kaggle.com/c/allstate-claims-severity/discussion/24520>`__
925

926
-  ``poisson_max_delta_step`` :raw-html:`<a id="poisson_max_delta_step" title="Permalink to this parameter" href="#poisson_max_delta_step">&#x1F517;&#xFE0E;</a>`, default = ``0.7``, type = double, constraints: ``poisson_max_delta_step > 0.0``
927

928
   -  used only in ``poisson`` ``regression`` application
929

930
931
   -  parameter for `Poisson regression <https://en.wikipedia.org/wiki/Poisson_regression>`__ to safeguard optimization

932
-  ``tweedie_variance_power`` :raw-html:`<a id="tweedie_variance_power" title="Permalink to this parameter" href="#tweedie_variance_power">&#x1F517;&#xFE0E;</a>`, default = ``1.5``, type = double, constraints: ``1.0 <= tweedie_variance_power < 2.0``
933
934
935
936
937
938

   -  used only in ``tweedie`` ``regression`` application

   -  used to control the variance of the tweedie distribution

   -  set this closer to ``2`` to shift towards a **Gamma** distribution
939

940
   -  set this closer to ``1`` to shift towards a **Poisson** distribution
941

942
-  ``lambdarank_truncation_level`` :raw-html:`<a id="lambdarank_truncation_level" title="Permalink to this parameter" href="#lambdarank_truncation_level">&#x1F517;&#xFE0E;</a>`, default = ``20``, type = int, constraints: ``lambdarank_truncation_level > 0``
943

944
   -  used only in ``lambdarank`` application
945

946
   -  used for truncating the max DCG, refer to "truncation level" in the Sec. 3 of `LambdaMART paper <https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/MSR-TR-2010-82.pdf>`__
947

948
-  ``lambdarank_norm`` :raw-html:`<a id="lambdarank_norm" title="Permalink to this parameter" href="#lambdarank_norm">&#x1F517;&#xFE0E;</a>`, default = ``true``, type = bool
949
950
951
952
953

   -  used only in ``lambdarank`` application

   -  set this to ``true`` to normalize the lambdas for different queries, and improve the performance for unbalanced data

954
   -  set this to ``false`` to enforce the original lambdarank algorithm
955

956
-  ``label_gain`` :raw-html:`<a id="label_gain" title="Permalink to this parameter" href="#label_gain">&#x1F517;&#xFE0E;</a>`, default = ``0,1,3,7,15,31,63,...,2^30-1``, type = multi-double
957

958
   -  used only in ``lambdarank`` application
Nikita Titov's avatar
Nikita Titov committed
959

960
   -  relevant gain for labels. For example, the gain of label ``2`` is ``3`` in case of default label gains
Nikita Titov's avatar
Nikita Titov committed
961

962
   -  separate by ``,``
Guolin Ke's avatar
Guolin Ke committed
963

964
965
966
Metric Parameters
-----------------

967
-  ``metric`` :raw-html:`<a id="metric" title="Permalink to this parameter" href="#metric">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = multi-enum, aliases: ``metrics``, ``metric_types``
968

969
   -  metric(s) to be evaluated on the evaluation set(s)
970

971
      -  ``""`` (empty string or not specified) means that metric corresponding to specified ``objective`` will be used (this is possible only for pre-defined objective functions, otherwise no evaluation metric will be added)
972

973
      -  ``"None"`` (string, **not** a ``None`` value) means that no metric will be registered, aliases: ``na``, ``null``, ``custom``
974
975
976
977
978

      -  ``l1``, absolute loss, aliases: ``mean_absolute_error``, ``mae``, ``regression_l1``

      -  ``l2``, square loss, aliases: ``mean_squared_error``, ``mse``, ``regression_l2``, ``regression``

979
      -  ``rmse``, root square loss, aliases: ``root_mean_squared_error``, ``l2_root``
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996

      -  ``quantile``, `Quantile regression <https://en.wikipedia.org/wiki/Quantile_regression>`__

      -  ``mape``, `MAPE loss <https://en.wikipedia.org/wiki/Mean_absolute_percentage_error>`__, aliases: ``mean_absolute_percentage_error``

      -  ``huber``, `Huber loss <https://en.wikipedia.org/wiki/Huber_loss>`__

      -  ``fair``, `Fair loss <https://www.kaggle.com/c/allstate-claims-severity/discussion/24520>`__

      -  ``poisson``, negative log-likelihood for `Poisson regression <https://en.wikipedia.org/wiki/Poisson_regression>`__

      -  ``gamma``, negative log-likelihood for **Gamma** regression

      -  ``gamma_deviance``, residual deviance for **Gamma** regression

      -  ``tweedie``, negative log-likelihood for **Tweedie** regression

997
      -  ``ndcg``, `NDCG <https://en.wikipedia.org/wiki/Discounted_cumulative_gain#Normalized_DCG>`__, aliases: ``lambdarank``, ``rank_xendcg``, ``xendcg``, ``xe_ndcg``, ``xe_ndcg_mart``, ``xendcg_mart``
998
999
1000
1001
1002
1003
1004

      -  ``map``, `MAP <https://makarandtapaswi.wordpress.com/2012/07/02/intuition-behind-average-precision-and-map/>`__, aliases: ``mean_average_precision``

      -  ``auc``, `AUC <https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve>`__

      -  ``binary_logloss``, `log loss <https://en.wikipedia.org/wiki/Cross_entropy>`__, aliases: ``binary``

Misha Lisovyi's avatar
Misha Lisovyi committed
1005
      -  ``binary_error``, for one sample: ``0`` for correct classification, ``1`` for error classification
1006

Belinda Trotta's avatar
Belinda Trotta committed
1007
1008
      -  ``auc_mu``, `AUC-mu <http://proceedings.mlr.press/v97/kleiman19a/kleiman19a.pdf>`__

1009
1010
1011
1012
      -  ``multi_logloss``, log loss for multi-class classification, aliases: ``multiclass``, ``softmax``, ``multiclassova``, ``multiclass_ova``, ``ova``, ``ovr``

      -  ``multi_error``, error rate for multi-class classification

Guolin Ke's avatar
Guolin Ke committed
1013
      -  ``cross_entropy``, cross-entropy (with optional linear weights), aliases: ``xentropy``
1014

Guolin Ke's avatar
Guolin Ke committed
1015
      -  ``cross_entropy_lambda``, "intensity-weighted" cross-entropy, aliases: ``xentlambda``
1016

Guolin Ke's avatar
Guolin Ke committed
1017
      -  ``kullback_leibler``, `Kullback-Leibler divergence <https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence>`__, aliases: ``kldiv``
1018

Misha Lisovyi's avatar
Misha Lisovyi committed
1019
   -  support multiple metrics, separated by ``,``
1020

1021
-  ``metric_freq`` :raw-html:`<a id="metric_freq" title="Permalink to this parameter" href="#metric_freq">&#x1F517;&#xFE0E;</a>`, default = ``1``, type = int, aliases: ``output_freq``, constraints: ``metric_freq > 0``
1022
1023
1024

   -  frequency for metric output

1025
1026
   -  **Note**: can be used only in CLI version

1027
-  ``is_provide_training_metric`` :raw-html:`<a id="is_provide_training_metric" title="Permalink to this parameter" href="#is_provide_training_metric">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool, aliases: ``training_metric``, ``is_training_metric``, ``train_metric``
1028

1029
   -  set this to ``true`` to output metric result over training dataset
1030

1031
1032
   -  **Note**: can be used only in CLI version

1033
-  ``eval_at`` :raw-html:`<a id="eval_at" title="Permalink to this parameter" href="#eval_at">&#x1F517;&#xFE0E;</a>`, default = ``1,2,3,4,5``, type = multi-int, aliases: ``ndcg_eval_at``, ``ndcg_at``, ``map_eval_at``, ``map_at``
1034

1035
1036
   -  used only with ``ndcg`` and ``map`` metrics

1037
   -  `NDCG <https://en.wikipedia.org/wiki/Discounted_cumulative_gain#Normalized_DCG>`__ and `MAP <https://makarandtapaswi.wordpress.com/2012/07/02/intuition-behind-average-precision-and-map/>`__ evaluation positions, separated by ``,``
1038

Belinda Trotta's avatar
Belinda Trotta committed
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
-  ``multi_error_top_k`` :raw-html:`<a id="multi_error_top_k" title="Permalink to this parameter" href="#multi_error_top_k">&#x1F517;&#xFE0E;</a>`, default = ``1``, type = int, constraints: ``multi_error_top_k > 0``

   -  used only with ``multi_error`` metric

   -  threshold for top-k multi-error metric

   -  the error on each sample is ``0`` if the true class is among the top ``multi_error_top_k`` predictions, and ``1`` otherwise

      -  more precisely, the error on a sample is ``0`` if there are at least ``num_classes - multi_error_top_k`` predictions strictly less than the prediction on the true class

   -  when ``multi_error_top_k=1`` this is equivalent to the usual multi-error metric

Belinda Trotta's avatar
Belinda Trotta committed
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
-  ``auc_mu_weights`` :raw-html:`<a id="auc_mu_weights" title="Permalink to this parameter" href="#auc_mu_weights">&#x1F517;&#xFE0E;</a>`, default = ``None``, type = multi-double

   -  used only with ``auc_mu`` metric

   -  list representing flattened matrix (in row-major order) giving loss weights for classification errors

   -  list should have ``n * n`` elements, where ``n`` is the number of classes

   -  the matrix co-ordinate ``[i, j]`` should correspond to the ``i * n + j``-th element of the list

   -  if not specified, will use equal weights for all classes

1063
1064
1065
Network Parameters
------------------

1066
-  ``num_machines`` :raw-html:`<a id="num_machines" title="Permalink to this parameter" href="#num_machines">&#x1F517;&#xFE0E;</a>`, default = ``1``, type = int, aliases: ``num_machine``, constraints: ``num_machines > 0``
1067

1068
   -  the number of machines for parallel learning application
1069

1070
   -  this parameter is needed to be set in both **socket** and **mpi** versions
1071

1072
-  ``local_listen_port`` :raw-html:`<a id="local_listen_port" title="Permalink to this parameter" href="#local_listen_port">&#x1F517;&#xFE0E;</a>`, default = ``12400``, type = int, aliases: ``local_port``, ``port``, constraints: ``local_listen_port > 0``
1073
1074
1075

   -  TCP listen port for local machines

1076
   -  **Note**: don't forget to allow this port in firewall settings before training
1077

1078
-  ``time_out`` :raw-html:`<a id="time_out" title="Permalink to this parameter" href="#time_out">&#x1F517;&#xFE0E;</a>`, default = ``120``, type = int, constraints: ``time_out > 0``
1079
1080
1081

   -  socket time-out in minutes

1082
-  ``machine_list_filename`` :raw-html:`<a id="machine_list_filename" title="Permalink to this parameter" href="#machine_list_filename">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string, aliases: ``machine_list_file``, ``machine_list``, ``mlist``
1083
1084

   -  path of file that lists machines for this parallel learning application
1085

1086
   -  each line contains one IP and one port for one machine. The format is ``ip port`` (space as a separator)
1087

1088
-  ``machines`` :raw-html:`<a id="machines" title="Permalink to this parameter" href="#machines">&#x1F517;&#xFE0E;</a>`, default = ``""``, type = string, aliases: ``workers``, ``nodes``
1089
1090

   -  list of machines in the following format: ``ip1:port1,ip2:port2``
1091
1092
1093
1094

GPU Parameters
--------------

1095
-  ``gpu_platform_id`` :raw-html:`<a id="gpu_platform_id" title="Permalink to this parameter" href="#gpu_platform_id">&#x1F517;&#xFE0E;</a>`, default = ``-1``, type = int
1096

1097
   -  OpenCL platform ID. Usually each GPU vendor exposes one OpenCL platform
1098

1099
   -  ``-1`` means the system-wide default platform
1100

1101
1102
   -  **Note**: refer to `GPU Targets <./GPU-Targets.rst#query-opencl-devices-in-your-system>`__ for more details

1103
-  ``gpu_device_id`` :raw-html:`<a id="gpu_device_id" title="Permalink to this parameter" href="#gpu_device_id">&#x1F517;&#xFE0E;</a>`, default = ``-1``, type = int
1104
1105
1106

   -  OpenCL device ID in the specified platform. Each GPU in the selected platform has a unique device ID

1107
   -  ``-1`` means the default device in the selected platform
1108

1109
1110
   -  **Note**: refer to `GPU Targets <./GPU-Targets.rst#query-opencl-devices-in-your-system>`__ for more details

1111
-  ``gpu_use_dp`` :raw-html:`<a id="gpu_use_dp" title="Permalink to this parameter" href="#gpu_use_dp">&#x1F517;&#xFE0E;</a>`, default = ``false``, type = bool
1112

1113
   -  set this to ``true`` to use double precision math on GPU (by default single precision is used)
1114

1115
1116
.. end params list

1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
Others
------

Continued Training with Input Score
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

LightGBM supports continued training with initial scores. It uses an additional file to store these initial scores, like the following:

::

    0.5
    -0.1
    0.9
    ...

It means the initial score of the first data row is ``0.5``, second is ``-0.1``, and so on.
The initial score file corresponds with data file line by line, and has per score per line.
1134

1135
And if the name of data file is ``train.txt``, the initial score file should be named as ``train.txt.init`` and placed in the same folder as the data file.
1136
In this case, LightGBM will auto load initial score file if it exists.
1137
1138
1139
1140

Weight Data
~~~~~~~~~~~

Nikita Titov's avatar
Nikita Titov committed
1141
LightGBM supports weighted training. It uses an additional file to store weight data, like the following:
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151

::

    1.0
    0.5
    0.8
    ...

It means the weight of the first data row is ``1.0``, second is ``0.5``, and so on.
The weight file corresponds with data file line by line, and has per weight per line.
1152

1153
And if the name of data file is ``train.txt``, the weight file should be named as ``train.txt.weight`` and placed in the same folder as the data file.
1154
In this case, LightGBM will load the weight file automatically if it exists.
1155

1156
Also, you can include weight column in your data file. Please refer to the ``weight_column`` `parameter <#weight_column>`__ in above.
1157
1158
1159
1160

Query Data
~~~~~~~~~~

1161
For learning to rank, it needs query information for training data.
Nikita Titov's avatar
Nikita Titov committed
1162
LightGBM uses an additional file to store query data, like the following:
1163
1164
1165
1166
1167
1168
1169
1170

::

    27
    18
    67
    ...

1171
It means first ``27`` lines samples belong to one query and next ``18`` lines belong to another, and so on.
1172
1173
1174

**Note**: data should be ordered by the query.

1175
If the name of data file is ``train.txt``, the query file should be named as ``train.txt.query`` and placed in the same folder as the data file.
1176
In this case, LightGBM will load the query file automatically if it exists.
1177

1178
Also, you can include query/group id column in your data file. Please refer to the ``group_column`` `parameter <#group_column>`__ in above.
1179
1180

.. _Laurae++ Interactive Documentation: https://sites.google.com/view/lauraepp/parameters