Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
tianlh
LightGBM-DCU
Commits
39e47323
Commit
39e47323
authored
Oct 19, 2016
by
Qiwei Ye
Browse files
updating comments for easy read
parent
70873a98
Changes
13
Hide whitespace changes
Inline
Side-by-side
Showing
13 changed files
with
79 additions
and
77 deletions
+79
-77
include/LightGBM/bin.h
include/LightGBM/bin.h
+1
-1
include/LightGBM/boosting.h
include/LightGBM/boosting.h
+3
-3
include/LightGBM/dataset.h
include/LightGBM/dataset.h
+9
-9
include/LightGBM/feature.h
include/LightGBM/feature.h
+1
-1
include/LightGBM/metric.h
include/LightGBM/metric.h
+3
-3
include/LightGBM/network.h
include/LightGBM/network.h
+16
-13
include/LightGBM/objective_function.h
include/LightGBM/objective_function.h
+3
-4
include/LightGBM/tree.h
include/LightGBM/tree.h
+11
-10
include/LightGBM/tree_learner.h
include/LightGBM/tree_learner.h
+3
-4
src/application/predictor.hpp
src/application/predictor.hpp
+6
-6
src/boosting/gbdt.h
src/boosting/gbdt.h
+10
-10
src/boosting/score_updater.hpp
src/boosting/score_updater.hpp
+7
-7
src/io/ordered_sparse_bin.hpp
src/io/ordered_sparse_bin.hpp
+6
-6
No files found.
include/LightGBM/bin.h
View file @
39e47323
...
...
@@ -291,7 +291,7 @@ public:
};
inline
unsigned
int
BinMapper
::
ValueToBin
(
double
value
)
const
{
//
use
binary search to find bin
// binary search to find bin
int
l
=
0
;
int
r
=
num_bin_
-
1
;
while
(
l
<
r
)
{
...
...
include/LightGBM/boosting.h
View file @
39e47323
...
...
@@ -23,8 +23,8 @@ public:
virtual
~
Boosting
()
{}
/*!
* \brief Initial logic
* \param config Config for boosting
* \brief Initial
ization
logic
* \param config Config
s
for boosting
* \param train_data Training data
* \param object_function Training objective function
* \param training_metrics Training metric
...
...
@@ -54,7 +54,7 @@ public:
virtual
double
PredictRaw
(
const
double
*
feature_values
)
const
=
0
;
/*!
* \brief Prediction for one record,
will use
sigmoid transform if needed
* \brief Prediction for one record, sigmoid transform
ation will be used
if needed
* \param feature_values Feature value on this record
* \return Prediction result for this record
*/
...
...
include/LightGBM/dataset.h
View file @
39e47323
...
...
@@ -20,14 +20,14 @@ class Feature;
* \brief This class is used to store some meta(non-feature) data for training data,
* e.g. labels, weights, initial scores, qurey level informations.
*
* Some details:
* 1. Label, used for traning.
* 2. Weights, weighs of record, optional
* 3. Query Boundaries, necessary for lambdarank.
* The documents of i-th query is in [ query_boundarise[i], query_boundarise[i+1] )
* 4. Query Weights, auto calculate by weights and query_boundarise(if both of them are existed)
* the weight for i-th query is sum(query_boundarise[i] , .., query_boundarise[i+1]) / (query_boundarise[i + 1] - query_boundarise[i+1])
* 5. Initial score. optional. if exsitng, the model will boost from this score, otherwise will start from 0.
*
Some details:
*
1. Label, used for traning.
*
2. Weights, weighs of record
s
, optional
*
3. Query Boundaries, necessary for lambdarank.
*
The documents of i-th query is in [ query_boundarise[i], query_boundarise[i+1] )
*
4. Query Weights, auto calculate by weights and query_boundarise(if both of them are existed)
*
the weight for i-th query is sum(query_boundarise[i] , .., query_boundarise[i+1]) / (query_boundarise[i + 1] - query_boundarise[i+1])
*
5. Initial score. optional. if exsitng, the model will boost from this score, otherwise will start from 0.
*/
class
Metadata
{
public:
...
...
@@ -36,7 +36,7 @@ public:
*/
Metadata
();
/*!
* \brief Initializ
e,
will load qurey level informations, since it is need for sampling data
* \brief Initializ
ation
will load qurey level informations, since it is need for sampling data
* \param data_filename Filename of data
* \param init_score_filename Filename of initial score
* \param is_int_label True if label is int type
...
...
include/LightGBM/feature.h
View file @
39e47323
...
...
@@ -12,7 +12,7 @@
namespace
LightGBM
{
/*! \brief Us
ed
to store data and provid
e
some operations on one feature*/
/*! \brief Us
ing
to store data and provid
ing
some operations on one feature*/
class
Feature
{
public:
/*!
...
...
include/LightGBM/metric.h
View file @
39e47323
...
...
@@ -28,7 +28,7 @@ public:
const
Metadata
&
metadata
,
data_size_t
num_data
)
=
0
;
/*!
* \brief Calcalut
e
and print metric result
* \brief Calcalut
ing
and print
ing
metric result
* \param iter Current iteration
* \param score Current prediction score
*/
...
...
@@ -55,7 +55,7 @@ public:
/*!
* \brief Calculate the DCG score at position k
* \param k The position
want
to eval
at
* \param k The position to eval
u
at
e
* \param label Pointer of label
* \param score Pointer of score
* \param num_data Number of data
...
...
@@ -66,7 +66,7 @@ public:
/*!
* \brief Calculate the DCG score at multi position
* \param ks The positions
want
to eval
at
* \param ks The positions to eval
u
at
e
* \param label Pointer of label
* \param score Pointer of score
* \param num_data Number of data
...
...
include/LightGBM/network.h
View file @
39e47323
...
...
@@ -14,7 +14,7 @@ namespace LightGBM {
/*! \brief forward declaration */
class
Linkers
;
/*! \brief The network structure for all
gather */
/*! \brief The network structure for all
_
gather */
class
BruckMap
{
public:
/*! \brief The communication times for one all gather operation */
...
...
@@ -37,11 +37,11 @@ public:
/*!
* \brief node type on recursive halving algorithm
* When number of machines is not power of 2, need group machines into power of 2 group.
* And we can let each group has at most 2 machines.
* if the group only has 1 machine. this machine is the normal node
* if the grou has 2 machines, this group will have two type of nodes, one is the leader.
* leader will represent this group and communication with others.
*
When number of machines is not power of 2, need group machines into power of 2 group.
*
And we can let each group has at most 2 machines.
*
if the group only has 1 machine. this machine is the normal node
*
if the grou has 2 machines, this group will have two type of nodes, one is the leader.
*
leader will represent this group and communication with others.
*/
enum
RecursiveHalvingNodeType
{
Normal
,
// normal node, 1 group only have 1 machine
...
...
@@ -98,7 +98,7 @@ public:
static
inline
int
num_machines
();
/*!
* \brief Perform all
reduce. if data size is small,
* \brief Perform all
_
reduce. if data size is small,
will perform AllreduceByAllGather, else with call ReduceScatter followed allgather
* \param input Input data
* \param input_size The size of input data
...
...
@@ -110,7 +110,7 @@ public:
char
*
output
,
const
ReduceFunction
&
reducer
);
/*!
* \brief Perform all
reduce
, use
all
gather.
When data is small, can use this
to reduce communication time
s
* \brief Perform all
_
reduce
by using
all
_
gather.
it can be use
to reduce communication time
when data is small
* \param input Input data
* \param input_size The size of input data
* \param output Output result
...
...
@@ -120,8 +120,9 @@ public:
const
ReduceFunction
&
reducer
);
/*!
* \brief Perform all gather, use bruck algorithm. Communication times is O(log(n)), and communication cost is O(send_size * number_machine)
* if all machine have same input size, can call this function
* \brief Performing all_gather by using bruck algorithm.
Communication times is O(log(n)), and communication cost is O(send_size * number_machine)
* It can be used when all nodes have same input size.
* \param input Input data
* \param send_size The size of input data
* \param output Output result
...
...
@@ -129,8 +130,9 @@ public:
static
void
Allgather
(
char
*
input
,
int
send_size
,
char
*
output
);
/*!
* \brief Perform all gather, use bruck algorithm. Communication times is O(log(n)), and communication cost is O(all_size)
* if all machine have different input size, can call this function
* \brief Performing all_gather by using bruck algorithm.
Communication times is O(log(n)), and communication cost is O(all_size)
* It can be used when nodes have different input size.
* \param input Input data
* \param all_size The size of input data
* \param block_start The block start for different machines
...
...
@@ -141,7 +143,8 @@ public:
int
*
block_len
,
char
*
output
);
/*!
* \brief Perform reduce scatter, use recursive halving algorithm. Communication times is O(log(n)), and communication cost is O(input_size)
* \brief Perform reduce scatter by using recursive halving algorithm.
Communication times is O(log(n)), and communication cost is O(input_size)
* \param input Input data
* \param input_size The size of input data
* \param block_start The block start for different machines
...
...
include/LightGBM/objective_function.h
View file @
39e47323
...
...
@@ -9,7 +9,6 @@ namespace LightGBM {
/*!
* \brief The interface of Objective Function.
* Objective function is used to get gradients
*/
class
ObjectiveFunction
{
public:
...
...
@@ -24,8 +23,8 @@ public:
virtual
void
Init
(
const
Metadata
&
metadata
,
data_size_t
num_data
)
=
0
;
/*!
* \brief calculat
e
first order derivative of loss function
* \param score
Current
prediction score
* \brief calculat
ing
first order derivative of loss function
* \param score prediction score
in this round
* \gradients Output gradients
* \hessians Output hessians
*/
...
...
@@ -34,7 +33,7 @@ public:
/*!
* \brief Get sigmoid param for this objective if has.
* This function is used for prediction task, if has sigmoid param, the prediction value will be transform by sigmoid function.
*
This function is used for prediction task, if has sigmoid param, the prediction value will be transform by sigmoid function.
* \return Sigmoid param, if <=0.0 means don't use sigmoid transform on this objective.
*/
virtual
double
GetSigmoid
()
const
=
0
;
...
...
include/LightGBM/tree.h
View file @
39e47323
...
...
@@ -31,9 +31,9 @@ public:
~
Tree
();
/*!
* \brief
S
plit
a
tree leave
,
* \param leaf Index of leaf t
hat want to
split
* \param feature Index of feature
,
the converted index after remov
e
useless features
* \brief
Performing a s
plit
on
tree leave
s.
* \param leaf Index of leaf t
o be
split
* \param feature Index of feature
;
the converted index after remov
ing
useless features
* \param threshold Threshold(bin) of split
* \param real_feature Index of feature, the original index on data
* \param threshold_double Threshold on feature value
...
...
@@ -50,7 +50,7 @@ public:
inline
score_t
LeafOutput
(
int
leaf
)
const
{
return
leaf_value_
[
leaf
];
}
/*!
* \brief Add prediction of this tree model to score
* \brief Add
ing
prediction
value
of this tree model to score
s
* \param data The dataset
* \param num_data Number of total data
* \param score Will add prediction to score
...
...
@@ -59,18 +59,18 @@ public:
score_t
*
score
)
const
;
/*!
* \brief Add prediction of this tree model to score
* \brief Add
ing
prediction
value
of this tree model to score
se
* \param data The dataset
* \param used_data_indices Indices of used data
* \param num_data Number of total data
* \param score Will add prediction to score
*/
void
AddPredictionToScore
(
const
Dataset
*
data
,
const
data_size_t
*
used_data_indices
,
data_size_t
num_data
,
score_t
*
score
)
const
;
const
data_size_t
*
used_data_indices
,
data_size_t
num_data
,
score_t
*
score
)
const
;
/*!
* \brief Prediction
for
one record
* \brief Prediction
on
one record
* \param feature_values Feature value of this record
* \return Prediction result
*/
...
...
@@ -81,6 +81,7 @@ public:
/*!
* \brief Shrinkage for the tree's output
* shrinkage rate (a.k.a learning rate) is used to tune the traning process
* \param rate The factor of shrinkage
*/
inline
void
Shrinkage
(
double
rate
)
{
...
...
@@ -98,7 +99,7 @@ public:
Tree
(
const
Tree
&
)
=
delete
;
private:
/*!
* \brief Find leaf index
that this
record belongs
* \brief Find leaf index
of which
record belongs
by data
* \param data The dataset
* \param data_idx Index of record
* \return Leaf index
...
...
@@ -107,7 +108,7 @@ private:
data_size_t
data_idx
)
const
;
/*!
* \brief Find leaf index
that this
record belongs
* \brief Find leaf index
of which
record belongs
by features
* \param feature_values Feature value of this record
* \return Leaf index
*/
...
...
include/LightGBM/tree_learner.h
View file @
39e47323
...
...
@@ -22,14 +22,13 @@ public:
virtual
~
TreeLearner
()
{}
/*!
* \brief Init tree learner with training data
set and
tree
config
* \brief Init
ialize
tree learner with training dataset and config
s
* \param train_data The used training data
* \param tree_config The tree setting
*/
virtual
void
Init
(
const
Dataset
*
train_data
)
=
0
;
/*!
* \brief
fit
train
data set and return a trained tree
* \brief train
ing tree model on dataset
* \param gradients The first order gradients
* \param hessians The second order gradients
* \return A trained tree
...
...
@@ -45,7 +44,7 @@ public:
data_size_t
num_data
)
=
0
;
/*!
* \brief Us
e
last trained tree to predit
ion training
score
, and
add to out_score;
* \brief Us
ing
last trained tree to predi
c
t score
then
add
ing
to out_score;
* \param out_score output score
*/
virtual
void
AddPredictionToScore
(
score_t
*
out_score
)
const
=
0
;
...
...
src/application/predictor.hpp
View file @
39e47323
...
...
@@ -54,7 +54,7 @@ public:
}
/*!
* \brief prediction for one record, only raw result(
no
t sigmoid transform)
* \brief prediction for one record, only raw result(
withou
t sigmoid transform
ation
)
* \param features Feature for this record
* \return Prediction result
*/
...
...
@@ -68,13 +68,13 @@ public:
features_
[
tid
][
p
.
first
]
=
p
.
second
;
}
}
// get result without sigmoid transform
// get result without sigmoid transform
ation
return
boosting_
->
PredictRaw
(
features_
[
tid
]);
}
/*!
* \brief prediction for one record, will use sigmoid transform if needed(only
needs in
binary classification no
w
)
* \param features Feature f
or
this record
* \brief prediction for one record, will use sigmoid transform
ation
if needed(only
enabled for
binary classification no
e
)
* \param features Feature
o
f this record
* \return Prediction result
*/
double
PredictOneLine
(
const
std
::
vector
<
std
::
pair
<
int
,
double
>>&
features
)
{
...
...
@@ -91,7 +91,7 @@ public:
return
boosting_
->
Predict
(
features_
[
tid
]);
}
/*!
* \brief predicti
on for a
data,
and
sav
e
result
* \brief predicti
ng on
data,
then
sav
ing
result
to disk
* \param data_filename Filename of data
* \param has_label True if this data contains label
* \param result_filename Filename of output result
...
...
@@ -112,7 +112,7 @@ public:
Parser
*
parser
=
Parser
::
CreateParser
(
data_filename
,
num_features_
,
&
has_label
);
if
(
parser
==
nullptr
)
{
Log
::
Stderr
(
"
can regonise
input data format, filename %s"
,
data_filename
);
Log
::
Stderr
(
"
recongnizing
input data format
failed
, filename %s"
,
data_filename
);
}
// function for parse data
...
...
src/boosting/gbdt.h
View file @
39e47323
...
...
@@ -24,7 +24,7 @@ public:
*/
~
GBDT
();
/*!
* \brief Initial logic
* \brief Initial
ization
logic
* \param config Config for boosting
* \param train_data Training data
* \param object_function Training objective function
...
...
@@ -36,9 +36,9 @@ public:
const
char
*
output_model_filename
)
override
;
/*!
* \brief Add a validation data
* \param valid_data Validation data
* \param valid_metrics Metrics for validation data
* \brief Add
ing
a validation data
set
* \param valid_data Validation data
set
* \param valid_metrics Metrics for validation data
set
*/
void
AddDataset
(
const
Dataset
*
valid_data
,
const
std
::
vector
<
const
Metric
*>&
valid_metrics
)
override
;
...
...
@@ -47,14 +47,14 @@ public:
*/
void
Train
()
override
;
/*!
* \brief Predtion for one record
, not use sigmoid
* \brief Predtion for one record
without sigmoid transformation
* \param feature_values Feature value on this record
* \return Prediction result for this record
*/
double
PredictRaw
(
const
double
*
feature_values
)
const
override
;
/*!
* \brief Predtion for one record
,
wi
ll use
sigmoid transform
if need
ed
* \brief Predtion for one record wi
th
sigmoid transform
ation if enabl
ed
* \param feature_values Feature value on this record
* \return Prediction result for this record
*/
...
...
@@ -87,8 +87,8 @@ private:
*/
void
Bagging
(
int
iter
);
/*!
* \brief updat
e
score for out-of-bag data.
*
It is necessary for this
update
,
since we may re-bagging data on training
* \brief updat
ing
score for out-of-bag data.
*
Data should be
update since we may re-bagging data on training
* \param tree Trained tree of this iteration
*/
void
UpdateScoreOutOfBag
(
const
Tree
*
tree
);
...
...
@@ -97,12 +97,12 @@ private:
*/
void
Boosting
();
/*!
* \brief train one tree
* \brief train
ing
one tree
* \return Trained tree of this iteration
*/
Tree
*
TrainOneTree
();
/*!
* \brief updat
e
score after tree trained
* \brief updat
ing
score after tree
was
trained
* \param tree Trained tree of this iteration
*/
void
UpdateScore
(
const
Tree
*
tree
);
...
...
src/boosting/score_updater.hpp
View file @
39e47323
...
...
@@ -37,25 +37,25 @@ public:
delete
[]
score_
;
}
/*!
* \brief Us
e
tree model to get prediction, then add to score for all data
* Note: this function generally will be used
for
validation data.
* \brief Us
ing
tree model to get prediction
number
, then add
ing
to score
s
for all data
*
Note: this function generally will be used
on
validation data
too
.
* \param tree Trained tree model
*/
inline
void
AddScore
(
const
Tree
*
tree
)
{
tree
->
AddPredictionToScore
(
data_
,
num_data_
,
score_
);
}
/*!
* \brief Add prediction score, only used for training data.
*
After trained a tree, t
he training data is partitioned into tree leaves
.
*
We can get prediction by faster speed based on this
.
* \brief Add
ing
prediction score, only used for training data.
*
T
he training data is partitioned into tree leaves
after training
*
Based on which We can get prediction quckily
.
* \param tree_learner
*/
inline
void
AddScore
(
const
TreeLearner
*
tree_learner
)
{
tree_learner
->
AddPredictionToScore
(
score_
);
}
/*!
* \brief
Like AddScore(const Tree* tree), but only
for part of data
* Used for prediction of training out-of-ba
d
data
* \brief
Using tree model to get prediction number, then adding to scores
for part
s
of data
*
Used for prediction of training out-of-ba
g
data
* \param tree Trained tree model
* \param data_indices Indices of data that want proccess to
* \param data_cnt Number of data that want proccess to
...
...
src/io/ordered_sparse_bin.hpp
View file @
39e47323
...
...
@@ -13,12 +13,12 @@
namespace
LightGBM
{
/*!
* \brief
Ordered bin for sparse feature
. efficient for construct histogram, especally for sparse bin
* There are 2 advantages
for
using ordered bin.
* 1. group the data by leaf
,
improve the cache hit.
* 2. only store the non-zero bin, which can speed up the histogram
c
consturction for sparse feature.
*
But it has a
additional cost
,
it need re-order the bins after
leaf
split, which will cost much for dense feature.
* So we only us
e
ordered bin for sparse
features now
.
* \brief
Interface for ordered bin data
. efficient for construct histogram, espec
i
ally for sparse bin
*
There are 2 advantages
by
using ordered bin.
*
1. group the data by leaf
s to
improve the cache hit.
*
2. only store the non-zero bin, which can speed up the histogram consturction for sparse feature
s
.
*
However it brings
additional cost
:
it need re-order the bins after
every
split, which will cost much for dense feature.
*
So we only us
ing
ordered bin for sparse
situations
.
*/
template
<
typename
VAL_T
>
class
OrderedSparseBin
:
public
OrderedBin
{
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment