@@ -14,7 +14,7 @@ Instead, please report them to the Microsoft Security Response Center (MSRC) at
...
@@ -14,7 +14,7 @@ Instead, please report them to the Microsoft Security Response Center (MSRC) at
If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey).
If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey).
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc).
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc).
Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
@@ -113,8 +113,8 @@ Unlike a categorical feature, however, ``positions`` are used to adjust the targ
...
@@ -113,8 +113,8 @@ Unlike a categorical feature, however, ``positions`` are used to adjust the targ
The position file corresponds with training data file line by line, and has one position per line. And if the name of training data file is ``train.txt``, the position file should be named as ``train.txt.position`` and placed in the same folder as the data file.
The position file corresponds with training data file line by line, and has one position per line. And if the name of training data file is ``train.txt``, the position file should be named as ``train.txt.position`` and placed in the same folder as the data file.
In this case, LightGBM will load the position file automatically if it exists. The positions can also be specified through the ``Dataset`` constructor when using Python API. If the positions are specified in both approaches, the ``.position`` file will be ignored.
In this case, LightGBM will load the position file automatically if it exists. The positions can also be specified through the ``Dataset`` constructor when using Python API. If the positions are specified in both approaches, the ``.position`` file will be ignored.
Currently, implemented is an approach to model position bias by using an idea of Generalized Additive Models (`GAM <https://en.wikipedia.org/wiki/Generalized_additive_model>`_) to linearly decompose the document score ``s`` into the sum of a relevance component ``f`` and a positional component ``g``: ``s(x, pos) = f(x) + g(pos)`` where the former component depends on the original query-document features and the latter depends on the position of an item.
Currently, implemented is an approach to model position bias by using an idea of Generalized Additive Models (`GAM <https://en.wikipedia.org/wiki/Generalized_additive_model>`_) to linearly decompose the document score ``s`` into the sum of a relevance component ``f`` and a positional component ``g``: ``s(x, pos) = f(x) + g(pos)`` where the former component depends on the original query-document features and the latter depends on the position of an item.
During the training, the compound scoring function ``s(x, pos)`` is fit with a standard ranking algorithm (e.g., LambdaMART) which boils down to jointly learning the relevance component ``f(x)`` (it is later returned as an unbiased model) and the position factors ``g(pos)`` that help better explain the observed (biased) labels.
During the training, the compound scoring function ``s(x, pos)`` is fit with a standard ranking algorithm (e.g., LambdaMART) which boils down to jointly learning the relevance component ``f(x)`` (it is later returned as an unbiased model) and the position factors ``g(pos)`` that help better explain the observed (biased) labels.
Similar score decomposition ideas have previously been applied for classification & pointwise ranking tasks with assumptions of binary labels and binary relevance (a.k.a. "two-tower" models, refer to the papers: `Towards Disentangling Relevance and Bias in Unbiased Learning to Rank <https://arxiv.org/abs/2212.13937>`_, `PAL: a position-bias aware learning framework for CTR prediction in live recommender systems <https://dl.acm.org/doi/10.1145/3298689.3347033>`_, `A General Framework for Debiasing in CTR Prediction <https://arxiv.org/abs/2112.02767>`_).
Similar score decomposition ideas have previously been applied for classification & pointwise ranking tasks with assumptions of binary labels and binary relevance (a.k.a. "two-tower" models, refer to the papers: `Towards Disentangling Relevance and Bias in Unbiased Learning to Rank <https://arxiv.org/abs/2212.13937>`_, `PAL: a position-bias aware learning framework for CTR prediction in live recommender systems <https://dl.acm.org/doi/10.1145/3298689.3347033>`_, `A General Framework for Debiasing in CTR Prediction <https://arxiv.org/abs/2112.02767>`_).
In LightGBM, we adapt this idea to general pairwise Lerarning-to-Rank with arbitrary ordinal relevance labels.
In LightGBM, we adapt this idea to general pairwise Lerarning-to-Rank with arbitrary ordinal relevance labels.
Besides, GAMs have been used in the context of explainable ML (`Accurate Intelligible Models with Pairwise Interactions <https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf>`_) to linearly decompose the contribution of each feature (and possibly their pairwise interactions) to the overall score, for subsequent analysis and interpretation of their effects in the trained models.
Besides, GAMs have been used in the context of explainable ML (`Accurate Intelligible Models with Pairwise Interactions <https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf>`_) to linearly decompose the contribution of each feature (and possibly their pairwise interactions) to the overall score, for subsequent analysis and interpretation of their effects in the trained models.
- To get one leaf's histograms in a binary tree, use the histogram subtraction of its parent and its neighbor
- To get one leaf's histograms in a binary tree, use the histogram subtraction of its parent and its neighbor
- So it needs to construct histograms for only one leaf (with smaller ``#data`` than its neighbor). It then can get histograms of its neighbor by histogram subtraction with small cost (``O(#bins)``)
- So it needs to construct histograms for only one leaf (with smaller ``#data`` than its neighbor). It then can get histograms of its neighbor by histogram subtraction with small cost (``O(#bins)``)
- **Reduce memory usage**
- **Reduce memory usage**
- Replaces continuous values with discrete bins. If ``#bins`` is small, can use small data type, e.g. uint8\_t, to store training data
- Replaces continuous values with discrete bins. If ``#bins`` is small, can use small data type, e.g. uint8\_t, to store training data
<svgxmlns="http://www.w3.org/2000/svg"xmlns:xlink="http://www.w3.org/1999/xlink"width="302"height="20"><linearGradientid="b"x2="0"y2="100%"><stopoffset="0"stop-color="#bbb"stop-opacity=".1"/><stopoffset="1"stop-opacity=".1"/></linearGradient><clipPathid="a"><rectwidth="302"height="20"rx="3"fill="#fff"/></clipPath><gclip-path="url(#a)"><pathfill="#555"d="M0 0h55v20H0z"/><pathfill="#9f9f9f"d="M55 0h247v20H55z"/><pathfill="url(#b)"d="M0 0h302v20H0z"/></g><gfill="#fff"text-anchor="middle"font-family="DejaVu Sans,Verdana,Geneva,sans-serif"font-size="110"><textx="285"y="150"fill="#010101"fill-opacity=".3"transform="scale(.1)"textLength="450">artifacts</text><textx="285"y="140"transform="scale(.1)"textLength="450">artifacts</text><textx="1775"y="150"fill="#010101"fill-opacity=".3"transform="scale(.1)"textLength="2370">link is available only on Read the Docs site</text><textx="1775"y="140"transform="scale(.1)"textLength="2370">link is available only on Read the Docs site</text></g></svg>
<svgxmlns="http://www.w3.org/2000/svg"xmlns:xlink="http://www.w3.org/1999/xlink"width="302"height="20"><linearGradientid="b"x2="0"y2="100%"><stopoffset="0"stop-color="#bbb"stop-opacity=".1"/><stopoffset="1"stop-opacity=".1"/></linearGradient><clipPathid="a"><rectwidth="302"height="20"rx="3"fill="#fff"/></clipPath><gclip-path="url(#a)"><pathfill="#555"d="M0 0h55v20H0z"/><pathfill="#9f9f9f"d="M55 0h247v20H55z"/><pathfill="url(#b)"d="M0 0h302v20H0z"/></g><gfill="#fff"text-anchor="middle"font-family="DejaVu Sans,Verdana,Geneva,sans-serif"font-size="110"><textx="285"y="150"fill="#010101"fill-opacity=".3"transform="scale(.1)"textLength="450">artifacts</text><textx="285"y="140"transform="scale(.1)"textLength="450">artifacts</text><textx="1775"y="150"fill="#010101"fill-opacity=".3"transform="scale(.1)"textLength="2370">link is available only on Read the Docs site</text><textx="1775"y="140"transform="scale(.1)"textLength="2370">link is available only on Read the Docs site</text></g></svg>
| 1st | [IEEE's Signal Processing Society, Camera Model Identification](https://www.kaggle.com/c/sp-society-camera-model-identification)| [link](https://www.kaggle.com/c/sp-society-camera-model-identification/discussion/49367) | 2018.2 |
| 1st | [IEEE's Signal Processing Society, Camera Model Identification](https://www.kaggle.com/c/sp-society-camera-model-identification)| [link](https://www.kaggle.com/c/sp-society-camera-model-identification/discussion/49367) | 2018.2 |
