README.rst 16 KB
Newer Older
1
LightGBM Python-package
Guolin Ke's avatar
Guolin Ke committed
2
3
=======================

Nikita Titov's avatar
Nikita Titov committed
4
|License| |Python Versions| |PyPI Version| |Downloads| |API Docs|
Guolin Ke's avatar
Guolin Ke committed
5

Guolin Ke's avatar
Guolin Ke committed
6
7
8
Installation
------------

Guolin Ke's avatar
Guolin Ke committed
9
10
11
Preparation
'''''''''''

12
32-bit Python is not supported. Please install 64-bit version. If you have a strong need to install with 32-bit Python, refer to `Build 32-bit Version with 32-bit Python section <#build-32-bit-version-with-32-bit-python>`__.
Guolin Ke's avatar
Guolin Ke committed
13

14
15
Install from `PyPI <https://pypi.org/project/lightgbm>`_
''''''''''''''''''''''''''''''''''''''''''''''''''''''''
16

17
18
19
20
.. code:: sh

    pip install lightgbm

21
Compiled library that is included in the wheel file supports both **GPU** and **CPU** versions out of the box. This feature is experimental and available only for **Windows** and **Linux** currently. To use **GPU** version you only need to install OpenCL Runtime libraries. For NVIDIA and AMD GPU they are included in the ordinary drivers for your graphics card, so no action is required. If you would like your AMD or Intel CPU to act like a GPU (for testing and debugging) you can install `AMD APP SDK <https://github.com/microsoft/LightGBM/releases/download/v2.0.12/AMD-APP-SDKInstaller-v3.0.130.135-GA-windows-F-x64.exe>`_ on **Windows** and `PoCL <http://portablecl.org>`_ on **Linux**. Many modern Linux distributions provide packages for PoCL, look for ``pocl-opencl-icd`` on Debian-based distributions and ``pocl`` on RedHat-based distributions.
22

23
For **Windows** users, `VC runtime <https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads>`_ is needed if **Visual Studio** (2015 or newer) is not installed.
Guolin Ke's avatar
Guolin Ke committed
24

25
For **Linux** users, **glibc** >= 2.14 is required for LightGBM ``<=3.3.3`` and **glibc** >= 2.28 is required for newer versions. Also, in some rare cases, when you hit ``OSError: libgomp.so.1: cannot open shared object file: No such file or directory`` error during importing LightGBM, you need to install OpenMP runtime library separately (use your package manager and search for ``lib[g|i]omp`` for doing this).
26

27
For **macOS** (we provide wheels for 3 newest macOS versions) users:
28

29
- Starting from version 2.2.1, the library file in distribution wheels is built by the **Apple Clang** (Xcode_8.3.3 for versions 2.2.1 - 2.3.1, Xcode_9.4.1 for versions 2.3.2 - 3.3.2 and Xcode_11.7 from version 4.0.0) compiler. This means that you don't need to install the **gcc** compiler anymore. Instead of that you need to install the **OpenMP** library, which is required for running LightGBM on the system with the **Apple Clang** compiler. You can install the **OpenMP** library by the following command: ``brew install libomp``.
30

31
- For version smaller than 2.2.1 and not smaller than 2.1.2, **gcc-8** with **OpenMP** support must be installed first. Refer to `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#gcc>`__ for installation of **gcc-8** with **OpenMP** support.
32
33

- For version smaller than 2.1.2, **gcc-7** with **OpenMP** is required.
34

35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
Use LightGBM with Dask
**********************

.. warning::

    Dask-package is only tested on Linux.

To install all dependencies needed to use ``lightgbm.dask``, append ``[dask]``.

.. code:: sh

    pip install 'lightgbm[dask]'

Use LightGBM with pandas
************************

To install all dependencies needed to use ``pandas`` in LightGBM, append ``[pandas]``.

.. code:: sh

    pip install 'lightgbm[pandas]'

57
58
59
60
61
62
63
64
65
Use LightGBM with scikit-learn
******************************

To install all dependencies needed to use ``scikit-learn`` in LightGBM, append ``[scikit-learn]``.

.. code:: sh

    pip install 'lightgbm[scikit-learn]'

66
Build from Sources
67
******************
Guolin Ke's avatar
Guolin Ke committed
68

69
.. code:: sh
Guolin Ke's avatar
Guolin Ke committed
70

71
    pip install --no-binary lightgbm lightgbm
Guolin Ke's avatar
Guolin Ke committed
72

73
For **Linux** and **macOS** users, installation from sources requires installed `CMake`_.
Guolin Ke's avatar
Guolin Ke committed
74

75
For **Linux** users, **glibc** >= 2.28 is required. Also, in some rare cases you may need to install OpenMP runtime library separately (use your package manager and search for ``lib[g|i]omp`` for doing this).
76

77
78
For **macOS** users, you can perform installation either with **Apple Clang** or **gcc**.

79
- In case you prefer **Apple Clang**, you should install **OpenMP** (details for installation can be found in `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#apple-clang>`__) first and **CMake** version 3.16 or higher is required.
80

81
- In case you prefer **gcc**, you need to install it (details for installation can be found in `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#gcc>`__) and specify compilers by running ``export CXX=g++-7 CC=gcc-7`` (replace "7" with version of **gcc** installed on your machine) first.
Guolin Ke's avatar
Guolin Ke committed
82

83
For **Windows** users, **Visual Studio** (or `VS Build Tools <https://visualstudio.microsoft.com/downloads/>`_) is needed. If you get any errors during installation, you may need to install `CMake`_ (version 3.8 or higher).
Guolin Ke's avatar
Guolin Ke committed
84

85
86
87
88
89
Build Threadless Version
~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: sh

90
    pip install lightgbm --config-settings=cmake.define.USE_OPENMP=OFF
91

92
All requirements, except the **OpenMP** requirement, from `Build from Sources section <#build-from-sources>`__ apply for this installation option as well.
93
94
95

It is **strongly not recommended** to use this version of LightGBM!

96
97
98
99
100
Build MPI Version
~~~~~~~~~~~~~~~~~

.. code:: sh

101
    pip install lightgbm --config-settings=cmake.define.USE_MPI=ON
102

103
All requirements from `Build from Sources section <#build-from-sources>`__ apply for this installation option as well.
104
105

For **Windows** users, compilation with **MinGW-w64** is not supported and `CMake`_ (version 3.8 or higher) is strongly required.
106

107
**MPI** libraries are needed: details for installation can be found in `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#build-mpi-version>`__.
108

109
Build GPU Version
110
~~~~~~~~~~~~~~~~~
Guolin Ke's avatar
Guolin Ke committed
111

112
.. code:: sh
113

114
    pip install lightgbm --config-settings=cmake.define.USE_GPU=ON
115

116
All requirements from `Build from Sources section <#build-from-sources>`__ apply for this installation option as well.
117

118
For **Windows** users, `CMake`_ (version 3.8 or higher) is strongly required.
119

120
**Boost** and **OpenCL** are needed: details for installation can be found in `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#build-gpu-version>`__. Almost always you also need to pass ``OpenCL_INCLUDE_DIR``, ``OpenCL_LIBRARY`` options for **Linux** and ``BOOST_ROOT``, ``BOOST_LIBRARYDIR`` options for **Windows** to **CMake** via ``pip`` options, like
121
122
123

.. code:: sh

124
125
126
127
    pip install lightgbm \
      --config-settings=cmake.define.USE_GPU=ON \
      --config-settings=cmake.define.OpenCL_INCLUDE_DIR="/usr/local/cuda/include/" \
      --config-settings=cmake.define.OpenCL_LIBRARY="/usr/local/cuda/lib64/libOpenCL.so"
128

129
All available options that can be passed via ``cmake.define.{option}``.
130

131
- Boost_ROOT
132

133
- Boost_DIR
134

135
- Boost_INCLUDE_DIR
136

137
- BOOST_LIBRARYDIR
138

139
- OpenCL_INCLUDE_DIR
140

141
- OpenCL_LIBRARY
142

143
For more details see `FindBoost <https://cmake.org/cmake/help/latest/module/FindBoost.html>`__ and `FindOpenCL <https://cmake.org/cmake/help/latest/module/FindOpenCL.html>`__.
144

145
146
147
148
149
Build CUDA Version
~~~~~~~~~~~~~~~~~~

.. code:: sh

150
    pip install lightgbm --config-settings=cmake.define.USE_CUDA=ON
151

152
All requirements from `Build from Sources section <#build-from-sources>`__ apply for this installation option as well, and `CMake`_ (version 3.16 or higher) is strongly required.
153

154
**CUDA** library (version 10.0 or higher) is needed: details for installation can be found in `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#build-cuda-version>`__.
155

156
To use the CUDA version within Python, pass ``{"device": "cuda"}`` respectively in parameters.
157

158
159
160
Build HDFS Version
~~~~~~~~~~~~~~~~~~

James Lamb's avatar
James Lamb committed
161
162
163
164
.. warning::
   HDFS support in LightGBM is deprecated, and will be removed in a future release.
   See https://github.com/microsoft/LightGBM/issues/6436.

165
166
.. code:: sh

167
    pip install lightgbm --config-settings=cmake.define.USE_HDFS=ON
168

169
170
171
172
All requirements from `Build from Sources section <#build-from-sources>`__ apply for this installation option as well.

**HDFS** library is needed: details for installation can be found in `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#build-hdfs-version>`__.

173
Note that the installation process of HDFS version was tested only on **Linux**.
174

175
176
177
178
179
Build with MinGW-w64 on Windows
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: sh

180
181
182
    # in sh.exe, git bash, or other Unix-like shell
    export CMAKE_GENERATOR='MinGW Makefiles'
    pip install lightgbm --config-settings=cmake.define.CMAKE_SH=CMAKE_SH-NOTFOUND
183

184
`CMake`_ and `MinGW-w64 <https://www.mingw-w64.org/>`_ should be installed first.
185

186
187
It is recommended to use **Visual Studio** for its better multithreading efficiency in **Windows** for many-core systems
(see `Question 4 <https://github.com/microsoft/LightGBM/blob/master/docs/FAQ.rst#4-i-am-using-windows-should-i-use-visual-studio-or-mingw-for-compiling-lightgbm>`__ and `Question 8 <https://github.com/microsoft/LightGBM/blob/master/docs/FAQ.rst#8-cpu-usage-is-low-like-10-in-windows-when-using-lightgbm-on-very-large-datasets-with-many-core-systems>`__).
Guolin Ke's avatar
Guolin Ke committed
188

189
190
191
192
193
Build 32-bit Version with 32-bit Python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: sh

194
195
196
197
    # in sh.exe, git bash, or other Unix-like shell
    export CMAKE_GENERATOR='Visual Studio 17 2022'
    export CMAKE_GENERATOR_PLATFORM='Win32'
    pip install --no-binary lightgbm lightgbm
198
199
200
201
202

By default, installation in environment with 32-bit Python is prohibited. However, you can remove this prohibition on your own risk by passing ``bit32`` option.

It is **strongly not recommended** to use this version of LightGBM!

203
Build with Time Costs Output
204
205
206
207
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: sh

208
    pip install lightgbm --config-settings=cmake.define.USE_TIMETAG=ON
209
210
211

Use this option to make LightGBM output time costs for different internal routines, to investigate and benchmark its performance.

212
213
214
215
216
Install from `conda-forge channel <https://anaconda.org/conda-forge/lightgbm>`_
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''

If you use ``conda`` to manage Python dependencies, you can install LightGBM using ``conda install``.

217
218
We strongly recommend installation from the ``conda-forge`` channel and not from the ``default`` one due to many reasons. The main ones are less time delay for new releases, greater number of supported architectures and better handling of dependency conflicts, especially workaround for OpenMP is crucial for LightGBM. More details can be found in `this comment <https://github.com/microsoft/LightGBM/issues/4948#issuecomment-1013766397>`_.

219
220
221
222
223
224
**Note**: The `lightgbm conda-forge feedstock <https://github.com/conda-forge/lightgbm-feedstock>`_ is not maintained by LightGBM maintainers.

.. code:: sh

    conda install -c conda-forge lightgbm

Guolin Ke's avatar
Guolin Ke committed
225
226
Install from GitHub
'''''''''''''''''''
Guolin Ke's avatar
Guolin Ke committed
227

228
All requirements from `Build from Sources section <#build-from-sources>`__ apply for this installation option as well.
Guolin Ke's avatar
Guolin Ke committed
229

230
For **Windows** users, if you get any errors during installation and there is the warning ``WARNING:LightGBM:Compilation with MSBuild from existing solution file failed.`` in the log, you should install `CMake`_ (version 3.8 or higher).
Guolin Ke's avatar
Guolin Ke committed
231

Guolin Ke's avatar
Guolin Ke committed
232
233
.. code:: sh

234
    git clone --recursive https://github.com/microsoft/LightGBM.git
235
    # export CXX=g++-7 CC=gcc-7  # macOS users, if you decided to compile with gcc, don't forget to specify compilers (replace "7" with version of gcc installed on your machine)
236
    sh ./build-python.sh install
Guolin Ke's avatar
Guolin Ke committed
237

238
Note: ``sudo`` (or administrator rights in **Windows**) may be needed to perform the command.
239

240
Run ``sh ./build-python.sh install --nomp`` to disable **OpenMP** support. All requirements from `Build Threadless Version section <#build-threadless-version>`__ apply for this installation option as well.
241

242
Run ``sh ./build-python.sh install --mpi`` to enable **MPI** support. All requirements from `Build MPI Version section <#build-mpi-version>`__ apply for this installation option as well.
Guolin Ke's avatar
Guolin Ke committed
243

244
Run ``sh ./build-python.sh install --mingw``, if you want to use **MinGW-w64** on **Windows** instead of **Visual Studio**. All requirements from `Build with MinGW-w64 on Windows section <#build-with-mingw-w64-on-windows>`__ apply for this installation option as well.
245

246
Run ``sh ./build-python.sh install --gpu`` to enable GPU support. All requirements from `Build GPU Version section <#build-gpu-version>`__ apply for this installation option as well. To pass additional options to **CMake** use the following syntax: ``sh ./build-python.sh install --gpu --opencl-include-dir="/usr/local/cuda/include/"``, see `Build GPU Version section <#build-gpu-version>`__ for the complete list of them.
247

248
Run ``sh ./build-python.sh install --cuda`` to enable CUDA support. All requirements from `Build CUDA Version section <#build-cuda-version>`__ apply for this installation option as well.
249

250
Run ``sh ./build-python.sh install --hdfs`` to enable HDFS support. All requirements from `Build HDFS Version section <#build-hdfs-version>`__ apply for this installation option as well.
Guolin Ke's avatar
Guolin Ke committed
251

252
Run ``sh ./build-python.sh install --bit32``, if you want to use 32-bit version. All requirements from `Build 32-bit Version with 32-bit Python section <#build-32-bit-version-with-32-bit-python>`__ apply for this installation option as well.
253

254
Run ``sh ./build-python.sh install --time-costs``, if you want to output time costs for different internal routines. All requirements from `Build with Time Costs Output section <#build-with-time-costs-output>`__ apply for this installation option as well.
255

256
If you get any errors during installation or due to any other reasons, you may want to build dynamic library from sources by any method you prefer (see `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst>`__) and then just run ``sh ./build-python.sh install --precompile``.
wxchan's avatar
wxchan committed
257

258
259
260
Build Wheel File
****************

261
262
263
264
265
266
267
You can use ``sh ./build-python.sh install bdist_wheel`` to build a wheel file but not install it.

That script requires some dependencies like ``build``, ``scikit-build-core``, and ``wheel``.
In environments with restricted or no internt access, install those tools and then pass ``--no-isolation``.

.. code:: sh

Kyle Engel's avatar
Kyle Engel committed
268
  sh ./build-python.sh bdist_wheel --no-isolation
269

270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
Build With MSBuild
******************

To use ``MSBuild`` (Windows-only), first build ``lib_lightgbm.dll`` by running the following from the root of the repo.

.. code:: sh

  MSBuild.exe windows/LightGBM.sln /p:Configuration=DLL /p:Platform=x64 /p:PlatformToolset=v143

Then install the Python package using that library.

.. code:: sh

  sh ./build-python.sh install --precompile

wxchan's avatar
wxchan committed
285
Troubleshooting
Guolin Ke's avatar
Guolin Ke committed
286
---------------
wxchan's avatar
wxchan committed
287

288
289
In case you are facing any errors during the installation process, you can examine ``$HOME/LightGBM_compilation.log`` file, in which all operations are logged, to get more details about occurred problem. Also, please attach this file to the issue on GitHub to help faster indicate the cause of the error.

290
Refer to `FAQ <https://github.com/microsoft/LightGBM/tree/master/docs/FAQ.rst>`_.
wxchan's avatar
wxchan committed
291

292
293
294
Examples
--------

295
Refer to the walk through examples in `Python guide folder <https://github.com/microsoft/LightGBM/tree/master/examples/python-guide>`_.
296

297
298
Development Guide
-----------------
wxchan's avatar
wxchan committed
299

300
301
302
303
To check that a contribution to the package matches its style expectations, run the following from the root of the repo.

.. code:: sh

304
    sh .ci/lint-python.sh
305

306
.. |License| image:: https://img.shields.io/github/license/microsoft/lightgbm.svg
307
   :target: https://github.com/microsoft/LightGBM/blob/master/LICENSE
308
.. |Python Versions| image:: https://img.shields.io/pypi/pyversions/lightgbm.svg?logo=python&logoColor=white
309
   :target: https://pypi.org/project/lightgbm
310
.. |PyPI Version| image:: https://img.shields.io/pypi/v/lightgbm.svg?logo=pypi&logoColor=white
311
   :target: https://pypi.org/project/lightgbm
312
313
.. |Downloads| image:: https://pepy.tech/badge/lightgbm
   :target: https://pepy.tech/project/lightgbm
Nikita Titov's avatar
Nikita Titov committed
314
315
.. |API Docs| image:: https://readthedocs.org/projects/lightgbm/badge/?version=latest
   :target: https://lightgbm.readthedocs.io/en/latest/Python-API.html
316
.. _CMake: https://cmake.org/