Unverified Commit 686b0cc2 authored by Nikita Titov's avatar Nikita Titov Committed by GitHub
Browse files

[docs][python] allow usage and compilation of 32-bit library (#2188)

* allow usage and compilation of 32-bit library

* added docs
parent aee92f63
...@@ -3,6 +3,10 @@ Installation Guide ...@@ -3,6 +3,10 @@ Installation Guide
Here is the guide for the build of LightGBM CLI version. Here is the guide for the build of LightGBM CLI version.
All instructions below are aimed to compile 64-bit version of LightGBM.
It is worth to compile 32-bit version only in very rare special cases of environmental limitations.
32-bit version is slow and untested, so use it on your own risk and don't forget to adjust some commands in this guide.
For the build of Python-package and R-package, please refer to `Python-package`_ and `R-package`_ folders respectively. For the build of Python-package and R-package, please refer to `Python-package`_ and `R-package`_ folders respectively.
Also you can download artifacts of the latest successful build in master branch: |download artifacts|. Also you can download artifacts of the latest successful build in master branch: |download artifacts|.
......
...@@ -9,7 +9,7 @@ Installation ...@@ -9,7 +9,7 @@ Installation
Preparation Preparation
''''''''''' '''''''''''
32-bit Python is not supported. Please install 64-bit version. 32-bit Python is not supported. Please install 64-bit version. If you have a strong need to install with 32-bit Python, refer to `Build 32-bit Version with 32-bit Python section <#build-32-bit-version-with-32-bit-python>`__.
`setuptools <https://pypi.org/project/setuptools>`_ is needed. `setuptools <https://pypi.org/project/setuptools>`_ is needed.
...@@ -135,6 +135,17 @@ Build with MinGW-w64 on Windows ...@@ -135,6 +135,17 @@ Build with MinGW-w64 on Windows
It is recommended to use **Visual Studio** for its better multithreading efficiency in **Windows** for many-core systems It is recommended to use **Visual Studio** for its better multithreading efficiency in **Windows** for many-core systems
(see `Question 4 <https://github.com/microsoft/LightGBM/blob/master/docs/FAQ.rst#4-i-am-using-windows-should-i-use-visual-studio-or-mingw-for-compiling-lightgbm>`__ and `Question 8 <https://github.com/microsoft/LightGBM/blob/master/docs/FAQ.rst#8-cpu-usage-is-low-like-10-in-windows-when-using-lightgbm-on-very-large-datasets-with-many-core-systems>`__). (see `Question 4 <https://github.com/microsoft/LightGBM/blob/master/docs/FAQ.rst#4-i-am-using-windows-should-i-use-visual-studio-or-mingw-for-compiling-lightgbm>`__ and `Question 8 <https://github.com/microsoft/LightGBM/blob/master/docs/FAQ.rst#8-cpu-usage-is-low-like-10-in-windows-when-using-lightgbm-on-very-large-datasets-with-many-core-systems>`__).
Build 32-bit Version with 32-bit Python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
pip install lightgbm --install-option=--bit32
By default, installation in environment with 32-bit Python is prohibited. However, you can remove this prohibition on your own risk by passing ``bit32`` option.
It is **strongly not recommended** to use this version of LightGBM!
Install from GitHub Install from GitHub
''''''''''''''''''' '''''''''''''''''''
...@@ -163,6 +174,8 @@ Run ``python setup.py install --gpu`` to enable GPU support. All remarks from `B ...@@ -163,6 +174,8 @@ Run ``python setup.py install --gpu`` to enable GPU support. All remarks from `B
Run ``python setup.py install --hdfs`` to enable HDFS support. All remarks from `Build HDFS Version section <#build-hdfs-version>`__ are actual in this case. Run ``python setup.py install --hdfs`` to enable HDFS support. All remarks from `Build HDFS Version section <#build-hdfs-version>`__ are actual in this case.
Run ``python setup.py install --bit32``, if you want to use 32-bit version. All remarks from `Build 32-bit Version with 32-bit Python section <#build-32-bit-version-with-32-bit-python>`__ are actual in this case.
If you get any errors during installation or due to any other reasons, you may want to build dynamic library from sources by any method you prefer (see `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst>`__) and then just run ``python setup.py install --precompile``. If you get any errors during installation or due to any other reasons, you may want to build dynamic library from sources by any method you prefer (see `Installation Guide <https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst>`__) and then just run ``python setup.py install --precompile``.
Troubleshooting Troubleshooting
......
...@@ -87,11 +87,12 @@ def silent_call(cmd, raise_error=False, error_msg=''): ...@@ -87,11 +87,12 @@ def silent_call(cmd, raise_error=False, error_msg=''):
return 1 return 1
def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False, nomp=False, def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False,
use_hdfs=False, boost_root=None, boost_dir=None, use_hdfs=False, boost_root=None, boost_dir=None,
boost_include_dir=None, boost_librarydir=None, boost_include_dir=None, boost_librarydir=None,
opencl_include_dir=None, opencl_library=None, opencl_include_dir=None, opencl_library=None,
openmp_include_dir=None, openmp_library=None): openmp_include_dir=None, openmp_library=None,
nomp=False, bit32=False):
if os.path.exists(os.path.join(CURRENT_DIR, "build_cpp")): if os.path.exists(os.path.join(CURRENT_DIR, "build_cpp")):
shutil.rmtree(os.path.join(CURRENT_DIR, "build_cpp")) shutil.rmtree(os.path.join(CURRENT_DIR, "build_cpp"))
...@@ -134,7 +135,7 @@ def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False, nomp=False, ...@@ -134,7 +135,7 @@ def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False, nomp=False,
else: else:
status = 1 status = 1
lib_path = os.path.join(CURRENT_DIR, "compile", "windows", "x64", "DLL", "lib_lightgbm.dll") lib_path = os.path.join(CURRENT_DIR, "compile", "windows", "x64", "DLL", "lib_lightgbm.dll")
if not any((use_gpu, use_mpi, use_hdfs)): if not any((use_gpu, use_mpi, use_hdfs, nomp, bit32)):
logger.info("Starting to compile with MSBuild from existing solution file.") logger.info("Starting to compile with MSBuild from existing solution file.")
platform_toolsets = ("v142", "v141", "v140") platform_toolsets = ("v142", "v141", "v140")
for pt in platform_toolsets: for pt in platform_toolsets:
...@@ -150,10 +151,11 @@ def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False, nomp=False, ...@@ -150,10 +151,11 @@ def compile_cpp(use_mingw=False, use_gpu=False, use_mpi=False, nomp=False,
if status != 0 or not os.path.exists(lib_path): if status != 0 or not os.path.exists(lib_path):
logger.warning("Compilation with MSBuild from existing solution file failed.") logger.warning("Compilation with MSBuild from existing solution file failed.")
if status != 0 or not os.path.exists(lib_path): if status != 0 or not os.path.exists(lib_path):
arch = "Win32" if bit32 else "x64"
vs_versions = ("Visual Studio 16 2019", "Visual Studio 15 2017", "Visual Studio 14 2015") vs_versions = ("Visual Studio 16 2019", "Visual Studio 15 2017", "Visual Studio 14 2015")
for vs in vs_versions: for vs in vs_versions:
logger.info("Starting to compile with %s." % vs) logger.info("Starting to compile with %s." % vs)
status = silent_call(cmake_cmd + ["-G", vs, "-A", "x64"]) status = silent_call(cmake_cmd + ["-G", vs, "-A", arch])
if status == 0: if status == 0:
break break
else: else:
...@@ -222,6 +224,7 @@ class CustomInstall(install): ...@@ -222,6 +224,7 @@ class CustomInstall(install):
('mpi', None, 'Compile MPI version'), ('mpi', None, 'Compile MPI version'),
('nomp', None, 'Compile version without OpenMP support'), ('nomp', None, 'Compile version without OpenMP support'),
('hdfs', 'h', 'Compile HDFS version'), ('hdfs', 'h', 'Compile HDFS version'),
('bit32', None, 'Compile 32-bit version'),
('precompile', 'p', 'Use precompiled library'), ('precompile', 'p', 'Use precompiled library'),
('boost-root=', None, 'Boost preferred installation prefix'), ('boost-root=', None, 'Boost preferred installation prefix'),
('boost-dir=', None, 'Directory with Boost package configuration file'), ('boost-dir=', None, 'Directory with Boost package configuration file'),
...@@ -249,16 +252,25 @@ class CustomInstall(install): ...@@ -249,16 +252,25 @@ class CustomInstall(install):
self.hdfs = 0 self.hdfs = 0
self.precompile = 0 self.precompile = 0
self.nomp = 0 self.nomp = 0
self.bit32 = 0
def run(self): def run(self):
if (8 * struct.calcsize("P")) != 64:
if self.bit32:
logger.warning("You're installing 32-bit version. "
"This version is slow and untested, so use it on your own risk.")
else:
raise Exception("Cannot install LightGBM in 32-bit Python, "
"please use 64-bit Python instead.")
open(LOG_PATH, 'wb').close() open(LOG_PATH, 'wb').close()
if not self.precompile: if not self.precompile:
copy_files(use_gpu=self.gpu) copy_files(use_gpu=self.gpu)
compile_cpp(use_mingw=self.mingw, use_gpu=self.gpu, use_mpi=self.mpi, nomp=self.nomp, compile_cpp(use_mingw=self.mingw, use_gpu=self.gpu, use_mpi=self.mpi,
use_hdfs=self.hdfs, boost_root=self.boost_root, boost_dir=self.boost_dir, use_hdfs=self.hdfs, boost_root=self.boost_root, boost_dir=self.boost_dir,
boost_include_dir=self.boost_include_dir, boost_librarydir=self.boost_librarydir, boost_include_dir=self.boost_include_dir, boost_librarydir=self.boost_librarydir,
opencl_include_dir=self.opencl_include_dir, opencl_library=self.opencl_library, opencl_include_dir=self.opencl_include_dir, opencl_library=self.opencl_library,
openmp_include_dir=self.openmp_include_dir, openmp_library=self.openmp_library) openmp_include_dir=self.openmp_include_dir, openmp_library=self.openmp_library,
nomp=self.nomp, bit32=self.bit32)
install.run(self) install.run(self)
if os.path.isfile(LOG_PATH): if os.path.isfile(LOG_PATH):
os.remove(LOG_PATH) os.remove(LOG_PATH)
...@@ -281,9 +293,6 @@ class CustomSdist(sdist): ...@@ -281,9 +293,6 @@ class CustomSdist(sdist):
if __name__ == "__main__": if __name__ == "__main__":
if (8 * struct.calcsize("P")) != 64:
raise Exception('Cannot install LightGBM in 32-bit Python, please use 64-bit Python instead.')
CURRENT_DIR = os.path.abspath(os.path.dirname(__file__)) CURRENT_DIR = os.path.abspath(os.path.dirname(__file__))
LOG_PATH = os.path.join(os.path.expanduser('~'), 'LightGBM_compilation.log') LOG_PATH = os.path.join(os.path.expanduser('~'), 'LightGBM_compilation.log')
LOG_NOTICE = "The full version of error log was saved into {0}".format(LOG_PATH) LOG_NOTICE = "The full version of error log was saved into {0}".format(LOG_PATH)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment