[R-package] Remove ability to install precompiled lib_lightgbm (fixes #3320) (#3360)

* [R-package] [docs] Reorganize installation instructions (fixes #3320)

* more changes

* remove ability to use precompiled lib_lightgbm

* remove language about installing from the CRAN section

* move installation stuff

* Apply suggestions from code review
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

* fix anchor
Co-authored-by: Nikita Titov <nekit94-08@mail.ru>

R-package/README.md

@@ -5,22 +5,51 @@
 ### Contents
 
 * [Installation](#installation)
+    - [Installing the CRAN Package](#installing-the-cran-package)
+    - [Installing from Source with CMake](#install)
+    - [Installing a GPU-enabled Build](#installing-a-gpu-enabled-build)
+    - [Installing Precompiled Binaries](#installing-precompiled-binaries)
+    - [Installing from a Pre-compiled lib_lightgbm](#lib_lightgbm)
 * [Examples](#examples)
 * [Testing](#testing)
-* [Preparing a CRAN Package and Installing It](#preparing-a-cran-package-and-installing-it)
+* [Preparing a CRAN Package](#preparing-a-cran-package)
 * [External Repositories](#external-unofficial-repositories)
 * [Known Issues](#known-issues)
 
 Installation
 ------------
 
+For the easiest installation, go to ["Installing the CRAN package"](#installing-the-cran-package).
+
+If you experience any issues with that, try ["Installing from Source with CMake"](#install). This can produce a more efficient version of the library on Windows systems with Visual Studio.
+
+To build a GPU-enabled version of the package, follow the steps in ["Installing a GPU-enabled Build"](#installing-a-gpu-enabled-build).
+
+If any of the above options do not work for you or do not meet your needs, please let the maintainers know by [opening an issue](https://github.com/microsoft/LightGBM/issues).
+
+When your package installation is done, you can check quickly if your LightGBM R-package is working by running the following:
+
+```r
+library(lightgbm)
+data(agaricus.train, package='lightgbm')
+train <- agaricus.train
+dtrain <- lgb.Dataset(train$data, label = train$label)
+model <- lgb.cv(
+    params = list(
+        objective = "regression"
+        , metric = "l2"
+    )
+    , data = dtrain
+)
+```
+
 ### Installing the CRAN package
 
-As of this writing, `LightGBM`'s R package is not available on CRAN. However, start with `LightGBM` 3.0.0, you can install a released source distribution. This is the same type of package that you'd install from CRAN. It does not require `CMake`, Visual Studio, or anything else outside the CRAN toolchain.
+As of this writing, `LightGBM`'s R package is not available on CRAN. However, start with `LightGBM` 3.0.0, you can install a released source distribution. This is the same type of package that you'd install from CRAN. It does not require `CMake`, `Visual Studio`, or anything else outside the CRAN toolchain.
 
 To install this package on any operating system:
 
-1. Choose a release from [the "Releases" page](https://github.com/microsoft/LightGBM/releases)
+1. Choose a release from [the "Releases" page](https://github.com/microsoft/LightGBM/releases).
 2. Look for the artifact with a name like `lightgbm-{VERSION}-r-cran.tar.gz`. Right-click it and choose "copy link address".
 3. Copy that link into `PKG_URL` in the code below and run it.
 
@@ -30,47 +59,31 @@ PKG_URL <- "https://github.com/microsoft/LightGBM/releases/download/v3.0.0/light
 remotes::install_url(PKG_URL)
 ```
 
-### Installing Precompiled Binaries
-
-Starting with `LightGBM` 3.0.0, precompiled binaries for the R package are created for each release. These packages do not require compilation, so they will be faster and easier to install than packages that are built from source. These packages are created with R 4.0 and are not guaranteed to work with other R versions.
-
-Binaries are available for Windows, Mac, and Linux systems. They are not guaranteed to work with all variants and versions of these operating systems. Please [open an issue](https://github.com/microsoft/LightGBM/issues) if you encounter any problems.
-
-To install a binary for the R package:
-
-1. Choose a release from [the "Releases" page](https://github.com/microsoft/LightGBM/releases).
-2. Choose a file based on your operating system. Right-click it and choose "copy link address".
-    * Linux: `lightgbm-{VERSION}-r40-linux.tgz`
-    * Mac: `lightgbm-{VERSION}-r40-macos.tgz`
-    * Windows: `lightgbm-{VERSION}-r40-windows.zip`
-3. Copy that link into `PKG_URL` in the code below and run it.
+#### Custom Installation (Linux, Mac)
 
-This sample code installs version 3.0.0-1 of the R package on Mac.
+The steps above should work on most systems, but users with highly-customized environments might want to change how R builds packages from source.
 
-```r
-PKG_URL <- "https://github.com/microsoft/LightGBM/releases/download/v3.0.0rc1/lightgbm-3.0.0-1-r40-macos.tgz"
+To change the compiler used when installing the CRAN package, you can create a file `~/.R/Makevars` which overrides `CC` (`C` compiler) and `CXX` (`C++` compiler).
 
-local_file <- paste0("lightgbm.", tools::file_ext(PKG_URL))
+For example, to use `gcc` instead of `clang` on Mac, you could use something like the following:
 
-download.file(
-    url = PKG_URL
-    , destfile = local_file
-)
-install.packages(
-    pkgs = local_file
-    , type = "binary"
-    , repos = NULL
-)
+```make
+# ~/.R/Makevars
+CC=gcc-8
+CXX=g++-8
+CXX11=g++-8
 ```
 
-### Preparation
+### Installing from Source with CMake <a name="install"></a>
 
 You need to install git and [CMake](https://cmake.org/) first.
 
-Note: 32-bit (i386) R/Rtools is currently not supported.
+Note: this method is only supported on 64-bit systems. If you need to run LightGBM on 32-bit Windows (i386), follow the instructions in ["Installing the CRAN Package"](#installing-the-cran-package).
 
 #### Windows Preparation
 
+NOTE: Windows users may need to run with administrator rights (either R or the command prompt, depending on the way you are installing this package).
+
 Installing a 64-bit version of [Rtools](https://cran.r-project.org/bin/windows/Rtools/) is mandatory.
 
 After installing `Rtools` and `CMake`, be sure the following paths are added to the environment variable `PATH`. These may have been automatically added when installing other software.
@@ -127,9 +140,9 @@ You can perform installation either with **Apple Clang** or **gcc**. In case you
 export CXX=/usr/local/bin/g++-8 CC=/usr/local/bin/gcc-8
 ```
 
-### Install
+#### Install with CMake
 
-Build and install R-package with the following commands:
+After following the "preparation" steps above for your operating system, build and install the R-package with the following commands:
 
 ```sh
 git clone --recursive https://github.com/microsoft/LightGBM
@@ -141,23 +154,53 @@ The `build_r.R` script builds the package in a temporary directory called `light
 
 Note: for the build with Visual Studio/VS Build Tools in Windows, you should use the Windows CMD or Powershell.
 
-Windows users may need to run with administrator rights (either R or the command prompt, depending on the way you are installing this package). Linux users might require the appropriate user write permissions for packages.
+### Installing a GPU-enabled Build
 
 Set `use_gpu` to `TRUE` in `R-package/src/install.libs.R` to enable the build with GPU support. You will need to install Boost and OpenCL first: details for installation can be found in [Installation-Guide](https://github.com/microsoft/LightGBM/blob/master/docs/Installation-Guide.rst#build-gpu-version).
 
-If you are using a precompiled dll/lib locally, you can move the dll/lib into LightGBM root folder, modify `LightGBM/R-package/src/install.libs.R`'s 2nd line (change `use_precompile <- FALSE` to `use_precompile <- TRUE`), and install R-package as usual.
+After installing these other libraries, follow the steps in ["Installing from Source with CMake"](#install).
 
-When your package installation is done, you can check quickly if your LightGBM R-package is working by running the following:
+### Installing Precompiled Binaries
+
+**NOTE:** As of this writing, the precompiled binaries of the R package should be considered experimental. If you try them an experience any problems, please [open an issue](https://github.com/microsoft/LightGBM/issues).
+
+Starting with `LightGBM` 3.0.0, precompiled binaries for the R package are created for each release. These packages do not require compilation, so they will be faster and easier to install than packages that are built from source. These packages are created with R 4.0 and are not guaranteed to work with other R versions.
+
+Binaries are available for Windows, Mac, and Linux systems. They are not guaranteed to work with all variants and versions of these operating systems. Please [open an issue](https://github.com/microsoft/LightGBM/issues) if you encounter any problems.
+
+To install a binary for the R package:
+
+1. Choose a release from [the "Releases" page](https://github.com/microsoft/LightGBM/releases).
+2. Choose a file based on your operating system. Right-click it and choose "copy link address".
+    * Linux: `lightgbm-{VERSION}-r40-linux.tgz`
+    * Mac: `lightgbm-{VERSION}-r40-macos.tgz`
+    * Windows: `lightgbm-{VERSION}-r40-windows.zip`
+3. Copy that link into `PKG_URL` in the code below and run it.
+
+This sample code installs version 3.0.0-1 of the R package on Mac.
 
 ```r
-library(lightgbm)
-data(agaricus.train, package='lightgbm')
-train <- agaricus.train
-dtrain <- lgb.Dataset(train$data, label=train$label)
-params <- list(objective="regression", metric="l2")
-model <- lgb.cv(params, dtrain, 10, nfold=5, min_data=1, learning_rate=1, early_stopping_rounds=10)
+PKG_URL <- "https://github.com/microsoft/LightGBM/releases/download/v3.0.0rc1/lightgbm-3.0.0-1-r40-macos.tgz"
+
+local_file <- paste0("lightgbm.", tools::file_ext(PKG_URL))
+
+download.file(
+    url = PKG_URL
+    , destfile = local_file
+)
+install.packages(
+    pkgs = local_file
+    , type = "binary"
+    , repos = NULL
+)
 ```
 
+### Installing from a Pre-compiled lib_lightgbm <a name="lib_lightgbm"></a>
+
+Previous versions of LightGBM offered the ability to first compile the C++ library (`lib_lightgbm.so` or `lib_lightgbm.dll`) and then build an R package that wraps it.
+
+As of version 3.0.0, this is no longer supported. If building from source is difficult for you, please [open an issue](https://github.com/microsoft/LightGBM/issues).
+
 Examples
 --------
 
@@ -194,8 +237,8 @@ Rscript -e " \
     "
 ```
 
-Preparing a CRAN Package and Installing It
-------------------------------------------
+Preparing a CRAN Package
+------------------------
 
 This section is primarily for maintainers, but may help users and contributors to understand the structure of the R package.
 
@@ -221,17 +264,6 @@ After building the package, install it with a command like the following:
 R CMD install lightgbm_*.tar.gz
 ```
 
-#### Custom Installation (Linux, Mac)
-
-To change the compiler used when installing the package, you can create a file `~/.R/Makevars` which overrides `CC` (`C` compiler) and `CXX` (`C++` compiler). For example, to use `gcc` instead of `clang` on Mac, you could use something like the following:
-
-```make
-# ~/.R/Makevars
-CC=gcc-8
-CXX=g++-8
-CXX11=g++-8
-```
-
 ### Changing the CRAN Package
 
 A lot of details are handled automatically by `R CMD build` and `R CMD install`, so it can be difficult to understand how the files in the R package are related to each other. An extensive treatment of those details is available in ["Writing R Extensions"](https://cran.r-project.org/doc/manuals/r-release/R-exts.html).

R-package/src/install.libs.R

 # User options
-use_precompile <- FALSE
 use_gpu <- FALSE
 
 # For Windows, the package will be built with Visual Studio
@@ -107,176 +106,144 @@ if (!write_succeeded) {
 source_dir <- file.path(R_PACKAGE_SOURCE, "src", fsep = "/")
 build_dir <- file.path(source_dir, "build", fsep = "/")
 
-# Check for precompilation
-if (!use_precompile) {
-
-  # Prepare building package
-  dir.create(
-    build_dir
-    , recursive = TRUE
-    , showWarnings = FALSE
-  )
-  setwd(build_dir)
+# Prepare building package
+dir.create(
+  build_dir
+  , recursive = TRUE
+  , showWarnings = FALSE
+)
+setwd(build_dir)
 
-  use_visual_studio <- !(use_mingw || use_msys2)
+use_visual_studio <- !(use_mingw || use_msys2)
 
-  # If using MSVC to build, pull in the script used
-  # to create R.def from R.dll
-  if (WINDOWS && use_visual_studio) {
-    write_succeeded <- file.copy(
-      "../../inst/make-r-def.R"
-      , file.path(build_dir, "make-r-def.R")
-      , overwrite = TRUE
-    )
-    if (!write_succeeded) {
-      stop("Copying make-r-def.R failed")
-    }
+# If using MSVC to build, pull in the script used
+# to create R.def from R.dll
+if (WINDOWS && use_visual_studio) {
+  write_succeeded <- file.copy(
+    "../../inst/make-r-def.R"
+    , file.path(build_dir, "make-r-def.R")
+    , overwrite = TRUE
+  )
+  if (!write_succeeded) {
+    stop("Copying make-r-def.R failed")
   }
+}
 
-  # Prepare installation steps
-  cmake_args <- NULL
-  build_cmd <- "make"
-  build_args <- "_lightgbm"
-  lib_folder <- file.path(source_dir, fsep = "/")
+# Prepare installation steps
+cmake_args <- NULL
+build_cmd <- "make"
+build_args <- "_lightgbm"
+lib_folder <- file.path(source_dir, fsep = "/")
 
-  WINDOWS_BUILD_TOOLS <- list(
-    "MinGW" = c(
-      build_tool = "mingw32-make.exe"
-      , makefile_generator = "MinGW Makefiles"
-    )
-    , "MSYS2" = c(
-      build_tool = "make.exe"
-      , makefile_generator = "MSYS Makefiles"
-    )
+WINDOWS_BUILD_TOOLS <- list(
+  "MinGW" = c(
+    build_tool = "mingw32-make.exe"
+    , makefile_generator = "MinGW Makefiles"
+  )
+  , "MSYS2" = c(
+    build_tool = "make.exe"
+    , makefile_generator = "MSYS Makefiles"
   )
+)
 
-  if (use_mingw) {
-    windows_toolchain <- "MinGW"
-  } else if (use_msys2) {
+if (use_mingw) {
+  windows_toolchain <- "MinGW"
+} else if (use_msys2) {
+  windows_toolchain <- "MSYS2"
+} else {
+  # Rtools 4.0 moved from MinGW to MSYS toolchain. If user tries
+  # Visual Studio install but that fails, fall back to the toolchain
+  # supported in Rtools
+  if (R_ver >= 4.0) {
     windows_toolchain <- "MSYS2"
   } else {
-    # Rtools 4.0 moved from MinGW to MSYS toolchain. If user tries
-    # Visual Studio install but that fails, fall back to the toolchain
-    # supported in Rtools
-    if (R_ver >= 4.0) {
-      windows_toolchain <- "MSYS2"
-    } else {
-      windows_toolchain <- "MinGW"
-    }
+    windows_toolchain <- "MinGW"
   }
-  windows_build_tool <- WINDOWS_BUILD_TOOLS[[windows_toolchain]][["build_tool"]]
-  windows_makefile_generator <- WINDOWS_BUILD_TOOLS[[windows_toolchain]][["makefile_generator"]]
+}
+windows_build_tool <- WINDOWS_BUILD_TOOLS[[windows_toolchain]][["build_tool"]]
+windows_makefile_generator <- WINDOWS_BUILD_TOOLS[[windows_toolchain]][["makefile_generator"]]
 
-  if (use_gpu) {
-    cmake_args <- c(cmake_args, "-DUSE_GPU=ON")
-  }
-  cmake_args <- c(cmake_args, "-D__BUILD_FOR_R=ON")
+if (use_gpu) {
+  cmake_args <- c(cmake_args, "-DUSE_GPU=ON")
+}
+cmake_args <- c(cmake_args, "-D__BUILD_FOR_R=ON")
 
-  # Pass in R version, used to help find R executable for linking
-  R_version_string <- paste(
-    R.Version()[["major"]]
-    , R.Version()[["minor"]]
-    , sep = "."
-  )
-  r_version_arg <- sprintf("-DCMAKE_R_VERSION='%s'", R_version_string)
-  cmake_args <- c(cmake_args, r_version_arg)
+# Pass in R version, used to help find R executable for linking
+R_version_string <- paste(
+  R.Version()[["major"]]
+  , R.Version()[["minor"]]
+  , sep = "."
+)
+r_version_arg <- sprintf("-DCMAKE_R_VERSION='%s'", R_version_string)
+cmake_args <- c(cmake_args, r_version_arg)
 
-  # the checks below might already run `cmake -G`. If they do, set this flag
-  # to TRUE to avoid re-running it later
-  makefiles_already_generated <- FALSE
+# the checks below might already run `cmake -G`. If they do, set this flag
+# to TRUE to avoid re-running it later
+makefiles_already_generated <- FALSE
 
-  # Check if Windows installation (for gcc vs Visual Studio)
-  if (WINDOWS) {
-    if (!use_visual_studio) {
-      message(sprintf("Trying to build with %s", windows_toolchain))
+# Check if Windows installation (for gcc vs Visual Studio)
+if (WINDOWS) {
+  if (!use_visual_studio) {
+    message(sprintf("Trying to build with %s", windows_toolchain))
+    # Must build twice for Windows due sh.exe in Rtools
+    cmake_args <- c(cmake_args, "-G", shQuote(windows_makefile_generator))
+    .run_shell_command("cmake", c(cmake_args, ".."), strict = FALSE)
+    build_cmd <- windows_build_tool
+    build_args <- "_lightgbm"
+  } else {
+    visual_studio_succeeded <- .generate_vs_makefiles(cmake_args)
+    if (!isTRUE(visual_studio_succeeded)) {
+      warning(sprintf("Building with Visual Studio failed. Attempting with %s", windows_toolchain))
       # Must build twice for Windows due sh.exe in Rtools
       cmake_args <- c(cmake_args, "-G", shQuote(windows_makefile_generator))
       .run_shell_command("cmake", c(cmake_args, ".."), strict = FALSE)
       build_cmd <- windows_build_tool
       build_args <- "_lightgbm"
     } else {
-      visual_studio_succeeded <- .generate_vs_makefiles(cmake_args)
-      if (!isTRUE(visual_studio_succeeded)) {
-        warning(sprintf("Building with Visual Studio failed. Attempting with %s", windows_toolchain))
-        # Must build twice for Windows due sh.exe in Rtools
-        cmake_args <- c(cmake_args, "-G", shQuote(windows_makefile_generator))
-        .run_shell_command("cmake", c(cmake_args, ".."), strict = FALSE)
-        build_cmd <- windows_build_tool
-        build_args <- "_lightgbm"
-      } else {
-        build_cmd <- "cmake"
-        build_args <- c("--build", ".", "--target", "_lightgbm", "--config", "Release")
-        lib_folder <- file.path(source_dir, "Release", fsep = "/")
-        makefiles_already_generated <- TRUE
-      }
-    }
-  } else {
-      .run_shell_command("cmake", c(cmake_args, ".."))
+      build_cmd <- "cmake"
+      build_args <- c("--build", ".", "--target", "_lightgbm", "--config", "Release")
+      lib_folder <- file.path(source_dir, "Release", fsep = "/")
       makefiles_already_generated <- TRUE
+    }
   }
-
-  # generate build files
-  if (!makefiles_already_generated) {
+} else {
     .run_shell_command("cmake", c(cmake_args, ".."))
-  }
-
-  # R CMD check complains about the .NOTPARALLEL directive created in the cmake
-  # Makefile. We don't need it here anyway since targets are built serially, so trying
-  # to remove it with this hack
-  generated_makefile <- file.path(
-    build_dir
-    , "Makefile"
-  )
-  if (file.exists(generated_makefile)) {
-    makefile_txt <- readLines(
-      con = generated_makefile
-    )
-    makefile_txt <- gsub(
-      pattern = ".*NOTPARALLEL.*"
-      , replacement = ""
-      , x = makefile_txt
-    )
-    writeLines(
-      text = makefile_txt
-      , con = generated_makefile
-      , sep = "\n"
-    )
-  }
-
-  # build the library
-  message("Building lib_lightgbm")
-  .run_shell_command(build_cmd, build_args)
-  src <- file.path(lib_folder, paste0("lib_lightgbm", SHLIB_EXT), fsep = "/")
+    makefiles_already_generated <- TRUE
+}
 
-} else {
+# generate build files
+if (!makefiles_already_generated) {
+  .run_shell_command("cmake", c(cmake_args, ".."))
+}
 
-  # Has precompiled package
-  lib_folder <- file.path(R_PACKAGE_SOURCE, "../", fsep = "/")
-  shared_object_file <- file.path(
-    lib_folder
-    , paste0("lib_lightgbm", SHLIB_EXT)
-    , fsep = "/"
+# R CMD check complains about the .NOTPARALLEL directive created in the cmake
+# Makefile. We don't need it here anyway since targets are built serially, so trying
+# to remove it with this hack
+generated_makefile <- file.path(
+  build_dir
+  , "Makefile"
+)
+if (file.exists(generated_makefile)) {
+  makefile_txt <- readLines(
+    con = generated_makefile
   )
-  release_file <- file.path(
-    lib_folder
-    , paste0("Release/lib_lightgbm", SHLIB_EXT)
-    , fsep = "/"
+  makefile_txt <- gsub(
+    pattern = ".*NOTPARALLEL.*"
+    , replacement = ""
+    , x = makefile_txt
   )
-  windows_shared_object_file <- file.path(
-    lib_folder
-    , paste0("/windows/x64/DLL/lib_lightgbm", SHLIB_EXT)
-    , fsep = "/"
+  writeLines(
+    text = makefile_txt
+    , con = generated_makefile
+    , sep = "\n"
   )
-  if (file.exists(shared_object_file)) {
-    src <- shared_object_file
-  } else if (file.exists(release_file)) {
-    src <- release_file
-  } else {
-    # Expected result: installation will fail if it is not here or any other
-    src <- windows_shared_object_file
-  }
 }
 
+# build the library
+message("Building lib_lightgbm")
+.run_shell_command(build_cmd, build_args)
+src <- file.path(lib_folder, paste0("lib_lightgbm", SHLIB_EXT), fsep = "/")
+
 # Packages with install.libs.R need to copy some artifacts into the
 # expected places in the package structure.
 # see https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Package-subdirectories,