installation.mdx

# Installation

bitsandbytes is only supported on CUDA GPUs for CUDA versions **11.0 - 12.3**.

The latest version of bitsandbytes (v0.43.0) builds on:

| OS | CUDA | Compiler |
|---|---|---|
| Linux | 11.7 - 12.3 | GCC 11.4 |
|  | 12.4+ | GCC 13.2 |
| Windows | 11.7 - 12.4 | MSVC 19.38+ (VS2022 17.8.0+) |

> [!TIP]
> MacOS support is still a work in progress! Subscribe to this [issue](https://github.com/TimDettmers/bitsandbytes/issues/1020) to get notified about discussions and to track the integration progress.

For Linux systems, make sure your hardware meets the following requirements to use bitsandbytes features.

| **Feature** | **Hardware requirement** |
|---|---|
| LLM.int8() | NVIDIA Turing (RTX 20 series, T4) or Ampere (RTX 30 series, A4-A100) GPUs |
| 8-bit optimizers/quantization | NVIDIA Kepler (GTX 780 or newer) |

> [!WARNING]
> bitsandbytes >= 0.39.1 no longer includes Kepler binaries in pip installations. This requires manual compilation, and you should follow the general steps and use `cuda11x_nomatmul_kepler` for Kepler-targeted compilation.

To install from PyPI.

```bash
pip install bitsandbytes
```

## Compile from source

For Linux and Windows systems, you can compile bitsandbytes from source. Installing from source allows for more build options with different CMake configurations.

<hfoptions id="source">
<hfoption id="Linux">

To compile from source, you need CMake >= **3.22.1** and Python >= **3.8** installed. Make sure you have a compiler installed to compile C++ (gcc, make, headers, etc.). For example, to install a compiler and CMake on Ubuntu:

```bash
apt-get install -y build-essential cmake
```

You should also install CUDA Toolkit by following the [NVIDIA CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html) guide from NVIDIA. The current expected CUDA Toolkit version is **11.1+** and it is recommended to install **GCC >= 7.3** and required to have at least **GCC >= 6**.

Refer to the following table if you're using another CUDA Toolkit version.

| CUDA Toolkit | GCC |
|---|---|
| >= 11.4.1 | >= 11 |
| >= 12.0 | >= 12 |
| >= 12.4 | >= 13 |

Now to install the bitsandbytes package from source, run the following commands:

```bash
git clone https://github.com/TimDettmers/bitsandbytes.git && cd bitsandbytes/
pip install -r requirements-dev.txt
cmake -DCOMPUTE_BACKEND=cuda -S .
make
pip install .
```

> [!TIP]
> If you have multiple versions of CUDA installed or installed it in a non-standard location, please refer to CMake CUDA documentation for how to configure the CUDA compiler.

</hfoption>
<hfoption id="Windows">

Windows systems require Visual Studio with C++ support as well as an installation of the CUDA SDK.

To compile from source, you need CMake >= **3.22.1** and Python >= **3.8** installed. You should also install CUDA Toolkit by following the [CUDA Installation Guide for Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html) guide from NVIDIA.

Refer to the following table if you're using another CUDA Toolkit version.

| CUDA Toolkit | MSVC |
|---|---|
| >= 11.6 | 19.30+ (VS2022) |

```bash
git clone https://github.com/TimDettmers/bitsandbytes.git && cd bitsandbytes/
pip install -r requirements-dev.txt
cmake -DCOMPUTE_BACKEND=cuda -S .
cmake --build . --config Release
python -m build --wheel
```

Big thanks to [wkpark](https://github.com/wkpark), [Jamezo97](https://github.com/Jamezo97), [rickardp](https://github.com/rickardp), [akx](https://github.com/akx) for their amazing contributions to make bitsandbytes compatible with Windows.

</hfoption>
</hfoptions>

## PyTorch CUDA versions

Some bitsandbytes features may need a newer CUDA version than the one currently supported by PyTorch binaries from Conda and pip. In this case, you should follow these instructions to load a precompiled bitsandbytes binary.

1. Determine the path of the CUDA version you want to use. Common paths include:

* `/usr/local/cuda`
* `/usr/local/cuda-XX.X` where `XX.X` is the CUDA version number

Then locally install the CUDA version you need with this script from bitsandbytes:

```bash
wget https://raw.githubusercontent.com/TimDettmers/bitsandbytes/main/install_cuda.sh
# Syntax cuda_install CUDA_VERSION INSTALL_PREFIX EXPORT_TO_BASH
#   CUDA_VERSION in {110, 111, 112, 113, 114, 115, 116, 117, 118, 120, 121, 122, 123, 124}
#   EXPORT_TO_BASH in {0, 1} with 0=False and 1=True

# For example, the following installs CUDA 11.7 to ~/local/cuda-11.7 and exports the path to your .bashrc

bash install_cuda.sh 117 ~/local 1
```

2. Set the environment variables `BNB_CUDA_VERSION` and `LD_LIBRARY_PATH` by manually overriding the CUDA version installed by PyTorch.

> [!TIP]
> It is recommended to add the following lines to the `.bashrc` file to make them permanent.

```bash
export BNB_CUDA_VERSION=<VERSION>
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<PATH>
```

For example, to use a local install path:

```bash
export BNB_CUDA_VERSION=117
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/tim/local/cuda-11.7
```

3. Now when you launch bitsandbytes with these environment variables, the PyTorch CUDA version is overridden by the new CUDA version (in this example, version 11.7) and a different bitsandbytes library is loaded.