cpu.apple.inc.md 2.69 KB
Newer Older
1
# --8<-- [start:installation]
2

3
vLLM has experimental support for macOS with Apple Silicon. For now, users must build from source to natively run on macOS.
4
5
6

Currently the CPU implementation for macOS supports FP32 and FP16 datatypes.

7
8
!!! warning
    There are no pre-built wheels or images for this device, so you must build vLLM from source.
9

10
11
# --8<-- [end:installation]
# --8<-- [start:requirements]
12

13
14
- OS: `macOS Sonoma` or later
- SDK: `XCode 15.4` or later with Command Line Tools
15
- Compiler: `Apple Clang >= 15.0.0`
16

17
18
# --8<-- [end:requirements]
# --8<-- [start:set-up-using-python]
19

20
21
# --8<-- [end:set-up-using-python]
# --8<-- [start:pre-built-wheels]
22

23
24
# --8<-- [end:pre-built-wheels]
# --8<-- [start:build-wheel-from-source]
25

26
After installation of XCode and the Command Line Tools, which include Apple Clang, execute the following commands to build and install vLLM from source.
27

28
```bash
29
30
git clone https://github.com/vllm-project/vllm.git
cd vllm
31
32
uv pip install -r requirements/cpu.txt
uv pip install -e .
33
34
```

35
!!! note
36
    On macOS the `VLLM_TARGET_DEVICE` is automatically set to `cpu`, which is currently the only supported device.
37

38
!!! example "Troubleshooting"
39
    If the build fails with errors like the following where standard C++ headers cannot be found, try to remove and reinstall your
40
41
42
43
44
45
46
47
48
49
50
51
52
53
    [Command Line Tools for Xcode](https://developer.apple.com/download/all/).

    ```text
    [...] fatal error: 'map' file not found
            1 | #include <map>
                |          ^~~~~
        1 error generated.
        [2/8] Building CXX object CMakeFiles/_C.dir/csrc/cpu/pos_encoding.cpp.o

    [...] fatal error: 'cstddef' file not found
            10 | #include <cstddef>
                |          ^~~~~~~~~
        1 error generated.
    ```
54

55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
    ---

    If the build fails with C++11/C++17 compatibility errors like the following, the issue is that the build system is defaulting to an older C++ standard:

    ```text
    [...] error: 'constexpr' is not a type
    [...] error: expected ';' before 'constexpr'
    [...] error: 'constexpr' does not name a type
    ```

    **Solution**: Your compiler might be using an older C++ standard. Edit `cmake/cpu_extension.cmake` and add `set(CMAKE_CXX_STANDARD 17)` before `set(CMAKE_CXX_STANDARD_REQUIRED ON)`.

    To check your compiler's C++ standard support:
    ```bash
    clang++ -std=c++17 -pedantic -dM -E -x c++ /dev/null | grep __cplusplus
    ```
    On Apple Clang 16 you should see: `#define __cplusplus 201703L`

73
74
# --8<-- [end:build-wheel-from-source]
# --8<-- [start:pre-built-images]
75

76
77
# --8<-- [end:pre-built-images]
# --8<-- [start:build-image-from-source]
78

79
80
81
# --8<-- [end:build-image-from-source]
# --8<-- [start:extra-information]
# --8<-- [end:extra-information]