development.md 4.52 KB
Newer Older
Bruce MacDonald's avatar
Bruce MacDonald committed
1
2
# Development

Michael Yang's avatar
Michael Yang committed
3
Install prerequisites:
4

Michael Yang's avatar
Michael Yang committed
5
- [Go](https://go.dev/doc/install)
6
- C/C++ Compiler e.g. Clang on macOS, [TDM-GCC](https://github.com/jmeubank/tdm-gcc/releases/latest) (Windows amd64) or [llvm-mingw](https://github.com/mstorsjo/llvm-mingw) (Windows arm64), GCC/Clang on Linux.
7

Michael Yang's avatar
Michael Yang committed
8
Then build and run Ollama from the root directory of the repository:
9

10
```shell
Michael Yang's avatar
Michael Yang committed
11
go run . serve
12
13
```

14
15
16
17
> [!NOTE]
> Ollama includes native code compiled with CGO.  From time to time these data structures can change and CGO can get out of sync resulting in unexpected crashes.  You can force a full build of the native code by running `go clean -cache` first. 


Michael Yang's avatar
Michael Yang committed
18
## macOS (Apple Silicon)
19

Michael Yang's avatar
Michael Yang committed
20
macOS Apple Silicon supports Metal which is built-in to the Ollama binary. No additional steps are required.
21

Michael Yang's avatar
Michael Yang committed
22
## macOS (Intel)
23

Michael Yang's avatar
Michael Yang committed
24
Install prerequisites:
25

Michael Yang's avatar
Michael Yang committed
26
- [CMake](https://cmake.org/download/) or `brew install cmake`
27

Michael Yang's avatar
Michael Yang committed
28
Then, configure and build the project:
29

30
```shell
Michael Yang's avatar
Michael Yang committed
31
32
cmake -B build
cmake --build build
33
34
```

Michael Yang's avatar
Michael Yang committed
35
Lastly, run Ollama:
36

37
```shell
Michael Yang's avatar
Michael Yang committed
38
go run . serve
39
40
```

Michael Yang's avatar
Michael Yang committed
41
## Windows
42

Michael Yang's avatar
Michael Yang committed
43
Install prerequisites:
44

Michael Yang's avatar
Michael Yang committed
45
46
47
- [CMake](https://cmake.org/download/)
- [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/) including the Native Desktop Workload
- (Optional) AMD GPU support
Chuanhui Liu's avatar
Chuanhui Liu committed
48
    - [ROCm](https://rocm.docs.amd.com/en/latest/)
Michael Yang's avatar
Michael Yang committed
49
50
51
    - [Ninja](https://github.com/ninja-build/ninja/releases)
- (Optional) NVIDIA GPU support
    - [CUDA SDK](https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64&target_version=11&target_type=exe_network)
52
53
- (Optional) VULKAN GPU support
    - [VULKAN SDK](https://vulkan.lunarg.com/sdk/home) - useful for AMD/Intel GPUs
54

Michael Yang's avatar
Michael Yang committed
55
Then, configure and build the project:
56

57
```shell
Michael Yang's avatar
Michael Yang committed
58
59
cmake -B build
cmake --build build --config Release
60
61
```

62
63
64
65
66
67
68
69
70
71
72
> Building for Vulkan requires VULKAN_SDK environment variable:
> 
> PowerShell
> ```powershell
> $env:VULKAN_SDK="C:\VulkanSDK\<version>"
> ```
> CMD
> ```cmd
> set VULKAN_SDK=C:\VulkanSDK\<version>
> ```

73
74
75
76
77
78
79
80
> [!IMPORTANT]
> Building for ROCm requires additional flags:
> ```
> cmake -B build -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
> cmake --build build --config Release
> ```


81

Michael Yang's avatar
Michael Yang committed
82
Lastly, run Ollama:
83

84
```shell
Michael Yang's avatar
Michael Yang committed
85
86
go run . serve
```
87

Michael Yang's avatar
Michael Yang committed
88
## Windows (ARM)
89

Daniel Hiltgen's avatar
Daniel Hiltgen committed
90
Windows ARM does not support additional acceleration libraries at this time.  Do not use cmake, simply `go run` or `go build`.
91

Michael Yang's avatar
Michael Yang committed
92
## Linux
93

Michael Yang's avatar
Michael Yang committed
94
Install prerequisites:
95

Michael Yang's avatar
Michael Yang committed
96
97
98
99
100
- [CMake](https://cmake.org/download/) or `sudo apt install cmake` or `sudo dnf install cmake`
- (Optional) AMD GPU support
    - [ROCm](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/quick-start.html)
- (Optional) NVIDIA GPU support
    - [CUDA SDK](https://developer.nvidia.com/cuda-downloads)
101
102
103
- (Optional) VULKAN GPU support
    - [VULKAN SDK](https://vulkan.lunarg.com/sdk/home) - useful for AMD/Intel GPUs
    - Or install via package manager: `sudo apt install vulkan-sdk` (Ubuntu/Debian) or `sudo dnf install vulkan-sdk` (Fedora/CentOS)
Michael Yang's avatar
Michael Yang committed
104
105
> [!IMPORTANT]
> Ensure prerequisites are in `PATH` before running CMake.
106
107


Michael Yang's avatar
Michael Yang committed
108
Then, configure and build the project:
109

110
```shell
Michael Yang's avatar
Michael Yang committed
111
112
cmake -B build
cmake --build build
113
114
```

Michael Yang's avatar
Michael Yang committed
115
Lastly, run Ollama:
116

117
```shell
Michael Yang's avatar
Michael Yang committed
118
go run . serve
119
120
```

Michael Yang's avatar
Michael Yang committed
121
## Docker
122

123
```shell
Michael Yang's avatar
Michael Yang committed
124
docker build .
125
126
```

Michael Yang's avatar
Michael Yang committed
127
128
### ROCm

129
```shell
Michael Yang's avatar
Michael Yang committed
130
docker build --build-arg FLAVOR=rocm .
131
132
```

Michael Yang's avatar
Michael Yang committed
133
134
135
## Running tests

To run tests, use `go test`:
136

137
```shell
Michael Yang's avatar
Michael Yang committed
138
go test ./...
139
```
140

141
> NOTE: In rare circumstances, you may need to change a package using the new
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
> "synctest" package in go1.24.
>
> If you do not have the "synctest" package enabled, you will not see build or
> test failures resulting from your change(s), if any, locally, but CI will
> break.
>
> If you see failures in CI, you can either keep pushing changes to see if the
> CI build passes, or you can enable the "synctest" package locally to see the
> failures before pushing.
>
> To enable the "synctest" package for testing, run the following command:
>
> ```shell
> GOEXPERIMENT=synctest go test ./...
> ```
>
> If you wish to enable synctest for all go commands, you can set the
> `GOEXPERIMENT` environment variable in your shell profile or by using:
>
> ```shell
> go env -w GOEXPERIMENT=synctest
> ```
>
> Which will enable the "synctest" package for all go commands without needing
> to set it for all shell sessions.
>
> The synctest package is not required for production builds.

170
171
172
173
174
175
176
177
178
## Library detection

Ollama looks for acceleration libraries in the following paths relative to the `ollama` executable:

* `./lib/ollama` (Windows)
* `../lib/ollama` (Linux)
* `.` (macOS)
* `build/lib/ollama` (for development)

Chuanhui Liu's avatar
Chuanhui Liu committed
179
If the libraries are not found, Ollama will not run with any acceleration libraries.