"vscode:/vscode.git/clone" did not exist on "81ca2a1df385eadc76354fc777ea71837962079e"
development.md 3.83 KB
Newer Older
Bruce MacDonald's avatar
Bruce MacDonald committed
1
2
# Development

3
4
5
6
- Install cmake or (optionally, required tools for GPUs)
- run `go generate ./...`
- run `go build .`

Jeffrey Morgan's avatar
Jeffrey Morgan committed
7
Install required tools:
Bruce MacDonald's avatar
Bruce MacDonald committed
8

9
10
11
12
- cmake version 3.24 or higher
- go version 1.20 or higher
- gcc version 11.4.0 or higher

13
```bash
14
brew install go cmake gcc
Bruce MacDonald's avatar
Bruce MacDonald committed
15
16
```

17
18
19
20
21
22
23
Optionally enable debugging and more verbose logging:

```bash
export CGO_CFLAGS="-g"
```

Get the required libraries and build the native LLM code:
24

25
```bash
26
go generate ./...
27
28
```

Matt Williams's avatar
Matt Williams committed
29
Then build ollama:
Bruce MacDonald's avatar
Bruce MacDonald committed
30

31
```bash
Matt Williams's avatar
Matt Williams committed
32
go build .
Bruce MacDonald's avatar
Bruce MacDonald committed
33
34
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
35
Now you can run `ollama`:
Bruce MacDonald's avatar
Bruce MacDonald committed
36

37
```bash
Jeffrey Morgan's avatar
Jeffrey Morgan committed
38
./ollama
Bruce MacDonald's avatar
Bruce MacDonald committed
39
```
40

41
### Linux
42

43
#### Linux CUDA (NVIDIA)
44
45
46

*Your operating system distribution may already have packages for NVIDIA CUDA. Distro packages are often preferable, but instructions are distro-specific. Please consult distro-specific docs for dependencies if available!*

47
48
49
50
51
52
53
54
Install `cmake` and `golang` as well as [NVIDIA CUDA](https://developer.nvidia.com/cuda-downloads)
development and runtime packages. 

Typically the build scripts will auto-detect CUDA, however, if your Linux distro
or installation approach uses unusual paths, you can specify the location by
specifying an environment variable `CUDA_LIB_DIR` to the location of the shared
libraries, and `CUDACXX` to the location of the nvcc compiler.

55
Then generate dependencies:
56

57
58
59
```
go generate ./...
```
60

61
Then build the binary:
62

63
64
65
66
```
go build .
```

67
68
#### Linux ROCm (AMD)

69
70
71
*Your operating system distribution may already have packages for AMD ROCm and CLBlast. Distro packages are often preferable, but instructions are distro-specific. Please consult distro-specific docs for dependencies if available!*

Install [CLBlast](https://github.com/CNugteren/CLBlast/blob/master/doc/installation.md) and [ROCm](https://rocm.docs.amd.com/en/latest/deploy/linux/quick_start.html) developement packages first, as well as `cmake` and `golang`.
72
73
74
75
76
77

Typically the build scripts will auto-detect ROCm, however, if your Linux distro
or installation approach uses unusual paths, you can specify the location by
specifying an environment variable `ROCM_PATH` to the location of the ROCm
install (typically `/opt/rocm`), and `CLBlast_DIR` to the location of the
CLBlast install (typically `/usr/lib/cmake/CLBlast`).
78

79
```
80
go generate ./...
81
```
82

83
Then build the binary:
84

85
86
87
88
89
90
```
go build .
```

ROCm requires elevated privileges to access the GPU at runtime.  On most distros you can add your user account to the `render` group, or run as root.

91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
#### Advanced CPU Settings

By default, running `go generate ./...` will compile a few different variations
of the LLM library based on common CPU families and vector math capabilities,
including a lowest-common-denominator which should run on almost any 64 bit CPU
somewhat slowly.  At runtime, Ollama will auto-detect the optimal variation to
load.  If you would like to build a CPU-based build customized for your
processor, you can set `OLLAMA_CUSTOM_CPU_DEFS` to the llama.cpp flags you would
like to use.  For example, to compile an optimized binary for an Intel i9-9880H,
you might use:

```
OLLAMA_CUSTOM_CPU_DEFS="-DLLAMA_AVX=on -DLLAMA_AVX2=on -DLLAMA_F16C=on -DLLAMA_FMA=on" go generate ./...
go build .
```

107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
#### Containerized Linux Build

If you have Docker available, you can build linux binaries with `./scripts/build_linux.sh` which has the CUDA and ROCm dependencies included.  The resulting binary is placed in `./dist`


### Windows

Note: The windows build for Ollama is still under development.

Install required tools:

- MSVC toolchain - C/C++ and cmake as minimal requirements
- go version 1.20 or higher
- MinGW (pick one variant) with GCC.
  - <https://www.mingw-w64.org/>
  - <https://www.msys2.org/>

```powershell
$env:CGO_ENABLED="1"

go generate ./...

go build .
```

#### Windows CUDA (NVIDIA)

In addition to the common Windows development tools described above, install:
135

136
- [NVIDIA CUDA](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html)