README.md 2.67 KB
Newer Older
Michael Chiang's avatar
Michael Chiang committed
1
2
<div align="center">
  <picture>
Michael Chiang's avatar
Michael Chiang committed
3
4
    <source media="(prefers-color-scheme: dark)" height="200px" srcset="https://github.com/jmorganca/ollama/assets/3325447/56ea1849-1284-4645-8970-956de6e51c3c">
    <img alt="logo" height="200px" src="https://github.com/jmorganca/ollama/assets/3325447/0d0b44e2-8f4a-4e99-9b52-a5c1c741c8f7">
Michael Chiang's avatar
Michael Chiang committed
5
6
  </picture>
</div>
Jeffrey Morgan's avatar
Jeffrey Morgan committed
7

Bruce MacDonald's avatar
Bruce MacDonald committed
8
# Ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
9

10
[![Discord](https://dcbadge.vercel.app/api/server/ollama?style=flat&compact=true)](https://discord.gg/ollama)
11
12

Create, run, and share large language models (LLMs). Ollama bundles a model’s weights, configuration, prompts, and more into self-contained packages that can run on any machine.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
13

14
> Note: Ollama is in early preview. Please report any issues you find.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
15

16
17
18
19
20
21
## Download

- [Download](https://ollama.ai/download) for macOS on Apple Silicon (Intel coming soon)
- Download for Windows and Linux (coming soon)
- Build [from source](#building)

22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
## Quickstart

To run and chat with [Llama 2](https://ai.meta.com/llama), the new model by Meta:

```
ollama run llama2
```

## Model library

Ollama includes a library of open-source, pre-trained models. More models are coming soon. You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.

| Model                    | Parameters | Size  | Download                    |
| ------------------------ | ---------- | ----- | --------------------------- |
| Llama2                   | 7B         | 3.8GB | `ollama pull llama2`        |
| Llama2 13B               | 13B        | 7.3GB | `ollama pull llama2:13b`    |
| Orca Mini                | 3B         | 1.9GB | `ollama pull orca`          |
| Vicuna                   | 7B         | 3.8GB | `ollama pull vicuna`        |
| Nous-Hermes              | 13B        | 7.3GB | `ollama pull nous-hermes`   |
| Wizard Vicuna Uncensored | 13B        | 7.3GB | `ollama pull wizard-vicuna` |

43
## Examples
Jeffrey Morgan's avatar
Jeffrey Morgan committed
44

45
### Run a model
46

Jeffrey Morgan's avatar
Jeffrey Morgan committed
47
```
48
49
50
ollama run llama2
>>> hi
Hello! How can I help you today?
Jeffrey Morgan's avatar
Jeffrey Morgan committed
51
52
```

53
54
55
56
57
### Create a custom character model

Pull a base model:

```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
58
ollama pull llama2
59
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
60

61
Create a `Modelfile`:
Jeffrey Morgan's avatar
Jeffrey Morgan committed
62

Jeffrey Morgan's avatar
Jeffrey Morgan committed
63
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
64
FROM llama2
65
66
67
68
69

# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1

# set the system prompt
Jeffrey Morgan's avatar
Jeffrey Morgan committed
70
SYSTEM """
Jeffrey Morgan's avatar
Jeffrey Morgan committed
71
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
72
"""
Jeffrey Morgan's avatar
Jeffrey Morgan committed
73
```
Bruce MacDonald's avatar
Bruce MacDonald committed
74

75
Next, create and run the model:
Bruce MacDonald's avatar
Bruce MacDonald committed
76
77

```
78
79
80
81
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
Bruce MacDonald's avatar
Bruce MacDonald committed
82
83
```

84
For more examples, see the [examples](./examples) directory.
85
86

### Pull a model from the registry
Jeffrey Morgan's avatar
Jeffrey Morgan committed
87

88
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
89
ollama pull orca
90
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
91

Jeffrey Morgan's avatar
Jeffrey Morgan committed
92
93
94
## Building

```
Michael Yang's avatar
Michael Yang committed
95
go build .
Jeffrey Morgan's avatar
Jeffrey Morgan committed
96
97
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
98
To run it start the server:
Bruce MacDonald's avatar
Bruce MacDonald committed
99

Jeffrey Morgan's avatar
Jeffrey Morgan committed
100
```
DavidZirinsky's avatar
DavidZirinsky committed
101
./ollama serve &
Jeffrey Morgan's avatar
Jeffrey Morgan committed
102
103
104
105
106
```

Finally, run a model!

```
107
./ollama run llama2
Jeffrey Morgan's avatar
Jeffrey Morgan committed
108
```