README.md 2.6 KB
Newer Older
Michael Chiang's avatar
Michael Chiang committed
1
2
3
4
5
6
<div align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" height="200px" srcset="https://github.com/jmorganca/ollama/assets/3325447/318048d2-b2dd-459c-925a-ac8449d5f02c">
    <img alt="logo" height="200px" src="https://github.com/jmorganca/ollama/assets/3325447/c7d6e15f-7f4d-4776-b568-c084afa297c2">
  </picture>
</div>
Jeffrey Morgan's avatar
Jeffrey Morgan committed
7

Bruce MacDonald's avatar
Bruce MacDonald committed
8
# Ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
9

10
[![Discord](https://dcbadge.vercel.app/api/server/ollama?style=flat&compact=true)](https://discord.gg/ollama)
11
12

Create, run, and share large language models (LLMs). Ollama bundles a model’s weights, configuration, prompts, and more into self-contained packages that can run on any machine.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
13

14
> Note: Ollama is in early preview. Please report any issues you find.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
15

16
17
18
19
20
21
## Download

- [Download](https://ollama.ai/download) for macOS on Apple Silicon (Intel coming soon)
- Download for Windows and Linux (coming soon)
- Build [from source](#building)

22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
## Quickstart

To run and chat with [Llama 2](https://ai.meta.com/llama), the new model by Meta:

```
ollama run llama2
```

## Model library

Ollama includes a library of open-source, pre-trained models. More models are coming soon. You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.

| Model                    | Parameters | Size  | Download                    |
| ------------------------ | ---------- | ----- | --------------------------- |
| Llama2                   | 7B         | 3.8GB | `ollama pull llama2`        |
| Llama2 13B               | 13B        | 7.3GB | `ollama pull llama2:13b`    |
| Orca Mini                | 3B         | 1.9GB | `ollama pull orca`          |
| Vicuna                   | 7B         | 3.8GB | `ollama pull vicuna`        |
| Nous-Hermes              | 13B        | 7.3GB | `ollama pull nous-hermes`   |
| Wizard Vicuna Uncensored | 13B        | 7.3GB | `ollama pull wizard-vicuna` |

43
## Examples
Jeffrey Morgan's avatar
Jeffrey Morgan committed
44

45
### Run a model
46

Jeffrey Morgan's avatar
Jeffrey Morgan committed
47
```
48
49
50
ollama run llama2
>>> hi
Hello! How can I help you today?
Jeffrey Morgan's avatar
Jeffrey Morgan committed
51
52
```

53
54
55
56
57
58
59
### Create a custom character model

Pull a base model:

```
ollama pull orca
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
60

61
Create a `Modelfile`:
Jeffrey Morgan's avatar
Jeffrey Morgan committed
62

Jeffrey Morgan's avatar
Jeffrey Morgan committed
63
```
64
FROM orca
65
PROMPT """
66
### System:
Jeffrey Morgan's avatar
Jeffrey Morgan committed
67
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
68

69
70
71
72
### User:
{{ .Prompt }}

### Response:
73
"""
Jeffrey Morgan's avatar
Jeffrey Morgan committed
74
```
Bruce MacDonald's avatar
Bruce MacDonald committed
75

76
Next, create and run the model:
Bruce MacDonald's avatar
Bruce MacDonald committed
77
78

```
79
80
81
82
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
Bruce MacDonald's avatar
Bruce MacDonald committed
83
84
```

85
For more examples, see the [examples](./examples) directory.
86
87

### Pull a model from the registry
Jeffrey Morgan's avatar
Jeffrey Morgan committed
88

89
90
91
```
ollama pull nous-hermes
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
92

Jeffrey Morgan's avatar
Jeffrey Morgan committed
93
94
95
## Building

```
Michael Yang's avatar
Michael Yang committed
96
go build .
Jeffrey Morgan's avatar
Jeffrey Morgan committed
97
98
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
99
To run it start the server:
Bruce MacDonald's avatar
Bruce MacDonald committed
100

Jeffrey Morgan's avatar
Jeffrey Morgan committed
101
```
DavidZirinsky's avatar
DavidZirinsky committed
102
./ollama serve &
Jeffrey Morgan's avatar
Jeffrey Morgan committed
103
104
105
106
107
```

Finally, run a model!

```
108
./ollama run llama2
Jeffrey Morgan's avatar
Jeffrey Morgan committed
109
```