README.md 1.51 KB
Newer Older
Bruce MacDonald's avatar
Bruce MacDonald committed
1
# Ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
2

Jeffrey Morgan's avatar
Jeffrey Morgan committed
3
Ollama is a tool for running any large language model on any machine. It's designed to be easy to use and fast, supporting the largest number of models possible by using the fastest loader available for your platform and model.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
4

Jeffrey Morgan's avatar
Jeffrey Morgan committed
5
> _Note: this project is a work in progress. Certain models that can be run with `ollama` are intended for research and/or non-commercial use only._
Jeffrey Morgan's avatar
Jeffrey Morgan committed
6
7

## Install
Jeffrey Morgan's avatar
Jeffrey Morgan committed
8
9

```
Bruce MacDonald's avatar
Bruce MacDonald committed
10
pip install ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
11
12
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
13
14
15
16
```
docker run ollama/ollama
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
17
## Quickstart
Jeffrey Morgan's avatar
Jeffrey Morgan committed
18

Jeffrey Morgan's avatar
Jeffrey Morgan committed
19
20
21
22
To run a model, use `ollama run`:

```
ollama run orca-mini-3b
Jeffrey Morgan's avatar
Jeffrey Morgan committed
23
24
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
25
You can also run models from hugging face:
Bruce MacDonald's avatar
Bruce MacDonald committed
26

Jeffrey Morgan's avatar
Jeffrey Morgan committed
27
28
29
```
ollama run huggingface.co/TheBloke/orca_mini_3B-GGML
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
30

Jeffrey Morgan's avatar
Jeffrey Morgan committed
31
32
33
34
Or directly via downloaded model files:

```
ollama run ~/Downloads/orca-mini-13b.ggmlv3.q4_0.bin
Jeffrey Morgan's avatar
Jeffrey Morgan committed
35
36
37
38
39
```

## Python SDK

### Example
Bruce MacDonald's avatar
Bruce MacDonald committed
40
41

```python
Jeffrey Morgan's avatar
Jeffrey Morgan committed
42
import ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
43
ollama.generate("orca-mini-3b", "hi")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
44
45
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
46
### `ollama.generate(model, message)`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
47

Jeffrey Morgan's avatar
Jeffrey Morgan committed
48
Generate a completion
Jeffrey Morgan's avatar
Jeffrey Morgan committed
49

Bruce MacDonald's avatar
Bruce MacDonald committed
50
```python
Jeffrey Morgan's avatar
Jeffrey Morgan committed
51
ollama.generate("./llama-7b-ggml.bin", "hi")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
52
53
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
54
### `ollama.models()`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
55

Jeffrey Morgan's avatar
Jeffrey Morgan committed
56
List available local models
Bruce MacDonald's avatar
Bruce MacDonald committed
57

Jeffrey Morgan's avatar
Jeffrey Morgan committed
58
```python
Bruce MacDonald's avatar
Bruce MacDonald committed
59
models = ollama.models()
Jeffrey Morgan's avatar
Jeffrey Morgan committed
60
61
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
### `ollama.load(model)`

Manually a model for generation

```python
ollama.load("model")
```

### `ollama.unload(model)`

Unload a model

```python
ollama.unload("model")
```

Bruce MacDonald's avatar
Bruce MacDonald committed
78
79
80
### `ollama.pull(model)`

Download a model
Bruce MacDonald's avatar
Bruce MacDonald committed
81
82

```python
Bruce MacDonald's avatar
Bruce MacDonald committed
83
ollama.pull("huggingface.co/thebloke/llama-7b-ggml")
Bruce MacDonald's avatar
Bruce MacDonald committed
84
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
85

Michael Chiang's avatar
Michael Chiang committed
86
## Coming Soon
Bruce MacDonald's avatar
Bruce MacDonald committed
87

Jeffrey Morgan's avatar
Jeffrey Morgan committed
88
### `ollama.search("query")`
89

Bruce MacDonald's avatar
Bruce MacDonald committed
90
Search for compatible models that Ollama can run
91

Bruce MacDonald's avatar
Bruce MacDonald committed
92
93
94
```python
ollama.search("llama-7b")
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
95

Bruce MacDonald's avatar
Bruce MacDonald committed
96
97
98
## Documentation

- [Development](docs/development.md)