README.md 1.96 KB
Newer Older
Bruce MacDonald's avatar
Bruce MacDonald committed
1
# Ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
2

Jeffrey Morgan's avatar
Jeffrey Morgan committed
3
Run ai models locally.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
4

Jeffrey Morgan's avatar
Jeffrey Morgan committed
5
> _Note: this project is a work in progress. The features below are still in development_
Jeffrey Morgan's avatar
Jeffrey Morgan committed
6

Jeffrey Morgan's avatar
Jeffrey Morgan committed
7
**Features**
Jeffrey Morgan's avatar
Jeffrey Morgan committed
8

Jeffrey Morgan's avatar
Jeffrey Morgan committed
9
- Run models locally on macOS (Windows, Linux and other platforms coming soon)
Jeffrey Morgan's avatar
Jeffrey Morgan committed
10
- Ollama uses the fastest loader available for your platform and model (e.g. llama.cpp, Core ML and other loaders coming soon)
Jeffrey Morgan's avatar
Jeffrey Morgan committed
11
12
13
14
15
16
17
- Import models from local files
- Find and download models on Hugging Face and other sources (coming soon)
- Support for running and switching between multiple models at a time (coming soon)
- Native desktop experience (coming soon)
- Built-in memory (coming soon)

## Install
Jeffrey Morgan's avatar
Jeffrey Morgan committed
18
19

```
Bruce MacDonald's avatar
Bruce MacDonald committed
20
pip install ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
21
22
```

Michael Yang's avatar
Michael Yang committed
23
24
25
26
27
28
29
30
31
## Install From Source

```
git clone git@github.com:jmorganca/ollama ollama
cd ollama
pip install -r requirements.txt
pip install -e .
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
32
## Quickstart
Jeffrey Morgan's avatar
Jeffrey Morgan committed
33
34

```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
35
36
% ollama run huggingface.co/TheBloke/orca_mini_3B-GGML
Pulling huggingface.co/TheBloke/orca_mini_3B-GGML...
Bruce MacDonald's avatar
Bruce MacDonald committed
37
Downloading [================           ] 66.67% 11.8MiB/s
Jeffrey Morgan's avatar
Jeffrey Morgan committed
38

Jeffrey Morgan's avatar
Jeffrey Morgan committed
39
40
41
...
...
...
Bruce MacDonald's avatar
Bruce MacDonald committed
42

Jeffrey Morgan's avatar
Jeffrey Morgan committed
43
44
45
46
47
48
49
50
> Hello

Hello, how may I help you?
```

## Python SDK

### Example
Bruce MacDonald's avatar
Bruce MacDonald committed
51
52

```python
Jeffrey Morgan's avatar
Jeffrey Morgan committed
53
import ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
54
ollama.generate("orca-mini-3b", "hi")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
55
56
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
57
### `ollama.generate(model, message)`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
58

Jeffrey Morgan's avatar
Jeffrey Morgan committed
59
Generate a completion
Jeffrey Morgan's avatar
Jeffrey Morgan committed
60

Bruce MacDonald's avatar
Bruce MacDonald committed
61
```python
Jeffrey Morgan's avatar
Jeffrey Morgan committed
62
ollama.generate("./llama-7b-ggml.bin", "hi")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
63
64
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
65
### `ollama.models()`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
66

Jeffrey Morgan's avatar
Jeffrey Morgan committed
67
List available local models
Bruce MacDonald's avatar
Bruce MacDonald committed
68

Jeffrey Morgan's avatar
Jeffrey Morgan committed
69
```python
Bruce MacDonald's avatar
Bruce MacDonald committed
70
models = ollama.models()
Jeffrey Morgan's avatar
Jeffrey Morgan committed
71
72
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
73
### `ollama.serve()`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
74

Bruce MacDonald's avatar
Bruce MacDonald committed
75
Serve the ollama http server
Jeffrey Morgan's avatar
Jeffrey Morgan committed
76

Jeffrey Morgan's avatar
Jeffrey Morgan committed
77
78
79
80
```
ollama.serve()
```

Bruce MacDonald's avatar
Bruce MacDonald committed
81
### `ollama.add(filepath)`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
82

Bruce MacDonald's avatar
Bruce MacDonald committed
83
Add a model by importing from a file
Bruce MacDonald's avatar
Bruce MacDonald committed
84
85

```python
Bruce MacDonald's avatar
Bruce MacDonald committed
86
ollama.add("./path/to/model")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
87
88
```

Jeffrey Morgan's avatar
Jeffrey Morgan committed
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
### `ollama.load(model)`

Manually a model for generation

```python
ollama.load("model")
```

### `ollama.unload(model)`

Unload a model

```python
ollama.unload("model")
```

Bruce MacDonald's avatar
Bruce MacDonald committed
105
106
107
### `ollama.pull(model)`

Download a model
Bruce MacDonald's avatar
Bruce MacDonald committed
108
109

```python
Bruce MacDonald's avatar
Bruce MacDonald committed
110
ollama.pull("huggingface.co/thebloke/llama-7b-ggml")
Bruce MacDonald's avatar
Bruce MacDonald committed
111
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
112

Michael Chiang's avatar
Michael Chiang committed
113
## Coming Soon
Bruce MacDonald's avatar
Bruce MacDonald committed
114

Jeffrey Morgan's avatar
Jeffrey Morgan committed
115
### `ollama.search("query")`
116

Bruce MacDonald's avatar
Bruce MacDonald committed
117
Search for compatible models that Ollama can run
118

Bruce MacDonald's avatar
Bruce MacDonald committed
119
120
121
```python
ollama.search("llama-7b")
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
122

Bruce MacDonald's avatar
Bruce MacDonald committed
123
124
125
## Documentation

- [Development](docs/development.md)