README.md 1.05 KB
Newer Older
Bruce MacDonald's avatar
Bruce MacDonald committed
1
# Ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
2

Jeffrey Morgan's avatar
Jeffrey Morgan committed
3
- Run models easily
Bruce MacDonald's avatar
Bruce MacDonald committed
4
- Download, manage and import models
Jeffrey Morgan's avatar
Jeffrey Morgan committed
5

Bruce MacDonald's avatar
Bruce MacDonald committed
6
## Install
Jeffrey Morgan's avatar
Jeffrey Morgan committed
7
8

```
Bruce MacDonald's avatar
Bruce MacDonald committed
9
pip install ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
10
11
```

Bruce MacDonald's avatar
Bruce MacDonald committed
12
## Example quickstart
Jeffrey Morgan's avatar
Jeffrey Morgan committed
13

Bruce MacDonald's avatar
Bruce MacDonald committed
14
15
```python
import ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
16
ollama.generate("./llama-7b-ggml.bin", "hi")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
17
18
```

Bruce MacDonald's avatar
Bruce MacDonald committed
19
## Reference
Jeffrey Morgan's avatar
Jeffrey Morgan committed
20

Bruce MacDonald's avatar
Bruce MacDonald committed
21
22
### `ollama.load`

Jeffrey Morgan's avatar
Jeffrey Morgan committed
23
Load a model for generation
Bruce MacDonald's avatar
Bruce MacDonald committed
24
25
26

```python
ollama.load("model name")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
27
28
```

Bruce MacDonald's avatar
Bruce MacDonald committed
29
### `ollama.generate("message")`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
30

Bruce MacDonald's avatar
Bruce MacDonald committed
31
Generate a completion
Jeffrey Morgan's avatar
Jeffrey Morgan committed
32

Bruce MacDonald's avatar
Bruce MacDonald committed
33
34
```python
ollama.generate(model, "hi")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
35
36
```

Bruce MacDonald's avatar
Bruce MacDonald committed
37
### `ollama.models`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
38

Jeffrey Morgan's avatar
Jeffrey Morgan committed
39
List available local models
Bruce MacDonald's avatar
Bruce MacDonald committed
40
41
42

```
models = ollama.models()
Jeffrey Morgan's avatar
Jeffrey Morgan committed
43
44
```

Bruce MacDonald's avatar
Bruce MacDonald committed
45
### `ollama.serve`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
46

Bruce MacDonald's avatar
Bruce MacDonald committed
47
Serve the ollama http server
Jeffrey Morgan's avatar
Jeffrey Morgan committed
48

Bruce MacDonald's avatar
Bruce MacDonald committed
49
## Cooing Soon
Jeffrey Morgan's avatar
Jeffrey Morgan committed
50

Bruce MacDonald's avatar
Bruce MacDonald committed
51
52
### `ollama.pull`

Jeffrey Morgan's avatar
Jeffrey Morgan committed
53
Download a model
Bruce MacDonald's avatar
Bruce MacDonald committed
54
55
56

```python
ollama.pull("huggingface.co/thebloke/llama-7b-ggml")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
57
58
```

Bruce MacDonald's avatar
Bruce MacDonald committed
59
60
### `ollama.import`

Jeffrey Morgan's avatar
Jeffrey Morgan committed
61
Import a model from a file
Bruce MacDonald's avatar
Bruce MacDonald committed
62
63
64
65

```python
ollama.import("./path/to/model")
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
66

Bruce MacDonald's avatar
Bruce MacDonald committed
67
### `ollama.search`
68

Bruce MacDonald's avatar
Bruce MacDonald committed
69
Search for compatible models that Ollama can run
70

Bruce MacDonald's avatar
Bruce MacDonald committed
71
72
73
```python
ollama.search("llama-7b")
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
74

Bruce MacDonald's avatar
Bruce MacDonald committed
75
## Future CLI
76

Jeffrey Morgan's avatar
Jeffrey Morgan committed
77
78
In the future, there will be an easy CLI for testing out models

Bruce MacDonald's avatar
Bruce MacDonald committed
79
80
```
ollama run huggingface.co/thebloke/llama-7b-ggml
Jeffrey Morgan's avatar
Jeffrey Morgan committed
81
> Downloading [================>          ] 66.67% (2/3) 30.2MB/s
Bruce MacDonald's avatar
Bruce MacDonald committed
82
```