README.md 1.14 KB
Newer Older
Bruce MacDonald's avatar
Bruce MacDonald committed
1
# Ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
2

Jeffrey Morgan's avatar
Jeffrey Morgan committed
3
- Run models easily
Bruce MacDonald's avatar
Bruce MacDonald committed
4
- Download, manage and import models
Jeffrey Morgan's avatar
Jeffrey Morgan committed
5

Bruce MacDonald's avatar
Bruce MacDonald committed
6
## Install
Jeffrey Morgan's avatar
Jeffrey Morgan committed
7
8

```
Bruce MacDonald's avatar
Bruce MacDonald committed
9
pip install ollama
Jeffrey Morgan's avatar
Jeffrey Morgan committed
10
11
```

Bruce MacDonald's avatar
Bruce MacDonald committed
12
## Example quickstart
Jeffrey Morgan's avatar
Jeffrey Morgan committed
13

Bruce MacDonald's avatar
Bruce MacDonald committed
14
15
16
17
18
19
```python
import ollama
model_name = "huggingface.co/thebloke/llama-7b-ggml"
model = ollama.pull(model_name)
ollama.load(model)
ollama.generate(model_name, "hi")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
20
21
```

Bruce MacDonald's avatar
Bruce MacDonald committed
22
## Reference
Jeffrey Morgan's avatar
Jeffrey Morgan committed
23

Bruce MacDonald's avatar
Bruce MacDonald committed
24
25
### `ollama.load`

Jeffrey Morgan's avatar
Jeffrey Morgan committed
26
Load a model for generation
Bruce MacDonald's avatar
Bruce MacDonald committed
27
28
29

```python
ollama.load("model name")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
30
31
```

Bruce MacDonald's avatar
Bruce MacDonald committed
32
### `ollama.generate("message")`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
33

Bruce MacDonald's avatar
Bruce MacDonald committed
34
Generate a completion
Jeffrey Morgan's avatar
Jeffrey Morgan committed
35

Bruce MacDonald's avatar
Bruce MacDonald committed
36
37
```python
ollama.generate(model, "hi")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
38
39
```

Bruce MacDonald's avatar
Bruce MacDonald committed
40
### `ollama.models`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
41

Jeffrey Morgan's avatar
Jeffrey Morgan committed
42
List available local models
Bruce MacDonald's avatar
Bruce MacDonald committed
43
44
45

```
models = ollama.models()
Jeffrey Morgan's avatar
Jeffrey Morgan committed
46
47
```

Bruce MacDonald's avatar
Bruce MacDonald committed
48
### `ollama.serve`
Jeffrey Morgan's avatar
Jeffrey Morgan committed
49

Bruce MacDonald's avatar
Bruce MacDonald committed
50
Serve the ollama http server
Jeffrey Morgan's avatar
Jeffrey Morgan committed
51

Bruce MacDonald's avatar
Bruce MacDonald committed
52
## Cooing Soon
Jeffrey Morgan's avatar
Jeffrey Morgan committed
53

Bruce MacDonald's avatar
Bruce MacDonald committed
54
55
### `ollama.pull`

Jeffrey Morgan's avatar
Jeffrey Morgan committed
56
Download a model
Bruce MacDonald's avatar
Bruce MacDonald committed
57
58
59

```python
ollama.pull("huggingface.co/thebloke/llama-7b-ggml")
Jeffrey Morgan's avatar
Jeffrey Morgan committed
60
61
```

Bruce MacDonald's avatar
Bruce MacDonald committed
62
63
### `ollama.import`

Jeffrey Morgan's avatar
Jeffrey Morgan committed
64
Import a model from a file
Bruce MacDonald's avatar
Bruce MacDonald committed
65
66
67
68

```python
ollama.import("./path/to/model")
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
69

Bruce MacDonald's avatar
Bruce MacDonald committed
70
### `ollama.search`
71

Bruce MacDonald's avatar
Bruce MacDonald committed
72
Search for compatible models that Ollama can run
73

Bruce MacDonald's avatar
Bruce MacDonald committed
74
75
76
```python
ollama.search("llama-7b")
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
77

Bruce MacDonald's avatar
Bruce MacDonald committed
78
## Future CLI
79

Jeffrey Morgan's avatar
Jeffrey Morgan committed
80
81
In the future, there will be an easy CLI for testing out models

Bruce MacDonald's avatar
Bruce MacDonald committed
82
83
```
ollama run huggingface.co/thebloke/llama-7b-ggml
Jeffrey Morgan's avatar
Jeffrey Morgan committed
84
> Downloading [================>          ] 66.67% (2/3) 30.2MB/s
Bruce MacDonald's avatar
Bruce MacDonald committed
85
```