modelfile.md 9.05 KB
Newer Older
Jeffrey Morgan's avatar
Jeffrey Morgan committed
1
2
3
4
5
6
# Ollama Model File

> Note: this model file syntax is in development

A model file is the blueprint to create and share models with Ollama.

Bruce MacDonald's avatar
Bruce MacDonald committed
7
8
9
10
11
12
13
14
## Table of Contents

- [Format](#format)
- [Examples](#examples)
- [Instructions](#instructions)
  - [FROM (Required)](#from-required)
    - [Build from llama2](#build-from-llama2)
    - [Build from a bin file](#build-from-a-bin-file)
Bruce MacDonald's avatar
Bruce MacDonald committed
15
  - [EMBED](#embed)
Bruce MacDonald's avatar
Bruce MacDonald committed
16
17
18
19
20
21
22
23
  - [PARAMETER](#parameter)
    - [Valid Parameters and Values](#valid-parameters-and-values)
  - [TEMPLATE](#template)
    - [Template Variables](#template-variables)
  - [SYSTEM](#system)
  - [LICENSE](#license)
- [Notes](#notes)

Jeffrey Morgan's avatar
Jeffrey Morgan committed
24
25
26
27
28
29
30
31
32
## Format

The format of the Modelfile:

```modelfile
# comment
INSTRUCTION arguments
```

Bruce MacDonald's avatar
Bruce MacDonald committed
33
34
35
36
37
38
39
| Instruction                         | Description                                                   |
| ----------------------------------- | ------------------------------------------------------------- |
| [`FROM`](#from-required) (required) | Defines the base model to use.                                |
| [`PARAMETER`](#parameter)           | Sets the parameters for how Ollama will run the model.        |
| [`TEMPLATE`](#template)             | The full prompt template to be sent to the model.             |
| [`SYSTEM`](#system)                 | Specifies the system prompt that will be set in the template. |
| [`LICENSE`](#license)               | Specifies the legal license.                                  |
Jeffrey Morgan's avatar
Jeffrey Morgan committed
40
41
42
43
44
45
46
47
48

## Examples

An example of a model file creating a mario blueprint:

```
FROM llama2
# sets the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
Bruce MacDonald's avatar
Bruce MacDonald committed
49
# sets the context window size to 4096, this controls how many tokens the LLM can use as context to generate the next token
Jeffrey Morgan's avatar
Jeffrey Morgan committed
50
51
PARAMETER num_ctx 4096

Bruce MacDonald's avatar
Bruce MacDonald committed
52
# sets a custom system prompt to specify the behavior of the chat assistant
Jeffrey Morgan's avatar
Jeffrey Morgan committed
53
54
55
56
57
SYSTEM You are Mario from super mario bros, acting as an assistant.
```

To use this:

58
1. Save it as a file (eg. `Modelfile`)
Jeffrey Morgan's avatar
Jeffrey Morgan committed
59
60
61
62
2. `ollama create NAME -f <location of the file eg. ./Modelfile>'`
3. `ollama run NAME`
4. Start using the model!

Bruce MacDonald's avatar
Bruce MacDonald committed
63
64
65
66
67
More examples are available in the [examples directory](../examples).

## Instructions

### FROM (Required)
Jeffrey Morgan's avatar
Jeffrey Morgan committed
68
69
70
71
72
73
74

The FROM instruction defines the base model to use when creating a model.

```
FROM <model name>:<tag>
```

Bruce MacDonald's avatar
Bruce MacDonald committed
75
#### Build from llama2
Jeffrey Morgan's avatar
Jeffrey Morgan committed
76
77
78
79
80
81
82
83

```
FROM llama2
```

A list of available base models:
<https://github.com/jmorganca/ollama#model-library>

Bruce MacDonald's avatar
Bruce MacDonald committed
84
#### Build from a bin file
Jeffrey Morgan's avatar
Jeffrey Morgan committed
85
86
87
88
89

```
FROM ./ollama-model.bin
```

Bruce MacDonald's avatar
Bruce MacDonald committed
90
91
This bin file location should be specified as an absolute path or relative to the Modelfile location.

Bruce MacDonald's avatar
Bruce MacDonald committed
92
93
94
95
96
97
98
99
100
### EMBED

The EMBED instruction is used to add embeddings of files to a model. This is useful for adding custom data that the model can reference when generating an answer.

```
FROM <model name>:<tag>
EMBED <file path>
```

Bruce MacDonald's avatar
Bruce MacDonald committed
101
### PARAMETER
Jeffrey Morgan's avatar
Jeffrey Morgan committed
102
103
104
105
106
107
108
109
110

The `PARAMETER` instruction defines a parameter that can be set when the model is run.

```
PARAMETER <parameter> <parametervalue>
```

### Valid Parameters and Values

Bruce MacDonald's avatar
Bruce MacDonald committed
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
| Parameter      | Description                                                                                                                                                                                                                                             | Value Type | Example Usage        |
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------- |
| mirostat       | Enable Mirostat sampling for controlling perplexity. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)                                                                                                                                         | int        | mirostat 0           |
| mirostat_eta   | Influences how quickly the algorithm responds to feedback from the generated text. A lower learning rate will result in slower adjustments, while a higher learning rate will make the algorithm more responsive. (Default: 0.1)                        | float      | mirostat_eta 0.1     |
| mirostat_tau   | Controls the balance between coherence and diversity of the output. A lower value will result in more focused and coherent text. (Default: 5.0)                                                                                                         | float      | mirostat_tau 5.0     |
| num_ctx        | Sets the size of the context window used to generate the next token. (Default: 2048)                                                                                                                                                                    | int        | num_ctx 4096         |
| num_gpu        | The number of GPUs to use. On macOS it defaults to 1 to enable metal support, 0 to disable.                                                                                                                                                             | int        | num_gpu 1            |
| num_thread     | Sets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number of cores). | int        | num_thread 8         |
| repeat_last_n  | Sets how far back for the model to look back to prevent repetition. (Default: 64, 0 = disabled, -1 = num_ctx)                                                                                                                                           | int        | repeat_last_n 64     |
| repeat_penalty | Sets how strongly to penalize repetitions. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. (Default: 1.1)                                                                     | float      | repeat_penalty 1.1   |
| temperature    | The temperature of the model. Increasing the temperature will make the model answer more creatively. (Default: 0.8)                                                                                                                                     | float      | temperature 0.7      |
| stop           | Sets the stop tokens to use.                                                                                                                                                                                                                            | string     | stop "AI assistant:" |
| tfs_z          | Tail free sampling is used to reduce the impact of less probable tokens from the output. A higher value (e.g., 2.0) will reduce the impact more, while a value of 1.0 disables this setting. (default: 1)                                               | float      | tfs_z 1              |
| top_k          | Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative. (Default: 40)                                                                        | int        | top_k 40             |
| top_p          | Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text. (Default: 0.9)                                                                 | float      | top_p 0.9            |
Jeffrey Morgan's avatar
Jeffrey Morgan committed
126

Bruce MacDonald's avatar
Bruce MacDonald committed
127
128
### TEMPLATE

Bruce MacDonald's avatar
Bruce MacDonald committed
129
`TEMPLATE` of the full prompt template to be passed into the model. It may include (optionally) a system prompt and a user's prompt. This is used to create a full custom prompt, and syntax may be model specific.
Bruce MacDonald's avatar
Bruce MacDonald committed
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147

#### Template Variables

| Variable        | Description                                                                                                  |
| --------------- | ------------------------------------------------------------------------------------------------------------ |
| `{{ .System }}` | The system prompt used to specify custom behavior, this must also be set in the Modelfile as an instruction. |
| `{{ .Prompt }}` | The incoming prompt, this is not specified in the model file and will be set based on input.                 |
| `{{ .First }}`  | A boolean value used to render specific template information for the first generation of a session.          |

```
TEMPLATE """
{{- if .First }}
### System:
{{ .System }}
{{- end }}

### User:
{{ .Prompt }}
Jeffrey Morgan's avatar
Jeffrey Morgan committed
148

Bruce MacDonald's avatar
Bruce MacDonald committed
149
150
### Response:
"""
Jeffrey Morgan's avatar
Jeffrey Morgan committed
151

Bruce MacDonald's avatar
Bruce MacDonald committed
152
153
154
155
156
157
158
159
160
161
162
163
164
SYSTEM """<system message>"""
```

### SYSTEM

The `SYSTEM` instruction specifies the system prompt to be used in the template, if applicable.

```
SYSTEM """<system message>"""
```

### LICENSE

Michael Yang's avatar
Michael Yang committed
165
The `LICENSE` instruction allows you to specify the legal license under which the model used with this Modelfile is shared or distributed.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
166

Bruce MacDonald's avatar
Bruce MacDonald committed
167
168
169
170
171
```
LICENSE """
<license text>
"""
```
Jeffrey Morgan's avatar
Jeffrey Morgan committed
172
173
174
175

## Notes

- the **modelfile is not case sensitive**. In the examples, we use uppercase for instructions to make it easier to distinguish it from arguments.
Bruce MacDonald's avatar
Bruce MacDonald committed
176
- Instructions can be in any order. In the examples, we start with FROM instruction to keep it easily readable.