api.md 48.4 KB
Newer Older
1
2
# API

3
4
5
## Endpoints

- [Generate a completion](#generate-a-completion)
6
- [Generate a chat completion](#generate-a-chat-completion)
Matt Williams's avatar
Matt Williams committed
7
8
9
10
11
12
13
14
- [Create a Model](#create-a-model)
- [List Local Models](#list-local-models)
- [Show Model Information](#show-model-information)
- [Copy a Model](#copy-a-model)
- [Delete a Model](#delete-a-model)
- [Pull a Model](#pull-a-model)
- [Push a Model](#push-a-model)
- [Generate Embeddings](#generate-embeddings)
royjhan's avatar
royjhan committed
15
- [List Running Models](#list-running-models)
16
- [Version](#version)
Matt Williams's avatar
Matt Williams committed
17

18
## Conventions
Matt Williams's avatar
Matt Williams committed
19

20
### Model names
Matt Williams's avatar
Matt Williams committed
21

Daniel Hiltgen's avatar
Daniel Hiltgen committed
22
Model names follow a `model:tag` format, where `model` can have an optional namespace such as `example/model`. Some examples are `orca-mini:3b-q8_0` and `llama3:70b`. The tag is optional and, if not provided, will default to `latest`. The tag is used to identify a specific version.
Matt Williams's avatar
Matt Williams committed
23
24
25

### Durations

26
All durations are returned in nanoseconds.
Matt Williams's avatar
Matt Williams committed
27

28
29
### Streaming responses

Jeffrey Morgan's avatar
Jeffrey Morgan committed
30
Certain endpoints stream responses as JSON objects. Streaming can be disabled by providing `{"stream": false}` for these endpoints.
Matt Williams's avatar
Matt Williams committed
31

32
## Generate a completion
Matt Williams's avatar
Matt Williams committed
33

34
```
35
36
POST /api/generate
```
37

Bruce MacDonald's avatar
Bruce MacDonald committed
38
Generate a response for a given prompt with a provided model. This is a streaming endpoint, so there will be a series of responses. The final response object will include statistics and additional data from the request.
Matt Williams's avatar
Matt Williams committed
39

40
### Parameters
Matt Williams's avatar
Matt Williams committed
41

42
43
- `model`: (required) the [model name](#model-names)
- `prompt`: the prompt to generate a response for
Michael Yang's avatar
Michael Yang committed
44
- `suffix`: the text after the model response
Matt Williams's avatar
Matt Williams committed
45
- `images`: (optional) a list of base64-encoded images (for multimodal models such as `llava`)
46
- `think`: (for thinking models) should the model think before responding?
Matt Williams's avatar
Matt Williams committed
47

48
Advanced parameters (optional):
Matt Williams's avatar
Matt Williams committed
49

50
- `format`: the format to return a response in. Format can be `json` or a JSON schema
51
- `options`: additional model parameters listed in the documentation for the [Modelfile](./modelfile.md#valid-parameters-and-values) such as `temperature`
52
- `system`: system message to (overrides what is defined in the `Modelfile`)
53
- `template`: the prompt template to use (overrides what is defined in the `Modelfile`)
54
- `stream`: if `false` the response will be returned as a single response object, rather than a stream of objects
Bruce MacDonald's avatar
Bruce MacDonald committed
55
56
- `raw`: if `true` no formatting will be applied to the prompt. You may choose to use the `raw` parameter if you are specifying a full templated prompt in your request to the API
- `keep_alive`: controls how long the model will stay loaded into memory following the request (default: `5m`)
57
- `context` (deprecated): the context parameter returned from a previous request to `/generate`, this can be used to keep a short conversational memory
58

59
60
61
62
#### Structured outputs

Structured outputs are supported by providing a JSON schema in the `format` parameter. The model will generate a response that matches the schema. See the [structured outputs](#request-structured-outputs) example below.

Matt Williams's avatar
Matt Williams committed
63
#### JSON mode
64

Jeffrey Morgan's avatar
Jeffrey Morgan committed
65
Enable JSON mode by setting the `format` parameter to `json`. This will structure the response as a valid JSON object. See the JSON mode [example](#request-json-mode) below.
66

Michael Yang's avatar
Michael Yang committed
67
68
> [!IMPORTANT]
> It's important to instruct the model to use JSON in the `prompt`. Otherwise, the model may generate large amounts whitespace.
69

70
71
### Examples

Matt Williams's avatar
Matt Williams committed
72
73
74
#### Generate request (Streaming)

##### Request
Matt Williams's avatar
Matt Williams committed
75

Matt Williams's avatar
Matt Williams committed
76
```shell
77
curl http://localhost:11434/api/generate -d '{
78
  "model": "llama3.2",
79
80
81
  "prompt": "Why is the sky blue?"
}'
```
Matt Williams's avatar
Matt Williams committed
82

Matt Williams's avatar
Matt Williams committed
83
##### Response
Matt Williams's avatar
Matt Williams committed
84

85
A stream of JSON objects is returned:
Matt Williams's avatar
Matt Williams committed
86

87
```json
Matt Williams's avatar
Matt Williams committed
88
{
89
  "model": "llama3.2",
90
91
  "created_at": "2023-08-04T08:52:19.385406455-07:00",
  "response": "The",
Matt Williams's avatar
Matt Williams committed
92
93
94
95
  "done": false
}
```

96
The final response in the stream also includes additional data about the generation:
Matt Williams's avatar
Matt Williams committed
97

98
99
100
101
- `total_duration`: time spent generating the response
- `load_duration`: time spent in nanoseconds loading the model
- `prompt_eval_count`: number of tokens in the prompt
- `prompt_eval_duration`: time spent in nanoseconds evaluating the prompt
Sri Siddhaarth's avatar
Sri Siddhaarth committed
102
- `eval_count`: number of tokens in the response
103
- `eval_duration`: time in nanoseconds spent generating the response
104
- `context`: an encoding of the conversation used in this response, this can be sent in the next request to keep a conversational memory
105
- `response`: empty if the response was streamed, if not streamed, this will contain the full response
106

Darinka's avatar
Darinka committed
107
To calculate how fast the response is generated in tokens per second (token/s), divide `eval_count` / `eval_duration` * `10^9`.
108

109
```json
Matt Williams's avatar
Matt Williams committed
110
{
111
  "model": "llama3.2",
112
  "created_at": "2023-08-04T19:22:45.499127Z",
113
  "response": "",
114
  "done": true,
Matt Williams's avatar
Matt Williams committed
115
  "context": [1, 2, 3],
Jeffrey Morgan's avatar
Jeffrey Morgan committed
116
117
118
119
120
121
  "total_duration": 10706818083,
  "load_duration": 6338219291,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 130079000,
  "eval_count": 259,
  "eval_duration": 4232710000
122
}
Matt Williams's avatar
Matt Williams committed
123
124
```

125
#### Request (No streaming)
126

Matt Williams's avatar
Matt Williams committed
127
128
129
##### Request

A response can be received in one reply when streaming is off.
Bruce MacDonald's avatar
Bruce MacDonald committed
130

131
```shell
132
curl http://localhost:11434/api/generate -d '{
133
  "model": "llama3.2",
134
135
136
137
138
  "prompt": "Why is the sky blue?",
  "stream": false
}'
```

Matt Williams's avatar
Matt Williams committed
139
##### Response
140

141
142
143
144
If `stream` is set to `false`, the response will be a single JSON object:

```json
{
145
  "model": "llama3.2",
146
147
  "created_at": "2023-08-04T19:22:45.499127Z",
  "response": "The sky is blue because it is the color of the sky.",
Matt Williams's avatar
Matt Williams committed
148
  "done": true,
149
  "context": [1, 2, 3],
Matt Williams's avatar
Matt Williams committed
150
151
152
153
154
155
156
157
158
  "total_duration": 5043500667,
  "load_duration": 5025959,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 325953000,
  "eval_count": 290,
  "eval_duration": 4709213000
}
```

Michael Yang's avatar
Michael Yang committed
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
#### Request (with suffix)

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
  "model": "codellama:code",
  "prompt": "def compute_gcd(a, b):",
  "suffix": "    return result",
  "options": {
    "temperature": 0
  },
  "stream": false
}'
```

##### Response

177
```json5
Michael Yang's avatar
Michael Yang committed
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
{
  "model": "codellama:code",
  "created_at": "2024-07-22T20:47:51.147561Z",
  "response": "\n  if a == 0:\n    return b\n  else:\n    return compute_gcd(b % a, a)\n\ndef compute_lcm(a, b):\n  result = (a * b) / compute_gcd(a, b)\n",
  "done": true,
  "done_reason": "stop",
  "context": [...],
  "total_duration": 1162761250,
  "load_duration": 6683708,
  "prompt_eval_count": 17,
  "prompt_eval_duration": 201222000,
  "eval_count": 63,
  "eval_duration": 953997000
}
```

194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
#### Request (Structured outputs)

##### Request

```shell
curl -X POST http://localhost:11434/api/generate -H "Content-Type: application/json" -d '{
  "model": "llama3.1:8b",
  "prompt": "Ollama is 22 years old and is busy saving the world. Respond using JSON",
  "stream": false,
  "format": {
    "type": "object",
    "properties": {
      "age": {
        "type": "integer"
      },
      "available": {
        "type": "boolean"
      }
    },
    "required": [
      "age",
      "available"
    ]
  }
}'
```

##### Response

```json
{
  "model": "llama3.1:8b",
  "created_at": "2024-12-06T00:48:09.983619Z",
  "response": "{\n  \"age\": 22,\n  \"available\": true\n}",
  "done": true,
  "done_reason": "stop",
  "context": [1, 2, 3],
  "total_duration": 1075509083,
  "load_duration": 567678166,
  "prompt_eval_count": 28,
  "prompt_eval_duration": 236000000,
  "eval_count": 16,
  "eval_duration": 269000000
}
```

Matt Williams's avatar
Matt Williams committed
240
241
#### Request (JSON mode)

Michael Yang's avatar
Michael Yang committed
242
> [!IMPORTANT]
Matt Williams's avatar
Matt Williams committed
243
244
245
246
247
248
> When `format` is set to `json`, the output will always be a well-formed JSON object. It's important to also instruct the model to respond in JSON.

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
249
  "model": "llama3.2",
Matt Williams's avatar
Matt Williams committed
250
251
252
253
254
255
256
257
258
259
  "prompt": "What color is the sky at different times of the day? Respond using JSON",
  "format": "json",
  "stream": false
}'
```

##### Response

```json
{
260
  "model": "llama3.2",
Matt Williams's avatar
Matt Williams committed
261
262
  "created_at": "2023-11-09T21:07:55.186497Z",
  "response": "{\n\"morning\": {\n\"color\": \"blue\"\n},\n\"noon\": {\n\"color\": \"blue-gray\"\n},\n\"afternoon\": {\n\"color\": \"warm gray\"\n},\n\"evening\": {\n\"color\": \"orange\"\n}\n}\n",
263
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
264
  "context": [1, 2, 3],
Matt Williams's avatar
Matt Williams committed
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
  "total_duration": 4648158584,
  "load_duration": 4071084,
  "prompt_eval_count": 36,
  "prompt_eval_duration": 439038000,
  "eval_count": 180,
  "eval_duration": 4196918000
}
```

The value of `response` will be a string containing JSON similar to:

```json
{
  "morning": {
    "color": "blue"
  },
  "noon": {
    "color": "blue-gray"
  },
  "afternoon": {
    "color": "warm gray"
  },
  "evening": {
    "color": "orange"
  }
290
291
292
}
```

293
294
295
296
#### Request (with images)

To submit images to multimodal models such as `llava` or `bakllava`, provide a list of base64-encoded `images`:

Matt Williams's avatar
Matt Williams committed
297
298
#### Request

299
300
301
302
303
304
305
306
307
308
309
```shell
curl http://localhost:11434/api/generate -d '{
  "model": "llava",
  "prompt":"What is in this picture?",
  "stream": false,
  "images": ["iVBORw0KGgoAAAANSUhEUgAAAG0AAABmCAYAAADBPx+VAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAA3VSURBVHgB7Z27r0zdG8fX743i1bi1ikMoFMQloXRpKFFIqI7LH4BEQ+NWIkjQuSWCRIEoULk0gsK1kCBI0IhrQVT7tz/7zZo888yz1r7MnDl7z5xvsjkzs2fP3uu71nNfa7lkAsm7d++Sffv2JbNmzUqcc8m0adOSzZs3Z+/XES4ZckAWJEGWPiCxjsQNLWmQsWjRIpMseaxcuTKpG/7HP27I8P79e7dq1ars/yL4/v27S0ejqwv+cUOGEGGpKHR37tzJCEpHV9tnT58+dXXCJDdECBE2Ojrqjh071hpNECjx4cMHVycM1Uhbv359B2F79+51586daxN/+pyRkRFXKyRDAqxEp4yMlDDzXG1NPnnyJKkThoK0VFd1ELZu3TrzXKxKfW7dMBQ6bcuWLW2v0VlHjx41z717927ba22U9APcw7Nnz1oGEPeL3m3p2mTAYYnFmMOMXybPPXv2bNIPpFZr1NHn4HMw0KRBjg9NuRw95s8PEcz/6DZELQd/09C9QGq5RsmSRybqkwHGjh07OsJSsYYm3ijPpyHzoiacg35MLdDSIS/O1yM778jOTwYUkKNHWUzUWaOsylE00MyI0fcnOwIdjvtNdW/HZwNLGg+sR1kMepSNJXmIwxBZiG8tDTpEZzKg0GItNsosY8USkxDhD0Rinuiko2gfL/RbiD2LZAjU9zKQJj8RDR0vJBR1/Phx9+PHj9Z7REF4nTZkxzX4LCXHrV271qXkBAPGfP/atWvu/PnzHe4C97F48eIsRLZ9+3a3f/9+87dwP1JxaF7/3r17ba+5l4EcaVo0lj3SBq5kGTJSQmLWMjgYNei2GPT1MuMqGTDEFHzeQSP2wi/jGnkmPJ/nhccs44jvDAxpVcxnq0F6eT8h4ni/iIWpR5lPyA6ETkNXoSukvpJAD3AsXLiwpZs49+fPn5ke4j10TqYvegSfn0OnafC+Tv9ooA/JPkgQysqQNBzagXY55nO/oa1F7qvIPWkRL12WRpMWUvpVDYmxAPehxWSe8ZEXL20sadYIozfmNch4QJPAfeJgW3rNsnzphBKNJM2KKODo1rVOMRYik5ETy3ix4qWNI81qAAirizgMIc+yhTytx0JWZuNI03qsrgWlGtwjoS9XwgUhWGyhUaRZZQNNIEwCiXD16tXcAHUs79co0vSD8rrJCIW98pzvxpAWyyo3HYwqS0+H0BjStClcZJT5coMm6D2LOF8TolGJtK9fvyZpyiC5ePFi9nc/oJU4eiEP0jVoAnHa9wyJycITMP78+eMeP37sXrx44d6+fdt6f82aNdkx1pg9e3Zb5W+RSRE+n+VjksQWifvVaTKFhn5O8my63K8Qabdv33b379/PiAP//vuvW7BggZszZ072/+TJk91YgkafPn166zXB1rQHFvouAWHq9z3SEevSUerqCn2/dDCeta2jxYbr69evk4MHDyY7d+7MjhMnTiTPnz9Pfv/+nfQT2ggpO2dMF8cghuoM7Ygj5iWCqRlGFml0QC/ftGmTmzt3rmsaKDsgBSPh0/8yPeLLBihLkOKJc0jp8H8vUzcxIA1k6QJ/c78tWEyj5P3o4u9+jywNPdJi5rAH9x0KHcl4Hg570eQp3+vHXGyrmEeigzQsQsjavXt38ujRo44LQuDDhw+TW7duRS1HGgMxhNXHgflaNTOsHyKvHK5Ijo2jbFjJBQK9YwFd6RVMzfgRBmEfP37suBBm/p49e1qjEP2mwTViNRo0VJWH1deMXcNK08uUjVUu7s/zRaL+oLNxz1bpANco4npUgX4G2eFbpDFyQoQxojBCpEGSytmOH8qrH5Q9vuzD6ofQylkCUmh8DBAr+q8JCyVNtWQIidKQE9wNtLSQnS4jDSsxNHogzFuQBw4cyM61UKVsjfr3ooBkPSqqQHesUPWVtzi9/vQi1T+rJj7WiTz4Pt/l3LxUkr5P2VYZaZ4URpsE+st/dujQoaBBYokbrz/8TJNQYLSonrPS9kUaSkPeZyj1AWSj+d+VBoy1pIWVNed8P0Ll/ee5HdGRhrHhR5GGN0r4LGZBaj8oFDJitBTJzIZgFcmU0Y8ytWMZMzJOaXUSrUs5RxKnrxmbb5YXO9VGUhtpXldhEUogFr3IzIsvlpmdosVcGVGXFWp2oU9kLFL3dEkSz6NHEY1sjSRdIuDFWEhd8KxFqsRi1uM/nz9/zpxnwlESONdg6dKlbsaMGS4EHFHtjFIDHwKOo46l4TxSuxgDzi+rE2jg+BaFruOX4HXa0Nnf1lwAPufZeF8/r6zD97WK2qFnGjBxTw5qNGPxT+5T/r7/7RawFC3j4vTp09koCxkeHjqbHJqArmH5UrFKKksnxrK7FuRIs8STfBZv+luugXZ2pR/pP9Ois4z+TiMzUUkUjD0iEi1fzX8GmXyuxUBRcaUfykV0YZnlJGKQpOiGB76x5GeWkWWJc3mOrK6S7xdND+W5N6XyaRgtWJFe13GkaZnKOsYqGdOVVVbGupsyA/l7emTLHi7vwTdirNEt0qxnzAvBFcnQF16xh/TMpUuXHDowhlA9vQVraQhkudRdzOnK+04ZSP3DUhVSP61YsaLtd/ks7ZgtPcXqPqEafHkdqa84X6aCeL7YWlv6edGFHb+ZFICPlljHhg0bKuk0CSvVznWsotRu433alNdFrqG45ejoaPCaUkWERpLXjzFL2Rpllp7PJU2a/v7Ab8N05/9t27Z16KUqoFGsxnI9EosS2niSYg9SpU6B4JgTrvVW1flt1sT+0ADIJU2maXzcUTraGCRaL1Wp9rUMk16PMom8QhruxzvZIegJjFU7LLCePfS8uaQdPny4jTTL0dbee5mYokQsXTIWNY46kuMbnt8Kmec+LGWtOVIl9cT1rCB0V8WqkjAsRwta93TbwNYoGKsUSChN44lgBNCoHLHzquYKrU6qZ8lolCIN0Rh6cP0Q3U6I6IXILYOQI513hJaSKAorFpuHXJNfVlpRtmYBk1Su1obZr5dnKAO+L10Hrj3WZW+E3qh6IszE37F6EB+68mGpvKm4eb9bFrlzrok7fvr0Kfv727dvWRmdVTJHw0qiiCUSZ6wCK+7XL/AcsgNyL74DQQ730sv78Su7+t/A36MdY0sW5o40ahslXr58aZ5HtZB8GH64m9EmMZ7FpYw4T6QnrZfgenrhFxaSiSGXtPnz57e9TkNZLvTjeqhr734CNtrK41L40sUQckmj1lGKQ0rC37x544r8eNXRpnVE3ZZY7zXo8NomiO0ZUCj2uHz58rbXoZ6gc0uA+F6ZeKS/jhRDUq8MKrTho9fEkihMmhxtBI1DxKFY9XLpVcSkfoi8JGnToZO5sU5aiDQIW716ddt7ZLYtMQlhECdBGXZZMWldY5BHm5xgAroWj4C0hbYkSc/jBmggIrXJWlZM6pSETsEPGqZOndr2uuuR5rF169a2HoHPdurUKZM4CO1WTPqaDaAd+GFGKdIQkxAn9RuEWcTRyN2KSUgiSgF5aWzPTeA/lN5rZubMmR2bE4SIC4nJoltgAV/dVefZm72AtctUCJU2CMJ327hxY9t7EHbkyJFseq+EJSY16RPo3Dkq1kkr7+q0bNmyDuLQcZBEPYmHVdOBiJyIlrRDq41YPWfXOxUysi5fvtyaj+2BpcnsUV/oSoEMOk2CQGlr4ckhBwaetBhjCwH0ZHtJROPJkyc7UjcYLDjmrH7ADTEBXFfOYmB0k9oYBOjJ8b4aOYSe7QkKcYhFlq3QYLQhSidNmtS2RATwy8YOM3EQJsUjKiaWZ+vZToUQgzhkHXudb/PW5YMHD9yZM2faPsMwoc7RciYJXbGuBqJ1UIGKKLv915jsvgtJxCZDubdXr165mzdvtr1Hz5LONA8jrUwKPqsmVesKa49S3Q4WxmRPUEYdTjgiUcfUwLx589ySJUva3oMkP6IYddq6HMS4o55xBJBUeRjzfa4Zdeg56QZ43LhxoyPo7Lf1kNt7oO8wWAbNwaYjIv5lhyS7kRf96dvm5Jah8vfvX3flyhX35cuX6HfzFHOToS1H4BenCaHvO8pr8iDuwoUL7tevX+b5ZdbBair0xkFIlFDlW4ZknEClsp/TzXyAKVOmmHWFVSbDNw1l1+4f90U6IY/q4V27dpnE9bJ+v87QEydjqx/UamVVPRG+mwkNTYN+9tjkwzEx+atCm/X9WvWtDtAb68Wy9LXa1UmvCDDIpPkyOQ5ZwSzJ4jMrvFcr0rSjOUh+GcT4LSg5ugkW1Io0/SCDQBojh0hPlaJdah+tkVYrnTZowP8iq1F1TgMBBauufyB33x1v+NWFYmT5KmppgHC+NkAgbmRkpD3yn9QIseXymoTQFGQmIOKTxiZIWpvAatenVqRVXf2nTrAWMsPnKrMZHz6bJq5jvce6QK8J1cQNgKxlJapMPdZSR64/UivS9NztpkVEdKcrs5alhhWP9NeqlfWopzhZScI6QxseegZRGeg5a8C3Re1Mfl1ScP36ddcUaMuv24iOJtz7sbUjTS4qBvKmstYJoUauiuD3k5qhyr7QdUHMeCgLa1Ear9NquemdXgmum4fvJ6w1lqsuDhNrg1qSpleJK7K3TF0Q2jSd94uSZ60kK1e3qyVpQK6PVWXp2/FC3mp6jBhKKOiY2h3gtUV64TWM6wDETRPLDfSakXmH3w8g9Jlug8ZtTt4kVF0kLUYYmCCtD/DrQ5YhMGbA9L3ucdjh0y8kOHW5gU/VEEmJTcL4Pz/f7mgoAbYkAAAAAElFTkSuQmCC"]
}'
```

#### Response

310
```json
311
312
313
314
315
{
  "model": "llava",
  "created_at": "2023-11-03T15:36:02.583064Z",
  "response": "A happy cartoon character, which is cute and cheerful.",
  "done": true,
Matt Williams's avatar
Matt Williams committed
316
317
318
319
320
321
322
  "context": [1, 2, 3],
  "total_duration": 2938432250,
  "load_duration": 2559292,
  "prompt_eval_count": 1,
  "prompt_eval_duration": 2195557000,
  "eval_count": 44,
  "eval_duration": 736432000
323
324
325
}
```

Bruce MacDonald's avatar
Bruce MacDonald committed
326
#### Request (Raw Mode)
327

Matt Williams's avatar
Matt Williams committed
328
In some cases, you may wish to bypass the templating system and provide a full prompt. In this case, you can use the `raw` parameter to disable templating. Also note that raw mode will not return a context.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
329

Matt Williams's avatar
Matt Williams committed
330
##### Request
331
332

```shell
333
curl http://localhost:11434/api/generate -d '{
334
335
336
337
338
339
340
  "model": "mistral",
  "prompt": "[INST] why is the sky blue? [/INST]",
  "raw": true,
  "stream": false
}'
```

341
342
#### Request (Reproducible outputs)

343
For reproducible outputs, set `seed` to a number:
344
345
346
347
348
349

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
  "model": "mistral",
Jeffrey Morgan's avatar
Jeffrey Morgan committed
350
  "prompt": "Why is the sky blue?",
351
  "options": {
352
    "seed": 123
353
354
355
356
  }
}'
```

Matt Williams's avatar
Matt Williams committed
357
##### Response
358
359
360
361
362
363
364

```json
{
  "model": "mistral",
  "created_at": "2023-11-03T15:36:02.583064Z",
  "response": " The sky appears blue because of a phenomenon called Rayleigh scattering.",
  "done": true,
Matt Williams's avatar
Matt Williams committed
365
366
  "total_duration": 8493852375,
  "load_duration": 6589624375,
367
  "prompt_eval_count": 14,
Matt Williams's avatar
Matt Williams committed
368
369
370
  "prompt_eval_duration": 119039000,
  "eval_count": 110,
  "eval_duration": 1779061000
371
372
373
}
```

Matt Williams's avatar
Matt Williams committed
374
#### Generate request (With options)
375
376
377

If you want to set custom options for the model at runtime rather than in the Modelfile, you can do so with the `options` parameter. This example sets every available option, but you can set any of them individually and omit the ones you do not want to override.

Matt Williams's avatar
Matt Williams committed
378
379
##### Request

380
```shell
381
curl http://localhost:11434/api/generate -d '{
382
  "model": "llama3.2",
383
384
385
386
387
388
389
390
  "prompt": "Why is the sky blue?",
  "stream": false,
  "options": {
    "num_keep": 5,
    "seed": 42,
    "num_predict": 100,
    "top_k": 20,
    "top_p": 0.9,
391
    "min_p": 0.0,
392
393
394
395
396
397
398
399
400
    "typical_p": 0.7,
    "repeat_last_n": 33,
    "temperature": 0.8,
    "repeat_penalty": 1.2,
    "presence_penalty": 1.5,
    "frequency_penalty": 1.0,
    "penalize_newline": true,
    "stop": ["\n", "user:"],
    "numa": false,
401
    "num_ctx": 1024,
402
403
404
405
    "num_batch": 2,
    "num_gpu": 1,
    "main_gpu": 0,
    "use_mmap": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
406
    "num_thread": 8
407
  }
408
409
410
}'
```

Matt Williams's avatar
Matt Williams committed
411
##### Response
412
413
414

```json
{
415
  "model": "llama3.2",
416
417
418
  "created_at": "2023-08-04T19:22:45.499127Z",
  "response": "The sky is blue because it is the color of the sky.",
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
419
  "context": [1, 2, 3],
Matt Williams's avatar
Matt Williams committed
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
  "total_duration": 4935886791,
  "load_duration": 534986708,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 107345000,
  "eval_count": 237,
  "eval_duration": 4289432000
}
```

#### Load a model

If an empty prompt is provided, the model will be loaded into memory.

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
437
  "model": "llama3.2"
Matt Williams's avatar
Matt Williams committed
438
439
440
441
442
443
444
445
446
}'
```

##### Response

A single JSON object is returned:

```json
{
447
  "model": "llama3.2",
Jeffrey Morgan's avatar
Jeffrey Morgan committed
448
449
450
  "created_at": "2023-12-18T19:52:07.071755Z",
  "response": "",
  "done": true
451
452
453
}
```

454
455
456
457
458
459
460
461
#### Unload a model

If an empty prompt is provided and the `keep_alive` parameter is set to `0`, a model will be unloaded from memory.

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
462
  "model": "llama3.2",
463
464
465
466
467
468
469
470
471
472
  "keep_alive": 0
}'
```

##### Response

A single JSON object is returned:

```json
{
473
  "model": "llama3.2",
474
475
476
477
478
479
480
  "created_at": "2024-09-12T03:54:03.516566Z",
  "response": "",
  "done": true,
  "done_reason": "unload"
}
```

481
## Generate a chat completion
482

483
```
Bruce MacDonald's avatar
Bruce MacDonald committed
484
485
486
POST /api/chat
```

Matt Williams's avatar
Matt Williams committed
487
Generate the next message in a chat with a provided model. This is a streaming endpoint, so there will be a series of responses. Streaming can be disabled using `"stream": false`. The final response object will include statistics and additional data from the request.
Bruce MacDonald's avatar
Bruce MacDonald committed
488
489
490
491
492

### Parameters

- `model`: (required) the [model name](#model-names)
- `messages`: the messages of the chat, this can be used to keep a chat memory
493
- `tools`: list of tools in JSON for the model to use if supported
494
- `think`: (for thinking models) should the model think before responding?
Bruce MacDonald's avatar
Bruce MacDonald committed
495

496
497
The `message` object has the following fields:

Michael Yang's avatar
Michael Yang committed
498
- `role`: the role of the message, either `system`, `user`, `assistant`, or `tool`
499
- `content`: the content of the message
500
- `thinking`: (for thinking models) the model's thinking process
501
- `images` (optional): a list of images to include in the message (for multimodal models such as `llava`)
502
- `tool_calls` (optional): a list of tools in JSON that the model wants to use
503

Bruce MacDonald's avatar
Bruce MacDonald committed
504
505
Advanced parameters (optional):

506
- `format`: the format to return a response in. Format can be `json` or a JSON schema. 
Bruce MacDonald's avatar
Bruce MacDonald committed
507
508
- `options`: additional model parameters listed in the documentation for the [Modelfile](./modelfile.md#valid-parameters-and-values) such as `temperature`
- `stream`: if `false` the response will be returned as a single response object, rather than a stream of objects
Bruce MacDonald's avatar
Bruce MacDonald committed
509
- `keep_alive`: controls how long the model will stay loaded into memory following the request (default: `5m`)
Bruce MacDonald's avatar
Bruce MacDonald committed
510

511
512
513
514
### Structured outputs

Structured outputs are supported by providing a JSON schema in the `format` parameter. The model will generate a response that matches the schema. See the [Chat request (Structured outputs)](#chat-request-structured-outputs) example below.

Bruce MacDonald's avatar
Bruce MacDonald committed
515
516
### Examples

Matt Williams's avatar
Matt Williams committed
517
518
519
#### Chat Request (Streaming)

##### Request
520

Bruce MacDonald's avatar
Bruce MacDonald committed
521
522
523
Send a chat message with a streaming response.

```shell
524
curl http://localhost:11434/api/chat -d '{
525
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
526
527
528
529
530
531
532
533
534
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
  ]
}'
```

Matt Williams's avatar
Matt Williams committed
535
##### Response
Bruce MacDonald's avatar
Bruce MacDonald committed
536
537
538
539
540

A stream of JSON objects is returned:

```json
{
541
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
542
543
  "created_at": "2023-08-04T08:52:19.385406455-07:00",
  "message": {
Robin Glauser's avatar
Robin Glauser committed
544
    "role": "assistant",
Jeffrey Morgan's avatar
Jeffrey Morgan committed
545
    "content": "The",
Matt Williams's avatar
Matt Williams committed
546
    "images": null
Bruce MacDonald's avatar
Bruce MacDonald committed
547
548
549
550
551
552
553
554
555
  },
  "done": false
}
```

Final response:

```json
{
556
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
557
  "created_at": "2023-08-04T19:22:45.499127Z",
558
559
560
561
  "message": {
    "role": "assistant",
    "content": ""
  },
Bruce MacDonald's avatar
Bruce MacDonald committed
562
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
563
564
565
566
567
568
  "total_duration": 4883583458,
  "load_duration": 1334875,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 342546000,
  "eval_count": 282,
  "eval_duration": 4535599000
Bruce MacDonald's avatar
Bruce MacDonald committed
569
570
571
}
```

Matt Williams's avatar
Matt Williams committed
572
#### Chat request (No streaming)
573

Matt Williams's avatar
Matt Williams committed
574
575
576
577
##### Request

```shell
curl http://localhost:11434/api/chat -d '{
578
  "model": "llama3.2",
Matt Williams's avatar
Matt Williams committed
579
580
581
582
583
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
Jeffrey Morgan's avatar
Jeffrey Morgan committed
584
  ],
Matt Williams's avatar
Matt Williams committed
585
586
587
588
589
590
591
592
  "stream": false
}'
```

##### Response

```json
{
593
  "model": "llama3.2",
Matt Williams's avatar
Matt Williams committed
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
  "created_at": "2023-12-12T14:13:43.416799Z",
  "message": {
    "role": "assistant",
    "content": "Hello! How are you today?"
  },
  "done": true,
  "total_duration": 5191566416,
  "load_duration": 2154458,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 383809000,
  "eval_count": 298,
  "eval_duration": 4799921000
}
```

609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
#### Chat request (Structured outputs)

##### Request

```shell
curl -X POST http://localhost:11434/api/chat -H "Content-Type: application/json" -d '{
  "model": "llama3.1",
  "messages": [{"role": "user", "content": "Ollama is 22 years old and busy saving the world. Return a JSON object with the age and availability."}],
  "stream": false,
  "format": {
    "type": "object",
    "properties": {
      "age": {
        "type": "integer"
      },
      "available": {
        "type": "boolean"
      }
    },
    "required": [
      "age",
      "available"
    ]
  },
  "options": {
    "temperature": 0
  }
}'
```

##### Response

```json
{
  "model": "llama3.1",
  "created_at": "2024-12-06T00:46:58.265747Z",
  "message": { "role": "assistant", "content": "{\"age\": 22, \"available\": false}" },
  "done_reason": "stop",
  "done": true,
  "total_duration": 2254970291,
  "load_duration": 574751416,
  "prompt_eval_count": 34,
  "prompt_eval_duration": 1502000000,
  "eval_count": 12,
  "eval_duration": 175000000
}
```

Matt Williams's avatar
Matt Williams committed
657
658
659
660
661
#### Chat request (With History)

Send a chat message with a conversation history. You can use this same approach to start the conversation using multi-shot or chain-of-thought prompting.

##### Request
Bruce MacDonald's avatar
Bruce MacDonald committed
662
663

```shell
664
curl http://localhost:11434/api/chat -d '{
665
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    },
    {
      "role": "assistant",
      "content": "due to rayleigh scattering."
    },
    {
      "role": "user",
      "content": "how is that different than mie scattering?"
    }
  ]
}'
```

Matt Williams's avatar
Matt Williams committed
683
##### Response
Bruce MacDonald's avatar
Bruce MacDonald committed
684
685
686
687
688

A stream of JSON objects is returned:

```json
{
689
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
690
691
  "created_at": "2023-08-04T08:52:19.385406455-07:00",
  "message": {
Robin Glauser's avatar
Robin Glauser committed
692
    "role": "assistant",
Bruce MacDonald's avatar
Bruce MacDonald committed
693
694
695
696
697
698
699
700
701
702
    "content": "The"
  },
  "done": false
}
```

Final response:

```json
{
703
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
704
705
  "created_at": "2023-08-04T19:22:45.499127Z",
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
706
707
708
709
710
711
  "total_duration": 8113331500,
  "load_duration": 6396458,
  "prompt_eval_count": 61,
  "prompt_eval_duration": 398801000,
  "eval_count": 468,
  "eval_duration": 7701267000
Bruce MacDonald's avatar
Bruce MacDonald committed
712
713
714
}
```

Matt Williams's avatar
Matt Williams committed
715
716
717
#### Chat request (with images)

##### Request
718

Veit Heller's avatar
Veit Heller committed
719
Send a chat message with images. The images should be provided as an array, with the individual images encoded in Base64.
720
721
722

```shell
curl http://localhost:11434/api/chat -d '{
Matt Williams's avatar
Matt Williams committed
723
  "model": "llava",
724
725
726
727
728
  "messages": [
    {
      "role": "user",
      "content": "what is in this image?",
      "images": ["iVBORw0KGgoAAAANSUhEUgAAAG0AAABmCAYAAADBPx+VAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAA3VSURBVHgB7Z27r0zdG8fX743i1bi1ikMoFMQloXRpKFFIqI7LH4BEQ+NWIkjQuSWCRIEoULk0gsK1kCBI0IhrQVT7tz/7zZo888yz1r7MnDl7z5xvsjkzs2fP3uu71nNfa7lkAsm7d++Sffv2JbNmzUqcc8m0adOSzZs3Z+/XES4ZckAWJEGWPiCxjsQNLWmQsWjRIpMseaxcuTKpG/7HP27I8P79e7dq1ars/yL4/v27S0ejqwv+cUOGEGGpKHR37tzJCEpHV9tnT58+dXXCJDdECBE2Ojrqjh071hpNECjx4cMHVycM1Uhbv359B2F79+51586daxN/+pyRkRFXKyRDAqxEp4yMlDDzXG1NPnnyJKkThoK0VFd1ELZu3TrzXKxKfW7dMBQ6bcuWLW2v0VlHjx41z717927ba22U9APcw7Nnz1oGEPeL3m3p2mTAYYnFmMOMXybPPXv2bNIPpFZr1NHn4HMw0KRBjg9NuRw95s8PEcz/6DZELQd/09C9QGq5RsmSRybqkwHGjh07OsJSsYYm3ijPpyHzoiacg35MLdDSIS/O1yM778jOTwYUkKNHWUzUWaOsylE00MyI0fcnOwIdjvtNdW/HZwNLGg+sR1kMepSNJXmIwxBZiG8tDTpEZzKg0GItNsosY8USkxDhD0Rinuiko2gfL/RbiD2LZAjU9zKQJj8RDR0vJBR1/Phx9+PHj9Z7REF4nTZkxzX4LCXHrV271qXkBAPGfP/atWvu/PnzHe4C97F48eIsRLZ9+3a3f/9+87dwP1JxaF7/3r17ba+5l4EcaVo0lj3SBq5kGTJSQmLWMjgYNei2GPT1MuMqGTDEFHzeQSP2wi/jGnkmPJ/nhccs44jvDAxpVcxnq0F6eT8h4ni/iIWpR5lPyA6ETkNXoSukvpJAD3AsXLiwpZs49+fPn5ke4j10TqYvegSfn0OnafC+Tv9ooA/JPkgQysqQNBzagXY55nO/oa1F7qvIPWkRL12WRpMWUvpVDYmxAPehxWSe8ZEXL20sadYIozfmNch4QJPAfeJgW3rNsnzphBKNJM2KKODo1rVOMRYik5ETy3ix4qWNI81qAAirizgMIc+yhTytx0JWZuNI03qsrgWlGtwjoS9XwgUhWGyhUaRZZQNNIEwCiXD16tXcAHUs79co0vSD8rrJCIW98pzvxpAWyyo3HYwqS0+H0BjStClcZJT5coMm6D2LOF8TolGJtK9fvyZpyiC5ePFi9nc/oJU4eiEP0jVoAnHa9wyJycITMP78+eMeP37sXrx44d6+fdt6f82aNdkx1pg9e3Zb5W+RSRE+n+VjksQWifvVaTKFhn5O8my63K8Qabdv33b379/PiAP//vuvW7BggZszZ072/+TJk91YgkafPn166zXB1rQHFvouAWHq9z3SEevSUerqCn2/dDCeta2jxYbr69evk4MHDyY7d+7MjhMnTiTPnz9Pfv/+nfQT2ggpO2dMF8cghuoM7Ygj5iWCqRlGFml0QC/ftGmTmzt3rmsaKDsgBSPh0/8yPeLLBihLkOKJc0jp8H8vUzcxIA1k6QJ/c78tWEyj5P3o4u9+jywNPdJi5rAH9x0KHcl4Hg570eQp3+vHXGyrmEeigzQsQsjavXt38ujRo44LQuDDhw+TW7duRS1HGgMxhNXHgflaNTOsHyKvHK5Ijo2jbFjJBQK9YwFd6RVMzfgRBmEfP37suBBm/p49e1qjEP2mwTViNRo0VJWH1deMXcNK08uUjVUu7s/zRaL+oLNxz1bpANco4npUgX4G2eFbpDFyQoQxojBCpEGSytmOH8qrH5Q9vuzD6ofQylkCUmh8DBAr+q8JCyVNtWQIidKQE9wNtLSQnS4jDSsxNHogzFuQBw4cyM61UKVsjfr3ooBkPSqqQHesUPWVtzi9/vQi1T+rJj7WiTz4Pt/l3LxUkr5P2VYZaZ4URpsE+st/dujQoaBBYokbrz/8TJNQYLSonrPS9kUaSkPeZyj1AWSj+d+VBoy1pIWVNed8P0Ll/ee5HdGRhrHhR5GGN0r4LGZBaj8oFDJitBTJzIZgFcmU0Y8ytWMZMzJOaXUSrUs5RxKnrxmbb5YXO9VGUhtpXldhEUogFr3IzIsvlpmdosVcGVGXFWp2oU9kLFL3dEkSz6NHEY1sjSRdIuDFWEhd8KxFqsRi1uM/nz9/zpxnwlESONdg6dKlbsaMGS4EHFHtjFIDHwKOo46l4TxSuxgDzi+rE2jg+BaFruOX4HXa0Nnf1lwAPufZeF8/r6zD97WK2qFnGjBxTw5qNGPxT+5T/r7/7RawFC3j4vTp09koCxkeHjqbHJqArmH5UrFKKksnxrK7FuRIs8STfBZv+luugXZ2pR/pP9Ois4z+TiMzUUkUjD0iEi1fzX8GmXyuxUBRcaUfykV0YZnlJGKQpOiGB76x5GeWkWWJc3mOrK6S7xdND+W5N6XyaRgtWJFe13GkaZnKOsYqGdOVVVbGupsyA/l7emTLHi7vwTdirNEt0qxnzAvBFcnQF16xh/TMpUuXHDowhlA9vQVraQhkudRdzOnK+04ZSP3DUhVSP61YsaLtd/ks7ZgtPcXqPqEafHkdqa84X6aCeL7YWlv6edGFHb+ZFICPlljHhg0bKuk0CSvVznWsotRu433alNdFrqG45ejoaPCaUkWERpLXjzFL2Rpllp7PJU2a/v7Ab8N05/9t27Z16KUqoFGsxnI9EosS2niSYg9SpU6B4JgTrvVW1flt1sT+0ADIJU2maXzcUTraGCRaL1Wp9rUMk16PMom8QhruxzvZIegJjFU7LLCePfS8uaQdPny4jTTL0dbee5mYokQsXTIWNY46kuMbnt8Kmec+LGWtOVIl9cT1rCB0V8WqkjAsRwta93TbwNYoGKsUSChN44lgBNCoHLHzquYKrU6qZ8lolCIN0Rh6cP0Q3U6I6IXILYOQI513hJaSKAorFpuHXJNfVlpRtmYBk1Su1obZr5dnKAO+L10Hrj3WZW+E3qh6IszE37F6EB+68mGpvKm4eb9bFrlzrok7fvr0Kfv727dvWRmdVTJHw0qiiCUSZ6wCK+7XL/AcsgNyL74DQQ730sv78Su7+t/A36MdY0sW5o40ahslXr58aZ5HtZB8GH64m9EmMZ7FpYw4T6QnrZfgenrhFxaSiSGXtPnz57e9TkNZLvTjeqhr734CNtrK41L40sUQckmj1lGKQ0rC37x544r8eNXRpnVE3ZZY7zXo8NomiO0ZUCj2uHz58rbXoZ6gc0uA+F6ZeKS/jhRDUq8MKrTho9fEkihMmhxtBI1DxKFY9XLpVcSkfoi8JGnToZO5sU5aiDQIW716ddt7ZLYtMQlhECdBGXZZMWldY5BHm5xgAroWj4C0hbYkSc/jBmggIrXJWlZM6pSETsEPGqZOndr2uuuR5rF169a2HoHPdurUKZM4CO1WTPqaDaAd+GFGKdIQkxAn9RuEWcTRyN2KSUgiSgF5aWzPTeA/lN5rZubMmR2bE4SIC4nJoltgAV/dVefZm72AtctUCJU2CMJ327hxY9t7EHbkyJFseq+EJSY16RPo3Dkq1kkr7+q0bNmyDuLQcZBEPYmHVdOBiJyIlrRDq41YPWfXOxUysi5fvtyaj+2BpcnsUV/oSoEMOk2CQGlr4ckhBwaetBhjCwH0ZHtJROPJkyc7UjcYLDjmrH7ADTEBXFfOYmB0k9oYBOjJ8b4aOYSe7QkKcYhFlq3QYLQhSidNmtS2RATwy8YOM3EQJsUjKiaWZ+vZToUQgzhkHXudb/PW5YMHD9yZM2faPsMwoc7RciYJXbGuBqJ1UIGKKLv915jsvgtJxCZDubdXr165mzdvtr1Hz5LONA8jrUwKPqsmVesKa49S3Q4WxmRPUEYdTjgiUcfUwLx589ySJUva3oMkP6IYddq6HMS4o55xBJBUeRjzfa4Zdeg56QZ43LhxoyPo7Lf1kNt7oO8wWAbNwaYjIv5lhyS7kRf96dvm5Jah8vfvX3flyhX35cuX6HfzFHOToS1H4BenCaHvO8pr8iDuwoUL7tevX+b5ZdbBair0xkFIlFDlW4ZknEClsp/TzXyAKVOmmHWFVSbDNw1l1+4f90U6IY/q4V27dpnE9bJ+v87QEydjqx/UamVVPRG+mwkNTYN+9tjkwzEx+atCm/X9WvWtDtAb68Wy9LXa1UmvCDDIpPkyOQ5ZwSzJ4jMrvFcr0rSjOUh+GcT4LSg5ugkW1Io0/SCDQBojh0hPlaJdah+tkVYrnTZowP8iq1F1TgMBBauufyB33x1v+NWFYmT5KmppgHC+NkAgbmRkpD3yn9QIseXymoTQFGQmIOKTxiZIWpvAatenVqRVXf2nTrAWMsPnKrMZHz6bJq5jvce6QK8J1cQNgKxlJapMPdZSR64/UivS9NztpkVEdKcrs5alhhWP9NeqlfWopzhZScI6QxseegZRGeg5a8C3Re1Mfl1ScP36ddcUaMuv24iOJtz7sbUjTS4qBvKmstYJoUauiuD3k5qhyr7QdUHMeCgLa1Ear9NquemdXgmum4fvJ6w1lqsuDhNrg1qSpleJK7K3TF0Q2jSd94uSZ60kK1e3qyVpQK6PVWXp2/FC3mp6jBhKKOiY2h3gtUV64TWM6wDETRPLDfSakXmH3w8g9Jlug8ZtTt4kVF0kLUYYmCCtD/DrQ5YhMGbA9L3ucdjh0y8kOHW5gU/VEEmJTcL4Pz/f7mgoAbYkAAAAAElFTkSuQmCC"]
Jeffrey Morgan's avatar
Jeffrey Morgan committed
729
    }
730
731
732
733
  ]
}'
```

Matt Williams's avatar
Matt Williams committed
734
735
736
737
738
739
740
741
742
743
744
745
##### Response

```json
{
  "model": "llava",
  "created_at": "2023-12-13T22:42:50.203334Z",
  "message": {
    "role": "assistant",
    "content": " The image features a cute, little pig with an angry facial expression. It's wearing a heart on its shirt and is waving in the air. This scene appears to be part of a drawing or sketching project.",
    "images": null
  },
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
746
747
748
749
750
751
  "total_duration": 1668506709,
  "load_duration": 1986209,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 359682000,
  "eval_count": 83,
  "eval_duration": 1303285000
Matt Williams's avatar
Matt Williams committed
752
753
754
}
```

755
756
757
758
759
760
#### Chat request (Reproducible outputs)

##### Request

```shell
curl http://localhost:11434/api/chat -d '{
761
  "model": "llama3.2",
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
  "messages": [
    {
      "role": "user",
      "content": "Hello!"
    }
  ],
  "options": {
    "seed": 101,
    "temperature": 0
  }
}'
```

##### Response

```json
{
779
  "model": "llama3.2",
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
  "created_at": "2023-12-12T14:13:43.416799Z",
  "message": {
    "role": "assistant",
    "content": "Hello! How are you today?"
  },
  "done": true,
  "total_duration": 5191566416,
  "load_duration": 2154458,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 383809000,
  "eval_count": 298,
  "eval_duration": 4799921000
}
```

Michael Yang's avatar
Michael Yang committed
795
796
797
798
#### Chat request (with tools)

##### Request

799
```shell
Michael Yang's avatar
Michael Yang committed
800
curl http://localhost:11434/api/chat -d '{
801
  "model": "llama3.2",
Michael Yang's avatar
Michael Yang committed
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
  "messages": [
    {
      "role": "user",
      "content": "What is the weather today in Paris?"
    }
  ],
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The location to get the weather for, e.g. San Francisco, CA"
            },
            "format": {
              "type": "string",
              "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location", "format"]
        }
      }
    }
  ]
}'
```

##### Response

```json
{
840
  "model": "llama3.2",
Michael Yang's avatar
Michael Yang committed
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
  "created_at": "2024-07-22T20:33:28.123648Z",
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "function": {
          "name": "get_current_weather",
          "arguments": {
            "format": "celsius",
            "location": "Paris, FR"
          }
        }
      }
    ]
  },
  "done_reason": "stop",
  "done": true,
  "total_duration": 885095291,
  "load_duration": 3753500,
  "prompt_eval_count": 122,
  "prompt_eval_duration": 328493000,
  "eval_count": 33,
  "eval_duration": 552222000
}
```

868
869
870
871
872
873
#### Load a model

If the messages array is empty, the model will be loaded into memory.

##### Request

874
```shell
875
curl http://localhost:11434/api/chat -d '{
876
  "model": "llama3.2",
877
878
879
880
881
  "messages": []
}'
```

##### Response
882

883
884
```json
{
885
  "model": "llama3.2",
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
  "created_at":"2024-09-12T21:17:29.110811Z",
  "message": {
    "role": "assistant",
    "content": ""
  },
  "done_reason": "load",
  "done": true
}
```

#### Unload a model

If the messages array is empty and the `keep_alive` parameter is set to `0`, a model will be unloaded from memory.

##### Request

902
```shell
903
curl http://localhost:11434/api/chat -d '{
904
  "model": "llama3.2",
905
906
907
908
909
910
911
912
913
914
915
  "messages": [],
  "keep_alive": 0
}'
```

##### Response

A single JSON object is returned:

```json
{
916
  "model": "llama3.2",
917
918
919
920
921
922
923
924
925
926
  "created_at":"2024-09-12T21:33:17.547535Z",
  "message": {
    "role": "assistant",
    "content": ""
  },
  "done_reason": "unload",
  "done": true
}
```

927
## Create a Model
Matt Williams's avatar
Matt Williams committed
928

929
```
930
931
932
POST /api/create
```

933
934
935
936
937
938
Create a model from:
 * another model;
 * a safetensors directory; or
 * a GGUF file.

If you are creating a model from a safetensors directory or from a GGUF file, you must [create a blob](#create-a-blob) for each of the files and then use the file name and SHA256 digest associated with each blob in the `files` field.
939

940
### Parameters
Matt Williams's avatar
Matt Williams committed
941

Patrick Devine's avatar
Patrick Devine committed
942
- `model`: name of the model to create
943
944
945
946
947
948
949
950
- `from`: (optional) name of an existing model to create the new model from
- `files`: (optional) a dictionary of file names to SHA256 digests of blobs to create the model from
- `adapters`: (optional) a dictionary of file names to SHA256 digests of blobs for LORA adapters
- `template`: (optional) the prompt template for the model
- `license`: (optional) a string or list of strings containing the license or licenses for the model
- `system`: (optional) a string containing the system prompt for the model
- `parameters`: (optional) a dictionary of parameters for the model (see [Modelfile](./modelfile.md#valid-parameters-and-values) for a list of parameters)
- `messages`: (optional) a list of message objects used to create a conversation
951
- `stream`: (optional) if `false` the response will be returned as a single response object, rather than a stream of objects
Patrick Devine's avatar
Patrick Devine committed
952
953
954
955
956
957
958
959
960
- `quantize` (optional): quantize a non-quantized (e.g. float16) model

#### Quantization types

| Type | Recommended |
| --- | :-: |
| q4_K_M | * |
| q4_K_S | |
| q8_0 | * |
Matt Williams's avatar
Matt Williams committed
961

962
963
### Examples

Matt Williams's avatar
Matt Williams committed
964
965
#### Create a new model

966
Create a new model from an existing model.
Matt Williams's avatar
Matt Williams committed
967
968

##### Request
Matt Williams's avatar
Matt Williams committed
969

Matt Williams's avatar
Matt Williams committed
970
```shell
971
curl http://localhost:11434/api/create -d '{
Patrick Devine's avatar
Patrick Devine committed
972
  "model": "mario",
973
974
  "from": "llama3.2",
  "system": "You are Mario from Super Mario Bros."
975
}'
Matt Williams's avatar
Matt Williams committed
976
977
```

Matt Williams's avatar
Matt Williams committed
978
##### Response
Matt Williams's avatar
Matt Williams committed
979

Patrick Devine's avatar
Patrick Devine committed
980
A stream of JSON objects is returned:
Matt Williams's avatar
Matt Williams committed
981

982
```json
Matt Williams's avatar
Matt Williams committed
983
984
985
986
987
988
989
990
991
992
993
{"status":"reading model metadata"}
{"status":"creating system layer"}
{"status":"using already created layer sha256:22f7f8ef5f4c791c1b03d7eb414399294764d7cc82c7e94aa81a1feb80a983a2"}
{"status":"using already created layer sha256:8c17c2ebb0ea011be9981cc3922db8ca8fa61e828c5d3f44cb6ae342bf80460b"}
{"status":"using already created layer sha256:7c23fb36d80141c4ab8cdbb61ee4790102ebd2bf7aeff414453177d4f2110e5d"}
{"status":"using already created layer sha256:2e0493f67d0c8c9c68a8aeacdf6a38a2151cb3c4c1d42accf296e19810527988"}
{"status":"using already created layer sha256:2759286baa875dc22de5394b4a925701b1896a7e3f8e53275c36f75a877a82c9"}
{"status":"writing layer sha256:df30045fe90f0d750db82a058109cecd6d4de9c90a3d75b19c09e5f64580bb42"}
{"status":"writing layer sha256:f18a68eb09bf925bb1b669490407c1b1251c5db98dc4d3d81f3088498ea55690"}
{"status":"writing manifest"}
{"status":"success"}
Matt Williams's avatar
Matt Williams committed
994
995
```

Patrick Devine's avatar
Patrick Devine committed
996
997
998
999
1000
1001
1002
1003
#### Quantize a model

Quantize a non-quantized model.

##### Request

```shell
curl http://localhost:11434/api/create -d '{
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1004
1005
  "model": "llama3.2:quantized",
  "from": "llama3.2:3b-instruct-fp16",
Patrick Devine's avatar
Patrick Devine committed
1006
1007
1008
1009
1010
1011
1012
1013
  "quantize": "q4_K_M"
}'
```

##### Response

A stream of JSON objects is returned:

1014
```json
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1015
1016
1017
1018
1019
1020
1021
{"status":"quantizing F16 model to Q4_K_M","digest":"0","total":6433687776,"completed":12302}
{"status":"quantizing F16 model to Q4_K_M","digest":"0","total":6433687776,"completed":6433687552}
{"status":"verifying conversion"}
{"status":"creating new layer sha256:fb7f4f211b89c6c4928ff4ddb73db9f9c0cfca3e000c3e40d6cf27ddc6ca72eb"}
{"status":"using existing layer sha256:966de95ca8a62200913e3f8bfbf84c8494536f1b94b49166851e76644e966396"}
{"status":"using existing layer sha256:fcc5a6bec9daf9b561a68827b67ab6088e1dba9d1fa2a50d7bbcc8384e0a265d"}
{"status":"using existing layer sha256:a70ff7e570d97baaf4e62ac6e6ad9975e04caa6d900d3742d37698494479e0cd"}
Patrick Devine's avatar
Patrick Devine committed
1022
1023
1024
1025
1026
{"status":"using existing layer sha256:56bb8bd477a519ffa694fc449c2413c6f0e1d3b1c88fa7e3c9d88d3ae49d4dcb"}
{"status":"writing manifest"}
{"status":"success"}
```

1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
#### Create a model from GGUF

Create a model from a GGUF file. The `files` parameter should be filled out with the file name and SHA256 digest of the GGUF file you wish to use. Use [/api/blobs/:digest](#push-a-blob) to push the GGUF file to the server before calling this API.


##### Request

```shell
curl http://localhost:11434/api/create -d '{
  "model": "my-gguf-model",
  "files": {
    "test.gguf": "sha256:432f310a77f4650a88d0fd59ecdd7cebed8d684bafea53cbff0473542964f0c3"
  }
}'
```

##### Response

A stream of JSON objects is returned:

1047
```json
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
{"status":"parsing GGUF"}
{"status":"using existing layer sha256:432f310a77f4650a88d0fd59ecdd7cebed8d684bafea53cbff0473542964f0c3"}
{"status":"writing manifest"}
{"status":"success"}
```


#### Create a model from a Safetensors directory

The `files` parameter should include a dictionary of files for the safetensors model which includes the file names and SHA256 digest of each file. Use [/api/blobs/:digest](#push-a-blob) to first push each of the files to the server before calling this API. Files will remain in the cache until the Ollama server is restarted.
Patrick Devine's avatar
Patrick Devine committed
1058

1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
##### Request

```shell
curl http://localhost:11434/api/create -d '{
  "model": "fred",
  "files": {
    "config.json": "sha256:dd3443e529fb2290423a0c65c2d633e67b419d273f170259e27297219828e389",
    "generation_config.json": "sha256:88effbb63300dbbc7390143fbbdd9d9fa50587b37e8bfd16c8c90d4970a74a36",
    "special_tokens_map.json": "sha256:b7455f0e8f00539108837bfa586c4fbf424e31f8717819a6798be74bef813d05",
    "tokenizer.json": "sha256:bbc1904d35169c542dffbe1f7589a5994ec7426d9e5b609d07bab876f32e97ab",
    "tokenizer_config.json": "sha256:24e8a6dc2547164b7002e3125f10b415105644fcf02bf9ad8b674c87b1eaaed6",
    "model.safetensors": "sha256:1ff795ff6a07e6a68085d206fb84417da2f083f68391c2843cd2b8ac6df8538f"
  }
}'
```

##### Response

A stream of JSON objects is returned:

```shell
{"status":"converting model"}
{"status":"creating new layer sha256:05ca5b813af4a53d2c2922933936e398958855c44ee534858fcfd830940618b6"}
{"status":"using autodetected template llama3-instruct"}
{"status":"using existing layer sha256:56bb8bd477a519ffa694fc449c2413c6f0e1d3b1c88fa7e3c9d88d3ae49d4dcb"}
{"status":"writing manifest"}
{"status":"success"}
```

## Check if a Blob Exists
Michael Yang's avatar
Michael Yang committed
1089
1090
1091
1092
1093

```shell
HEAD /api/blobs/:digest
```

1094
Ensures that the file blob (Binary Large Object) used with create a model exists on the server. This checks your Ollama server and not ollama.com.
Matt Williams's avatar
Matt Williams committed
1095

1096
### Query Parameters
Michael Yang's avatar
Michael Yang committed
1097
1098
1099

- `digest`: the SHA256 digest of the blob

1100
### Examples
Michael Yang's avatar
Michael Yang committed
1101

1102
#### Request
Michael Yang's avatar
Michael Yang committed
1103
1104
1105
1106
1107

```shell
curl -I http://localhost:11434/api/blobs/sha256:29fdb92e57cf0827ded04ae6461b5931d01fa595843f55d36f5b275a52087dd2
```

1108
#### Response
Michael Yang's avatar
Michael Yang committed
1109
1110
1111

Return 200 OK if the blob exists, 404 Not Found if it does not.

1112
## Push a Blob
Michael Yang's avatar
Michael Yang committed
1113

1114
```
Michael Yang's avatar
Michael Yang committed
1115
1116
1117
POST /api/blobs/:digest
```

1118
Push a file to the Ollama server to create a "blob" (Binary Large Object).
Michael Yang's avatar
Michael Yang committed
1119

1120
### Query Parameters
Michael Yang's avatar
Michael Yang committed
1121
1122
1123

- `digest`: the expected SHA256 digest of the file

1124
### Examples
Michael Yang's avatar
Michael Yang committed
1125

1126
#### Request
Michael Yang's avatar
Michael Yang committed
1127

Michael Yang's avatar
Michael Yang committed
1128
```shell
1129
curl -T model.gguf -X POST http://localhost:11434/api/blobs/sha256:29fdb92e57cf0827ded04ae6461b5931d01fa595843f55d36f5b275a52087dd2
Michael Yang's avatar
Michael Yang committed
1130
1131
```

1132
#### Response
Michael Yang's avatar
Michael Yang committed
1133

Matt Williams's avatar
Matt Williams committed
1134
Return 201 Created if the blob was successfully created, 400 Bad Request if the digest used is not expected.
Michael Yang's avatar
Michael Yang committed
1135

1136
## List Local Models
Matt Williams's avatar
Matt Williams committed
1137

1138
```
1139
GET /api/tags
Matt Williams's avatar
Matt Williams committed
1140
1141
```

1142
List models that are available locally.
Matt Williams's avatar
Matt Williams committed
1143

1144
1145
1146
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1147

Matt Williams's avatar
Matt Williams committed
1148
```shell
1149
1150
curl http://localhost:11434/api/tags
```
Matt Williams's avatar
Matt Williams committed
1151

1152
#### Response
Matt Williams's avatar
Matt Williams committed
1153

1154
1155
A single JSON object will be returned.

1156
```json
Matt Williams's avatar
Matt Williams committed
1157
1158
1159
{
  "models": [
    {
1160
1161
1162
1163
1164
1165
1166
1167
1168

      "model": "codellama:13b",
      "modified_at": "2023-11-04T14:56:49.277302595-07:00",
      "size": 7365960935,
      "digest": "9f438cb9cd581fc025612d27f7c1a6669ff83a8bb0ed86c94fcf4c5440555697",
      "capabilities": [
        "completion"
      ],

Matt Williams's avatar
Matt Williams committed
1169
      "details": {
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1170
        "parent_model": "",
Matt Williams's avatar
Matt Williams committed
1171
        "format": "gguf",
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1172
1173
1174
1175
1176
1177
        "family": "qwen2",
        "families": [
          "qwen2"
        ],
        "parameter_size": "7.6B",
        "quantization_level": "Q4_K_M"
Matt Williams's avatar
Matt Williams committed
1178
      }
1179
1180
    },
    {
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190

      "model": "llama4:latest",
      "modified_at": "2023-12-07T09:32:18.757212583-08:00",
      "size": 3825819519,
      "digest": "fe938a131f40e6f6d40083c9f0f430a515233eb2edaa6d72eb85c50d64f2300e",
      "capabilities": [
        "completion",
        "vision"
      ],

Matt Williams's avatar
Matt Williams committed
1191
      "details": {
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1192
        "parent_model": "",
Matt Williams's avatar
Matt Williams committed
1193
1194
        "format": "gguf",
        "family": "llama",
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1195
1196
1197
1198
1199
        "families": [
          "llama"
        ],
        "parameter_size": "3.2B",
        "quantization_level": "Q4_K_M"
Matt Williams's avatar
Matt Williams committed
1200
      }
Matt Williams's avatar
Matt Williams committed
1201
1202
    }
  ]
Matt Williams's avatar
Matt Williams committed
1203
1204
1205
}
```

Matt Williams's avatar
Matt Williams committed
1206
1207
## Show Model Information

1208
```
Matt Williams's avatar
Matt Williams committed
1209
1210
1211
POST /api/show
```

1212
Show information about a model including details, modelfile, template, parameters, license, system prompt.
Matt Williams's avatar
Matt Williams committed
1213
1214
1215

### Parameters

Patrick Devine's avatar
Patrick Devine committed
1216
- `model`: name of the model to show
1217
- `verbose`: (optional) if set to `true`, returns full data for verbose response fields
Matt Williams's avatar
Matt Williams committed
1218

1219
1220
1221
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1222

1223
```shell
Matt Williams's avatar
Matt Williams committed
1224
curl http://localhost:11434/api/show -d '{
1225
  "model": "llava"
Matt Williams's avatar
Matt Williams committed
1226
}'
Matt Williams's avatar
Matt Williams committed
1227
```
Matt Williams's avatar
Matt Williams committed
1228

1229
#### Response
Matt Williams's avatar
Matt Williams committed
1230

1231
```json5
Matt Williams's avatar
Matt Williams committed
1232
{
睡觉型学渣's avatar
睡觉型学渣 committed
1233
  "modelfile": "# Modelfile generated by \"ollama show\"\n# To build a new Modelfile based on this one, replace the FROM line with:\n# FROM llava:latest\n\nFROM /Users/matt/.ollama/models/blobs/sha256:200765e1283640ffbd013184bf496e261032fa75b99498a9613be4e94d63ad52\nTEMPLATE \"\"\"{{ .System }}\nUSER: {{ .Prompt }}\nASSISTANT: \"\"\"\nPARAMETER num_ctx 4096\nPARAMETER stop \"\u003c/s\u003e\"\nPARAMETER stop \"USER:\"\nPARAMETER stop \"ASSISTANT:\"",
1234
1235
  "parameters": "num_keep                       24\nstop                           \"<|start_header_id|>\"\nstop                           \"<|end_header_id|>\"\nstop                           \"<|eot_id|>\"",
  "template": "{{ if .System }}<|start_header_id|>system<|end_header_id|>\n\n{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>\n\n{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>\n\n{{ .Response }}<|eot_id|>",
1236
  "details": {
1237
    "parent_model": "",
1238
    "format": "gguf",
Matt Williams's avatar
Matt Williams committed
1239
    "family": "llama",
1240
1241
1242
1243
    "families": [
      "llama"
    ],
    "parameter_size": "8.0B",
1244
    "quantization_level": "Q4_0"
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
  },
  "model_info": {
    "general.architecture": "llama",
    "general.file_type": 2,
    "general.parameter_count": 8030261248,
    "general.quantization_version": 2,
    "llama.attention.head_count": 32,
    "llama.attention.head_count_kv": 8,
    "llama.attention.layer_norm_rms_epsilon": 0.00001,
    "llama.block_count": 32,
    "llama.context_length": 8192,
    "llama.embedding_length": 4096,
    "llama.feed_forward_length": 14336,
    "llama.rope.dimension_count": 128,
    "llama.rope.freq_base": 500000,
    "llama.vocab_size": 128256,
    "tokenizer.ggml.bos_token_id": 128000,
    "tokenizer.ggml.eos_token_id": 128009,
    "tokenizer.ggml.merges": [],            // populates if `verbose=true`
    "tokenizer.ggml.model": "gpt2",
    "tokenizer.ggml.pre": "llama-bpe",
    "tokenizer.ggml.token_type": [],        // populates if `verbose=true`
    "tokenizer.ggml.tokens": []             // populates if `verbose=true`
1268
1269
1270
1271
1272
  },
  "capabilities": [
    "completion",
    "vision"
  ],
Matt Williams's avatar
Matt Williams committed
1273
1274
1275
1276
1277
}
```

## Copy a Model

1278
```
1279
POST /api/copy
Matt Williams's avatar
Matt Williams committed
1280
```
1281

1282
Copy a model. Creates a model with another name from an existing model.
Matt Williams's avatar
Matt Williams committed
1283

1284
1285
1286
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1287

Matt Williams's avatar
Matt Williams committed
1288
```shell
1289
curl http://localhost:11434/api/copy -d '{
1290
  "source": "llama3.2",
1291
  "destination": "llama3-backup"
Matt Williams's avatar
Matt Williams committed
1292
1293
1294
}'
```

1295
#### Response
1296

Matt Williams's avatar
Matt Williams committed
1297
Returns a 200 OK if successful, or a 404 Not Found if the source model doesn't exist.
1298

Matt Williams's avatar
Matt Williams committed
1299
## Delete a Model
Matt Williams's avatar
Matt Williams committed
1300

1301
```
1302
DELETE /api/delete
Matt Williams's avatar
Matt Williams committed
1303
1304
```

1305
Delete a model and its data.
Matt Williams's avatar
Matt Williams committed
1306

1307
### Parameters
Matt Williams's avatar
Matt Williams committed
1308

Patrick Devine's avatar
Patrick Devine committed
1309
- `model`: model name to delete
Matt Williams's avatar
Matt Williams committed
1310

1311
1312
1313
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1314

Matt Williams's avatar
Matt Williams committed
1315
```shell
1316
curl -X DELETE http://localhost:11434/api/delete -d '{
Patrick Devine's avatar
Patrick Devine committed
1317
  "model": "llama3:13b"
Matt Williams's avatar
Matt Williams committed
1318
1319
1320
}'
```

1321
#### Response
1322

Matt Williams's avatar
Matt Williams committed
1323
Returns a 200 OK if successful, 404 Not Found if the model to be deleted doesn't exist.
1324

1325
## Pull a Model
Matt Williams's avatar
Matt Williams committed
1326

1327
```
1328
1329
1330
POST /api/pull
```

Matt Williams's avatar
Matt Williams committed
1331
Download a model from the ollama library. Cancelled pulls are resumed from where they left off, and multiple calls will share the same download progress.
Matt Williams's avatar
Matt Williams committed
1332

1333
### Parameters
Matt Williams's avatar
Matt Williams committed
1334

Patrick Devine's avatar
Patrick Devine committed
1335
- `model`: name of the model to pull
Matt Williams's avatar
Matt Williams committed
1336
- `insecure`: (optional) allow insecure connections to the library. Only use this if you are pulling from your own library during development.
1337
- `stream`: (optional) if `false` the response will be returned as a single response object, rather than a stream of objects
Matt Williams's avatar
Matt Williams committed
1338

1339
1340
1341
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1342

Matt Williams's avatar
Matt Williams committed
1343
```shell
1344
curl http://localhost:11434/api/pull -d '{
Patrick Devine's avatar
Patrick Devine committed
1345
  "model": "llama3.2"
1346
}'
Matt Williams's avatar
Matt Williams committed
1347
1348
```

1349
#### Response
1350

1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
If `stream` is not specified, or set to `true`, a stream of JSON objects is returned:

The first object is the manifest:

```json
{
  "status": "pulling manifest"
}
```

Then there is a series of downloading responses. Until any of the download is completed, the `completed` key may not be included. The number of files to be downloaded depends on the number of layers specified in the manifest.

1363
```json
Matt Williams's avatar
Matt Williams committed
1364
{
1365
1366
  "status": "downloading digestname",
  "digest": "digestname",
1367
  "total": 2142590208,
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
  "completed": 241970
}
```

After all the files are downloaded, the final responses are:

```json
{
    "status": "verifying sha256 digest"
}
{
    "status": "writing manifest"
}
{
    "status": "removing any unused layers"
}
{
    "status": "success"
}
```

if `stream` is set to false, then the response is a single JSON object:

```json
{
  "status": "success"
Matt Williams's avatar
Matt Williams committed
1394
1395
}
```
1396

Matt Williams's avatar
Matt Williams committed
1397
1398
## Push a Model

1399
```
Matt Williams's avatar
Matt Williams committed
1400
1401
1402
1403
1404
1405
1406
POST /api/push
```

Upload a model to a model library. Requires registering for ollama.ai and adding a public key first.

### Parameters

Patrick Devine's avatar
Patrick Devine committed
1407
- `model`: name of the model to push in the form of `<namespace>/<model>:<tag>`
1408
- `insecure`: (optional) allow insecure connections to the library. Only use this if you are pushing to your library during development.
1409
- `stream`: (optional) if `false` the response will be returned as a single response object, rather than a stream of objects
Matt Williams's avatar
Matt Williams committed
1410

1411
1412
1413
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1414
1415

```shell
1416
curl http://localhost:11434/api/push -d '{
Patrick Devine's avatar
Patrick Devine committed
1417
  "model": "mattw/pygmalion:latest"
Matt Williams's avatar
Matt Williams committed
1418
1419
1420
}'
```

1421
#### Response
1422

1423
If `stream` is not specified, or set to `true`, a stream of JSON objects is returned:
Matt Williams's avatar
Matt Williams committed
1424
1425

```json
1426
{ "status": "retrieving manifest" }
1427
```
Matt Williams's avatar
Matt Williams committed
1428
1429
1430
1431
1432

and then:

```json
{
1433
1434
1435
  "status": "starting upload",
  "digest": "sha256:bc07c81de745696fdf5afca05e065818a8149fb0c77266fb584d9b2cba3711ab",
  "total": 1928429856
Matt Williams's avatar
Matt Williams committed
1436
1437
1438
1439
1440
1441
1442
}
```

Then there is a series of uploading responses:

```json
{
1443
1444
1445
1446
  "status": "starting upload",
  "digest": "sha256:bc07c81de745696fdf5afca05e065818a8149fb0c77266fb584d9b2cba3711ab",
  "total": 1928429856
}
Matt Williams's avatar
Matt Williams committed
1447
1448
1449
1450
1451
1452
1453
1454
1455
```

Finally, when the upload is complete:

```json
{"status":"pushing manifest"}
{"status":"success"}
```

1456
1457
1458
If `stream` is set to `false`, then the response is a single JSON object:

```json
1459
{ "status": "success" }
1460
1461
```

Matt Williams's avatar
Matt Williams committed
1462
1463
## Generate Embeddings

1464
```
royjhan's avatar
royjhan committed
1465
POST /api/embed
1466
1467
1468
1469
1470
1471
1472
```

Generate embeddings from a model

### Parameters

- `model`: name of model to generate embeddings from
royjhan's avatar
royjhan committed
1473
- `input`: text or list of text to generate embeddings for
1474

1475
1476
Advanced parameters:

royjhan's avatar
royjhan committed
1477
- `truncate`: truncates the end of each input to fit within context length. Returns error if `false` and context length is exceeded. Defaults to `true`
1478
- `options`: additional model parameters listed in the documentation for the [Modelfile](./modelfile.md#valid-parameters-and-values) such as `temperature`
Bruce MacDonald's avatar
Bruce MacDonald committed
1479
- `keep_alive`: controls how long the model will stay loaded into memory following the request (default: `5m`)
1480

1481
1482
1483
### Examples

#### Request
1484

Matt Williams's avatar
Matt Williams committed
1485
```shell
royjhan's avatar
royjhan committed
1486
curl http://localhost:11434/api/embed -d '{
Jeffrey Morgan's avatar
Jeffrey Morgan committed
1487
  "model": "all-minilm",
royjhan's avatar
royjhan committed
1488
  "input": "Why is the sky blue?"
1489
1490
1491
}'
```

1492
#### Response
1493
1494
1495

```json
{
royjhan's avatar
royjhan committed
1496
1497
1498
1499
  "model": "all-minilm",
  "embeddings": [[
    0.010071029, -0.0017594862, 0.05007221, 0.04692972, 0.054916814,
    0.008599704, 0.105441414, -0.025878139, 0.12958129, 0.031952348
royjhan's avatar
royjhan committed
1500
1501
1502
1503
  ]],
  "total_duration": 14143917,
  "load_duration": 1019500,
  "prompt_eval_count": 8
royjhan's avatar
royjhan committed
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
}
```

#### Request (Multiple input)

```shell
curl http://localhost:11434/api/embed -d '{
  "model": "all-minilm",
  "input": ["Why is the sky blue?", "Why is the grass green?"]
}'
```

#### Response

```json
{
  "model": "all-minilm",
  "embeddings": [[
    0.010071029, -0.0017594862, 0.05007221, 0.04692972, 0.054916814,
    0.008599704, 0.105441414, -0.025878139, 0.12958129, 0.031952348
  ],[
    -0.0098027075, 0.06042469, 0.025257962, -0.006364387, 0.07272725,
    0.017194884, 0.09032035, -0.051705178, 0.09951512, 0.09072481
  ]]
Costa Alexoglou's avatar
Costa Alexoglou committed
1528
1529
}
```
royjhan's avatar
royjhan committed
1530
1531

## List Running Models
1532
```
royjhan's avatar
royjhan committed
1533
1534
1535
1536
1537
1538
1539
1540
GET /api/ps
```

List models that are currently loaded into memory.

#### Examples

### Request
Jeffrey Morgan's avatar
Jeffrey Morgan committed
1541

royjhan's avatar
royjhan committed
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
```shell
curl http://localhost:11434/api/ps
```

#### Response

A single JSON object will be returned.

```json
{
  "models": [
    {
      "name": "mistral:latest",
      "model": "mistral:latest",
      "size": 5137025024,
      "digest": "2ae6f6dd7a3dd734790bbbf58b8909a606e0e7e97e94b7604e0aa7ae4490e6d8",
      "details": {
        "parent_model": "",
        "format": "gguf",
        "family": "llama",
        "families": [
          "llama"
        ],
        "parameter_size": "7.2B",
        "quantization_level": "Q4_0"
      },
      "expires_at": "2024-06-04T14:38:31.83753-07:00",
      "size_vram": 5137025024
    }
  ]
}
Jeffrey Morgan's avatar
Jeffrey Morgan committed
1573
```
royjhan's avatar
royjhan committed
1574
1575
1576
1577
1578

## Generate Embedding

> Note: this endpoint has been superseded by `/api/embed`

1579
```
royjhan's avatar
royjhan committed
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
POST /api/embeddings
```

Generate embeddings from a model

### Parameters

- `model`: name of model to generate embeddings from
- `prompt`: text to generate embeddings for

Advanced parameters:

- `options`: additional model parameters listed in the documentation for the [Modelfile](./modelfile.md#valid-parameters-and-values) such as `temperature`
- `keep_alive`: controls how long the model will stay loaded into memory following the request (default: `5m`)

### Examples

#### Request

```shell
curl http://localhost:11434/api/embeddings -d '{
  "model": "all-minilm",
  "prompt": "Here is an article about llamas..."
}'
```

#### Response

```json
{
  "embedding": [
    0.5670403838157654, 0.009260174818336964, 0.23178744316101074, -0.2916173040866852, -0.8924556970596313,
    0.8785552978515625, -0.34576427936553955, 0.5742510557174683, -0.04222835972905159, -0.137906014919281
  ]
}
Jeffrey Morgan's avatar
Jeffrey Morgan committed
1615
```
1616
1617
1618

## Version

1619
```
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
GET /api/version
```

Retrieve the Ollama version

### Examples

#### Request

```shell
curl http://localhost:11434/api/version
```

#### Response

```json
{
  "version": "0.5.1"
}
```