api.md 53.3 KB
Newer Older
1
2
# API

3
4
5
## Endpoints

- [Generate a completion](#generate-a-completion)
6
- [Generate a chat completion](#generate-a-chat-completion)
Matt Williams's avatar
Matt Williams committed
7
8
9
10
11
12
13
14
- [Create a Model](#create-a-model)
- [List Local Models](#list-local-models)
- [Show Model Information](#show-model-information)
- [Copy a Model](#copy-a-model)
- [Delete a Model](#delete-a-model)
- [Pull a Model](#pull-a-model)
- [Push a Model](#push-a-model)
- [Generate Embeddings](#generate-embeddings)
royjhan's avatar
royjhan committed
15
- [List Running Models](#list-running-models)
16
- [Version](#version)
Matt Williams's avatar
Matt Williams committed
17

18
## Conventions
Matt Williams's avatar
Matt Williams committed
19

20
### Model names
Matt Williams's avatar
Matt Williams committed
21

Daniel Hiltgen's avatar
Daniel Hiltgen committed
22
Model names follow a `model:tag` format, where `model` can have an optional namespace such as `example/model`. Some examples are `orca-mini:3b-q8_0` and `llama3:70b`. The tag is optional and, if not provided, will default to `latest`. The tag is used to identify a specific version.
Matt Williams's avatar
Matt Williams committed
23
24
25

### Durations

26
All durations are returned in nanoseconds.
Matt Williams's avatar
Matt Williams committed
27

28
29
### Streaming responses

Jeffrey Morgan's avatar
Jeffrey Morgan committed
30
Certain endpoints stream responses as JSON objects. Streaming can be disabled by providing `{"stream": false}` for these endpoints.
Matt Williams's avatar
Matt Williams committed
31

32
## Generate a completion
Matt Williams's avatar
Matt Williams committed
33

34
```
35
36
POST /api/generate
```
37

Bruce MacDonald's avatar
Bruce MacDonald committed
38
Generate a response for a given prompt with a provided model. This is a streaming endpoint, so there will be a series of responses. The final response object will include statistics and additional data from the request.
Matt Williams's avatar
Matt Williams committed
39

40
### Parameters
Matt Williams's avatar
Matt Williams committed
41

42
43
- `model`: (required) the [model name](#model-names)
- `prompt`: the prompt to generate a response for
Michael Yang's avatar
Michael Yang committed
44
- `suffix`: the text after the model response
Matt Williams's avatar
Matt Williams committed
45
- `images`: (optional) a list of base64-encoded images (for multimodal models such as `llava`)
46
- `think`: (for thinking models) should the model think before responding?
Matt Williams's avatar
Matt Williams committed
47

48
Advanced parameters (optional):
Matt Williams's avatar
Matt Williams committed
49

50
- `format`: the format to return a response in. Format can be `json` or a JSON schema
51
- `options`: additional model parameters listed in the documentation for the [Modelfile](./modelfile.md#valid-parameters-and-values) such as `temperature`
52
- `system`: system message to (overrides what is defined in the `Modelfile`)
53
- `template`: the prompt template to use (overrides what is defined in the `Modelfile`)
54
- `stream`: if `false` the response will be returned as a single response object, rather than a stream of objects
Bruce MacDonald's avatar
Bruce MacDonald committed
55
56
- `raw`: if `true` no formatting will be applied to the prompt. You may choose to use the `raw` parameter if you are specifying a full templated prompt in your request to the API
- `keep_alive`: controls how long the model will stay loaded into memory following the request (default: `5m`)
57
- `context` (deprecated): the context parameter returned from a previous request to `/generate`, this can be used to keep a short conversational memory
58

59
60
61
62
#### Structured outputs

Structured outputs are supported by providing a JSON schema in the `format` parameter. The model will generate a response that matches the schema. See the [structured outputs](#request-structured-outputs) example below.

Matt Williams's avatar
Matt Williams committed
63
#### JSON mode
64

Jeffrey Morgan's avatar
Jeffrey Morgan committed
65
Enable JSON mode by setting the `format` parameter to `json`. This will structure the response as a valid JSON object. See the JSON mode [example](#request-json-mode) below.
66

Michael Yang's avatar
Michael Yang committed
67
68
> [!IMPORTANT]
> It's important to instruct the model to use JSON in the `prompt`. Otherwise, the model may generate large amounts whitespace.
69

70
71
### Examples

Matt Williams's avatar
Matt Williams committed
72
73
74
#### Generate request (Streaming)

##### Request
Matt Williams's avatar
Matt Williams committed
75

Matt Williams's avatar
Matt Williams committed
76
```shell
77
curl http://localhost:11434/api/generate -d '{
78
  "model": "llama3.2",
79
80
81
  "prompt": "Why is the sky blue?"
}'
```
Matt Williams's avatar
Matt Williams committed
82

Matt Williams's avatar
Matt Williams committed
83
##### Response
Matt Williams's avatar
Matt Williams committed
84

85
A stream of JSON objects is returned:
Matt Williams's avatar
Matt Williams committed
86

87
```json
Matt Williams's avatar
Matt Williams committed
88
{
89
  "model": "llama3.2",
90
91
  "created_at": "2023-08-04T08:52:19.385406455-07:00",
  "response": "The",
Matt Williams's avatar
Matt Williams committed
92
93
94
95
  "done": false
}
```

96
The final response in the stream also includes additional data about the generation:
Matt Williams's avatar
Matt Williams committed
97

98
99
100
101
- `total_duration`: time spent generating the response
- `load_duration`: time spent in nanoseconds loading the model
- `prompt_eval_count`: number of tokens in the prompt
- `prompt_eval_duration`: time spent in nanoseconds evaluating the prompt
Sri Siddhaarth's avatar
Sri Siddhaarth committed
102
- `eval_count`: number of tokens in the response
103
- `eval_duration`: time in nanoseconds spent generating the response
104
- `context`: an encoding of the conversation used in this response, this can be sent in the next request to keep a conversational memory
105
- `response`: empty if the response was streamed, if not streamed, this will contain the full response
106

Darinka's avatar
Darinka committed
107
To calculate how fast the response is generated in tokens per second (token/s), divide `eval_count` / `eval_duration` * `10^9`.
108

109
```json
Matt Williams's avatar
Matt Williams committed
110
{
111
  "model": "llama3.2",
112
  "created_at": "2023-08-04T19:22:45.499127Z",
113
  "response": "",
114
  "done": true,
Matt Williams's avatar
Matt Williams committed
115
  "context": [1, 2, 3],
Jeffrey Morgan's avatar
Jeffrey Morgan committed
116
117
118
119
120
121
  "total_duration": 10706818083,
  "load_duration": 6338219291,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 130079000,
  "eval_count": 259,
  "eval_duration": 4232710000
122
}
Matt Williams's avatar
Matt Williams committed
123
124
```

125
#### Request (No streaming)
126

Matt Williams's avatar
Matt Williams committed
127
128
129
##### Request

A response can be received in one reply when streaming is off.
Bruce MacDonald's avatar
Bruce MacDonald committed
130

131
```shell
132
curl http://localhost:11434/api/generate -d '{
133
  "model": "llama3.2",
134
135
136
137
138
  "prompt": "Why is the sky blue?",
  "stream": false
}'
```

Matt Williams's avatar
Matt Williams committed
139
##### Response
140

141
142
143
144
If `stream` is set to `false`, the response will be a single JSON object:

```json
{
145
  "model": "llama3.2",
146
147
  "created_at": "2023-08-04T19:22:45.499127Z",
  "response": "The sky is blue because it is the color of the sky.",
Matt Williams's avatar
Matt Williams committed
148
  "done": true,
149
  "context": [1, 2, 3],
Matt Williams's avatar
Matt Williams committed
150
151
152
153
154
155
156
157
158
  "total_duration": 5043500667,
  "load_duration": 5025959,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 325953000,
  "eval_count": 290,
  "eval_duration": 4709213000
}
```

Michael Yang's avatar
Michael Yang committed
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
#### Request (with suffix)

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
  "model": "codellama:code",
  "prompt": "def compute_gcd(a, b):",
  "suffix": "    return result",
  "options": {
    "temperature": 0
  },
  "stream": false
}'
```

##### Response

177
```json5
Michael Yang's avatar
Michael Yang committed
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
{
  "model": "codellama:code",
  "created_at": "2024-07-22T20:47:51.147561Z",
  "response": "\n  if a == 0:\n    return b\n  else:\n    return compute_gcd(b % a, a)\n\ndef compute_lcm(a, b):\n  result = (a * b) / compute_gcd(a, b)\n",
  "done": true,
  "done_reason": "stop",
  "context": [...],
  "total_duration": 1162761250,
  "load_duration": 6683708,
  "prompt_eval_count": 17,
  "prompt_eval_duration": 201222000,
  "eval_count": 63,
  "eval_duration": 953997000
}
```

194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
#### Request (Structured outputs)

##### Request

```shell
curl -X POST http://localhost:11434/api/generate -H "Content-Type: application/json" -d '{
  "model": "llama3.1:8b",
  "prompt": "Ollama is 22 years old and is busy saving the world. Respond using JSON",
  "stream": false,
  "format": {
    "type": "object",
    "properties": {
      "age": {
        "type": "integer"
      },
      "available": {
        "type": "boolean"
      }
    },
    "required": [
      "age",
      "available"
    ]
  }
}'
```

##### Response

```json
{
  "model": "llama3.1:8b",
  "created_at": "2024-12-06T00:48:09.983619Z",
  "response": "{\n  \"age\": 22,\n  \"available\": true\n}",
  "done": true,
  "done_reason": "stop",
  "context": [1, 2, 3],
  "total_duration": 1075509083,
  "load_duration": 567678166,
  "prompt_eval_count": 28,
  "prompt_eval_duration": 236000000,
  "eval_count": 16,
  "eval_duration": 269000000
}
```

Matt Williams's avatar
Matt Williams committed
240
241
#### Request (JSON mode)

Michael Yang's avatar
Michael Yang committed
242
> [!IMPORTANT]
Matt Williams's avatar
Matt Williams committed
243
244
245
246
247
248
> When `format` is set to `json`, the output will always be a well-formed JSON object. It's important to also instruct the model to respond in JSON.

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
249
  "model": "llama3.2",
Matt Williams's avatar
Matt Williams committed
250
251
252
253
254
255
256
257
258
259
  "prompt": "What color is the sky at different times of the day? Respond using JSON",
  "format": "json",
  "stream": false
}'
```

##### Response

```json
{
260
  "model": "llama3.2",
Matt Williams's avatar
Matt Williams committed
261
262
  "created_at": "2023-11-09T21:07:55.186497Z",
  "response": "{\n\"morning\": {\n\"color\": \"blue\"\n},\n\"noon\": {\n\"color\": \"blue-gray\"\n},\n\"afternoon\": {\n\"color\": \"warm gray\"\n},\n\"evening\": {\n\"color\": \"orange\"\n}\n}\n",
263
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
264
  "context": [1, 2, 3],
Matt Williams's avatar
Matt Williams committed
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
  "total_duration": 4648158584,
  "load_duration": 4071084,
  "prompt_eval_count": 36,
  "prompt_eval_duration": 439038000,
  "eval_count": 180,
  "eval_duration": 4196918000
}
```

The value of `response` will be a string containing JSON similar to:

```json
{
  "morning": {
    "color": "blue"
  },
  "noon": {
    "color": "blue-gray"
  },
  "afternoon": {
    "color": "warm gray"
  },
  "evening": {
    "color": "orange"
  }
290
291
292
}
```

293
294
295
296
#### Request (with images)

To submit images to multimodal models such as `llava` or `bakllava`, provide a list of base64-encoded `images`:

Matt Williams's avatar
Matt Williams committed
297
298
#### Request

299
300
301
302
303
304
305
306
307
308
309
```shell
curl http://localhost:11434/api/generate -d '{
  "model": "llava",
  "prompt":"What is in this picture?",
  "stream": false,
  "images": ["iVBORw0KGgoAAAANSUhEUgAAAG0AAABmCAYAAADBPx+VAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAA3VSURBVHgB7Z27r0zdG8fX743i1bi1ikMoFMQloXRpKFFIqI7LH4BEQ+NWIkjQuSWCRIEoULk0gsK1kCBI0IhrQVT7tz/7zZo888yz1r7MnDl7z5xvsjkzs2fP3uu71nNfa7lkAsm7d++Sffv2JbNmzUqcc8m0adOSzZs3Z+/XES4ZckAWJEGWPiCxjsQNLWmQsWjRIpMseaxcuTKpG/7HP27I8P79e7dq1ars/yL4/v27S0ejqwv+cUOGEGGpKHR37tzJCEpHV9tnT58+dXXCJDdECBE2Ojrqjh071hpNECjx4cMHVycM1Uhbv359B2F79+51586daxN/+pyRkRFXKyRDAqxEp4yMlDDzXG1NPnnyJKkThoK0VFd1ELZu3TrzXKxKfW7dMBQ6bcuWLW2v0VlHjx41z717927ba22U9APcw7Nnz1oGEPeL3m3p2mTAYYnFmMOMXybPPXv2bNIPpFZr1NHn4HMw0KRBjg9NuRw95s8PEcz/6DZELQd/09C9QGq5RsmSRybqkwHGjh07OsJSsYYm3ijPpyHzoiacg35MLdDSIS/O1yM778jOTwYUkKNHWUzUWaOsylE00MyI0fcnOwIdjvtNdW/HZwNLGg+sR1kMepSNJXmIwxBZiG8tDTpEZzKg0GItNsosY8USkxDhD0Rinuiko2gfL/RbiD2LZAjU9zKQJj8RDR0vJBR1/Phx9+PHj9Z7REF4nTZkxzX4LCXHrV271qXkBAPGfP/atWvu/PnzHe4C97F48eIsRLZ9+3a3f/9+87dwP1JxaF7/3r17ba+5l4EcaVo0lj3SBq5kGTJSQmLWMjgYNei2GPT1MuMqGTDEFHzeQSP2wi/jGnkmPJ/nhccs44jvDAxpVcxnq0F6eT8h4ni/iIWpR5lPyA6ETkNXoSukvpJAD3AsXLiwpZs49+fPn5ke4j10TqYvegSfn0OnafC+Tv9ooA/JPkgQysqQNBzagXY55nO/oa1F7qvIPWkRL12WRpMWUvpVDYmxAPehxWSe8ZEXL20sadYIozfmNch4QJPAfeJgW3rNsnzphBKNJM2KKODo1rVOMRYik5ETy3ix4qWNI81qAAirizgMIc+yhTytx0JWZuNI03qsrgWlGtwjoS9XwgUhWGyhUaRZZQNNIEwCiXD16tXcAHUs79co0vSD8rrJCIW98pzvxpAWyyo3HYwqS0+H0BjStClcZJT5coMm6D2LOF8TolGJtK9fvyZpyiC5ePFi9nc/oJU4eiEP0jVoAnHa9wyJycITMP78+eMeP37sXrx44d6+fdt6f82aNdkx1pg9e3Zb5W+RSRE+n+VjksQWifvVaTKFhn5O8my63K8Qabdv33b379/PiAP//vuvW7BggZszZ072/+TJk91YgkafPn166zXB1rQHFvouAWHq9z3SEevSUerqCn2/dDCeta2jxYbr69evk4MHDyY7d+7MjhMnTiTPnz9Pfv/+nfQT2ggpO2dMF8cghuoM7Ygj5iWCqRlGFml0QC/ftGmTmzt3rmsaKDsgBSPh0/8yPeLLBihLkOKJc0jp8H8vUzcxIA1k6QJ/c78tWEyj5P3o4u9+jywNPdJi5rAH9x0KHcl4Hg570eQp3+vHXGyrmEeigzQsQsjavXt38ujRo44LQuDDhw+TW7duRS1HGgMxhNXHgflaNTOsHyKvHK5Ijo2jbFjJBQK9YwFd6RVMzfgRBmEfP37suBBm/p49e1qjEP2mwTViNRo0VJWH1deMXcNK08uUjVUu7s/zRaL+oLNxz1bpANco4npUgX4G2eFbpDFyQoQxojBCpEGSytmOH8qrH5Q9vuzD6ofQylkCUmh8DBAr+q8JCyVNtWQIidKQE9wNtLSQnS4jDSsxNHogzFuQBw4cyM61UKVsjfr3ooBkPSqqQHesUPWVtzi9/vQi1T+rJj7WiTz4Pt/l3LxUkr5P2VYZaZ4URpsE+st/dujQoaBBYokbrz/8TJNQYLSonrPS9kUaSkPeZyj1AWSj+d+VBoy1pIWVNed8P0Ll/ee5HdGRhrHhR5GGN0r4LGZBaj8oFDJitBTJzIZgFcmU0Y8ytWMZMzJOaXUSrUs5RxKnrxmbb5YXO9VGUhtpXldhEUogFr3IzIsvlpmdosVcGVGXFWp2oU9kLFL3dEkSz6NHEY1sjSRdIuDFWEhd8KxFqsRi1uM/nz9/zpxnwlESONdg6dKlbsaMGS4EHFHtjFIDHwKOo46l4TxSuxgDzi+rE2jg+BaFruOX4HXa0Nnf1lwAPufZeF8/r6zD97WK2qFnGjBxTw5qNGPxT+5T/r7/7RawFC3j4vTp09koCxkeHjqbHJqArmH5UrFKKksnxrK7FuRIs8STfBZv+luugXZ2pR/pP9Ois4z+TiMzUUkUjD0iEi1fzX8GmXyuxUBRcaUfykV0YZnlJGKQpOiGB76x5GeWkWWJc3mOrK6S7xdND+W5N6XyaRgtWJFe13GkaZnKOsYqGdOVVVbGupsyA/l7emTLHi7vwTdirNEt0qxnzAvBFcnQF16xh/TMpUuXHDowhlA9vQVraQhkudRdzOnK+04ZSP3DUhVSP61YsaLtd/ks7ZgtPcXqPqEafHkdqa84X6aCeL7YWlv6edGFHb+ZFICPlljHhg0bKuk0CSvVznWsotRu433alNdFrqG45ejoaPCaUkWERpLXjzFL2Rpllp7PJU2a/v7Ab8N05/9t27Z16KUqoFGsxnI9EosS2niSYg9SpU6B4JgTrvVW1flt1sT+0ADIJU2maXzcUTraGCRaL1Wp9rUMk16PMom8QhruxzvZIegJjFU7LLCePfS8uaQdPny4jTTL0dbee5mYokQsXTIWNY46kuMbnt8Kmec+LGWtOVIl9cT1rCB0V8WqkjAsRwta93TbwNYoGKsUSChN44lgBNCoHLHzquYKrU6qZ8lolCIN0Rh6cP0Q3U6I6IXILYOQI513hJaSKAorFpuHXJNfVlpRtmYBk1Su1obZr5dnKAO+L10Hrj3WZW+E3qh6IszE37F6EB+68mGpvKm4eb9bFrlzrok7fvr0Kfv727dvWRmdVTJHw0qiiCUSZ6wCK+7XL/AcsgNyL74DQQ730sv78Su7+t/A36MdY0sW5o40ahslXr58aZ5HtZB8GH64m9EmMZ7FpYw4T6QnrZfgenrhFxaSiSGXtPnz57e9TkNZLvTjeqhr734CNtrK41L40sUQckmj1lGKQ0rC37x544r8eNXRpnVE3ZZY7zXo8NomiO0ZUCj2uHz58rbXoZ6gc0uA+F6ZeKS/jhRDUq8MKrTho9fEkihMmhxtBI1DxKFY9XLpVcSkfoi8JGnToZO5sU5aiDQIW716ddt7ZLYtMQlhECdBGXZZMWldY5BHm5xgAroWj4C0hbYkSc/jBmggIrXJWlZM6pSETsEPGqZOndr2uuuR5rF169a2HoHPdurUKZM4CO1WTPqaDaAd+GFGKdIQkxAn9RuEWcTRyN2KSUgiSgF5aWzPTeA/lN5rZubMmR2bE4SIC4nJoltgAV/dVefZm72AtctUCJU2CMJ327hxY9t7EHbkyJFseq+EJSY16RPo3Dkq1kkr7+q0bNmyDuLQcZBEPYmHVdOBiJyIlrRDq41YPWfXOxUysi5fvtyaj+2BpcnsUV/oSoEMOk2CQGlr4ckhBwaetBhjCwH0ZHtJROPJkyc7UjcYLDjmrH7ADTEBXFfOYmB0k9oYBOjJ8b4aOYSe7QkKcYhFlq3QYLQhSidNmtS2RATwy8YOM3EQJsUjKiaWZ+vZToUQgzhkHXudb/PW5YMHD9yZM2faPsMwoc7RciYJXbGuBqJ1UIGKKLv915jsvgtJxCZDubdXr165mzdvtr1Hz5LONA8jrUwKPqsmVesKa49S3Q4WxmRPUEYdTjgiUcfUwLx589ySJUva3oMkP6IYddq6HMS4o55xBJBUeRjzfa4Zdeg56QZ43LhxoyPo7Lf1kNt7oO8wWAbNwaYjIv5lhyS7kRf96dvm5Jah8vfvX3flyhX35cuX6HfzFHOToS1H4BenCaHvO8pr8iDuwoUL7tevX+b5ZdbBair0xkFIlFDlW4ZknEClsp/TzXyAKVOmmHWFVSbDNw1l1+4f90U6IY/q4V27dpnE9bJ+v87QEydjqx/UamVVPRG+mwkNTYN+9tjkwzEx+atCm/X9WvWtDtAb68Wy9LXa1UmvCDDIpPkyOQ5ZwSzJ4jMrvFcr0rSjOUh+GcT4LSg5ugkW1Io0/SCDQBojh0hPlaJdah+tkVYrnTZowP8iq1F1TgMBBauufyB33x1v+NWFYmT5KmppgHC+NkAgbmRkpD3yn9QIseXymoTQFGQmIOKTxiZIWpvAatenVqRVXf2nTrAWMsPnKrMZHz6bJq5jvce6QK8J1cQNgKxlJapMPdZSR64/UivS9NztpkVEdKcrs5alhhWP9NeqlfWopzhZScI6QxseegZRGeg5a8C3Re1Mfl1ScP36ddcUaMuv24iOJtz7sbUjTS4qBvKmstYJoUauiuD3k5qhyr7QdUHMeCgLa1Ear9NquemdXgmum4fvJ6w1lqsuDhNrg1qSpleJK7K3TF0Q2jSd94uSZ60kK1e3qyVpQK6PVWXp2/FC3mp6jBhKKOiY2h3gtUV64TWM6wDETRPLDfSakXmH3w8g9Jlug8ZtTt4kVF0kLUYYmCCtD/DrQ5YhMGbA9L3ucdjh0y8kOHW5gU/VEEmJTcL4Pz/f7mgoAbYkAAAAAElFTkSuQmCC"]
}'
```

#### Response

310
```json
311
312
313
314
315
{
  "model": "llava",
  "created_at": "2023-11-03T15:36:02.583064Z",
  "response": "A happy cartoon character, which is cute and cheerful.",
  "done": true,
Matt Williams's avatar
Matt Williams committed
316
317
318
319
320
321
322
  "context": [1, 2, 3],
  "total_duration": 2938432250,
  "load_duration": 2559292,
  "prompt_eval_count": 1,
  "prompt_eval_duration": 2195557000,
  "eval_count": 44,
  "eval_duration": 736432000
323
324
325
}
```

Bruce MacDonald's avatar
Bruce MacDonald committed
326
#### Request (Raw Mode)
327

Matt Williams's avatar
Matt Williams committed
328
In some cases, you may wish to bypass the templating system and provide a full prompt. In this case, you can use the `raw` parameter to disable templating. Also note that raw mode will not return a context.
Jeffrey Morgan's avatar
Jeffrey Morgan committed
329

Matt Williams's avatar
Matt Williams committed
330
##### Request
331
332

```shell
333
curl http://localhost:11434/api/generate -d '{
334
335
336
337
338
339
340
  "model": "mistral",
  "prompt": "[INST] why is the sky blue? [/INST]",
  "raw": true,
  "stream": false
}'
```

341
342
#### Request (Reproducible outputs)

343
For reproducible outputs, set `seed` to a number:
344
345
346
347
348
349

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
  "model": "mistral",
Jeffrey Morgan's avatar
Jeffrey Morgan committed
350
  "prompt": "Why is the sky blue?",
351
  "options": {
352
    "seed": 123
353
354
355
356
  }
}'
```

Matt Williams's avatar
Matt Williams committed
357
##### Response
358
359
360
361
362
363
364

```json
{
  "model": "mistral",
  "created_at": "2023-11-03T15:36:02.583064Z",
  "response": " The sky appears blue because of a phenomenon called Rayleigh scattering.",
  "done": true,
Matt Williams's avatar
Matt Williams committed
365
366
  "total_duration": 8493852375,
  "load_duration": 6589624375,
367
  "prompt_eval_count": 14,
Matt Williams's avatar
Matt Williams committed
368
369
370
  "prompt_eval_duration": 119039000,
  "eval_count": 110,
  "eval_duration": 1779061000
371
372
373
}
```

Matt Williams's avatar
Matt Williams committed
374
#### Generate request (With options)
375
376
377

If you want to set custom options for the model at runtime rather than in the Modelfile, you can do so with the `options` parameter. This example sets every available option, but you can set any of them individually and omit the ones you do not want to override.

Matt Williams's avatar
Matt Williams committed
378
379
##### Request

380
```shell
381
curl http://localhost:11434/api/generate -d '{
382
  "model": "llama3.2",
383
384
385
386
387
388
389
390
  "prompt": "Why is the sky blue?",
  "stream": false,
  "options": {
    "num_keep": 5,
    "seed": 42,
    "num_predict": 100,
    "top_k": 20,
    "top_p": 0.9,
391
    "min_p": 0.0,
392
393
394
395
396
397
398
399
400
    "typical_p": 0.7,
    "repeat_last_n": 33,
    "temperature": 0.8,
    "repeat_penalty": 1.2,
    "presence_penalty": 1.5,
    "frequency_penalty": 1.0,
    "penalize_newline": true,
    "stop": ["\n", "user:"],
    "numa": false,
401
    "num_ctx": 1024,
402
403
404
405
    "num_batch": 2,
    "num_gpu": 1,
    "main_gpu": 0,
    "use_mmap": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
406
    "num_thread": 8
407
  }
408
409
410
}'
```

Matt Williams's avatar
Matt Williams committed
411
##### Response
412
413
414

```json
{
415
  "model": "llama3.2",
416
417
418
  "created_at": "2023-08-04T19:22:45.499127Z",
  "response": "The sky is blue because it is the color of the sky.",
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
419
  "context": [1, 2, 3],
Matt Williams's avatar
Matt Williams committed
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
  "total_duration": 4935886791,
  "load_duration": 534986708,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 107345000,
  "eval_count": 237,
  "eval_duration": 4289432000
}
```

#### Load a model

If an empty prompt is provided, the model will be loaded into memory.

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
437
  "model": "llama3.2"
Matt Williams's avatar
Matt Williams committed
438
439
440
441
442
443
444
445
446
}'
```

##### Response

A single JSON object is returned:

```json
{
447
  "model": "llama3.2",
Jeffrey Morgan's avatar
Jeffrey Morgan committed
448
449
450
  "created_at": "2023-12-18T19:52:07.071755Z",
  "response": "",
  "done": true
451
452
453
}
```

454
455
456
457
458
459
460
461
#### Unload a model

If an empty prompt is provided and the `keep_alive` parameter is set to `0`, a model will be unloaded from memory.

##### Request

```shell
curl http://localhost:11434/api/generate -d '{
462
  "model": "llama3.2",
463
464
465
466
467
468
469
470
471
472
  "keep_alive": 0
}'
```

##### Response

A single JSON object is returned:

```json
{
473
  "model": "llama3.2",
474
475
476
477
478
479
480
  "created_at": "2024-09-12T03:54:03.516566Z",
  "response": "",
  "done": true,
  "done_reason": "unload"
}
```

481
## Generate a chat completion
482

483
```
Bruce MacDonald's avatar
Bruce MacDonald committed
484
485
486
POST /api/chat
```

Matt Williams's avatar
Matt Williams committed
487
Generate the next message in a chat with a provided model. This is a streaming endpoint, so there will be a series of responses. Streaming can be disabled using `"stream": false`. The final response object will include statistics and additional data from the request.
Bruce MacDonald's avatar
Bruce MacDonald committed
488
489
490
491
492

### Parameters

- `model`: (required) the [model name](#model-names)
- `messages`: the messages of the chat, this can be used to keep a chat memory
493
- `tools`: list of tools in JSON for the model to use if supported
494
- `think`: (for thinking models) should the model think before responding?
Bruce MacDonald's avatar
Bruce MacDonald committed
495

496
497
The `message` object has the following fields:

Michael Yang's avatar
Michael Yang committed
498
- `role`: the role of the message, either `system`, `user`, `assistant`, or `tool`
499
- `content`: the content of the message
500
- `thinking`: (for thinking models) the model's thinking process
501
- `images` (optional): a list of images to include in the message (for multimodal models such as `llava`)
502
- `tool_calls` (optional): a list of tools in JSON that the model wants to use
503
- `tool_name` (optional): add the name of the tool that was executed to inform the model of the result
504

Bruce MacDonald's avatar
Bruce MacDonald committed
505
506
Advanced parameters (optional):

507
- `format`: the format to return a response in. Format can be `json` or a JSON schema.
Bruce MacDonald's avatar
Bruce MacDonald committed
508
509
- `options`: additional model parameters listed in the documentation for the [Modelfile](./modelfile.md#valid-parameters-and-values) such as `temperature`
- `stream`: if `false` the response will be returned as a single response object, rather than a stream of objects
Bruce MacDonald's avatar
Bruce MacDonald committed
510
- `keep_alive`: controls how long the model will stay loaded into memory following the request (default: `5m`)
Bruce MacDonald's avatar
Bruce MacDonald committed
511

512
513
514
515
516
517
518
519
### Tool calling

Tool calling is supported by providing a list of tools in the `tools` parameter. The model will generate a response that includes a list of tool calls. See the [Chat request (Streaming with tools)](#chat-request-streaming-with-tools) example below.

Models can also explain the result of the tool call in the response. See the [Chat request (With history, with tools)](#chat-request-with-history-with-tools) example below.

[See models with tool calling capabilities](https://ollama.com/search?c=tool).

520
521
522
523
### Structured outputs

Structured outputs are supported by providing a JSON schema in the `format` parameter. The model will generate a response that matches the schema. See the [Chat request (Structured outputs)](#chat-request-structured-outputs) example below.

Bruce MacDonald's avatar
Bruce MacDonald committed
524
525
### Examples

526
#### Chat request (Streaming)
Matt Williams's avatar
Matt Williams committed
527
528

##### Request
529

Bruce MacDonald's avatar
Bruce MacDonald committed
530
531
532
Send a chat message with a streaming response.

```shell
533
curl http://localhost:11434/api/chat -d '{
534
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
535
536
537
538
539
540
541
542
543
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
  ]
}'
```

Matt Williams's avatar
Matt Williams committed
544
##### Response
Bruce MacDonald's avatar
Bruce MacDonald committed
545
546
547
548
549

A stream of JSON objects is returned:

```json
{
550
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
551
552
  "created_at": "2023-08-04T08:52:19.385406455-07:00",
  "message": {
Robin Glauser's avatar
Robin Glauser committed
553
    "role": "assistant",
Jeffrey Morgan's avatar
Jeffrey Morgan committed
554
    "content": "The",
Matt Williams's avatar
Matt Williams committed
555
    "images": null
Bruce MacDonald's avatar
Bruce MacDonald committed
556
557
558
559
560
561
562
563
564
  },
  "done": false
}
```

Final response:

```json
{
565
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
566
  "created_at": "2023-08-04T19:22:45.499127Z",
567
568
569
570
  "message": {
    "role": "assistant",
    "content": ""
  },
Bruce MacDonald's avatar
Bruce MacDonald committed
571
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
572
573
574
575
576
577
  "total_duration": 4883583458,
  "load_duration": 1334875,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 342546000,
  "eval_count": 282,
  "eval_duration": 4535599000
Bruce MacDonald's avatar
Bruce MacDonald committed
578
579
580
}
```

581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
#### Chat request (Streaming with tools)

##### Request

```shell
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    {
      "role": "user",
      "content": "what is the weather in tokyo?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the weather in a given city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string",
              "description": "The city to get the weather for"
            }
          },
          "required": ["city"]
        }
      }
    }
  ],
  "stream": true
}'
```

##### Response

A stream of JSON objects is returned:
```json
{
    "model": "llama3.2",
    "created_at": "2025-07-07T20:22:19.184789Z",
    "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
            {
                "function": {
                    "name": "get_weather",
                    "arguments": {
                        "city": "Tokyo"
                    }
                },
            }
        ]
    },
    "done": false
}
```

Final response:

```json
{
  "model":"llama3.2",
  "created_at":"2025-07-07T20:22:19.19314Z",
  "message": {
    "role": "assistant",
    "content": ""
  },
  "done_reason": "stop",
  "done": true,
  "total_duration": 182242375,
  "load_duration": 41295167,
  "prompt_eval_count": 169,
  "prompt_eval_duration": 24573166,
  "eval_count": 15,
  "eval_duration": 115959084
}
```

Matt Williams's avatar
Matt Williams committed
663
#### Chat request (No streaming)
664

Matt Williams's avatar
Matt Williams committed
665
666
667
668
##### Request

```shell
curl http://localhost:11434/api/chat -d '{
669
  "model": "llama3.2",
Matt Williams's avatar
Matt Williams committed
670
671
672
673
674
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
Jeffrey Morgan's avatar
Jeffrey Morgan committed
675
  ],
Matt Williams's avatar
Matt Williams committed
676
677
678
679
680
681
682
683
  "stream": false
}'
```

##### Response

```json
{
684
  "model": "llama3.2",
Matt Williams's avatar
Matt Williams committed
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
  "created_at": "2023-12-12T14:13:43.416799Z",
  "message": {
    "role": "assistant",
    "content": "Hello! How are you today?"
  },
  "done": true,
  "total_duration": 5191566416,
  "load_duration": 2154458,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 383809000,
  "eval_count": 298,
  "eval_duration": 4799921000
}
```

700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
#### Chat request (No streaming, with tools)

##### Request


```shell
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    {
      "role": "user",
      "content": "what is the weather in tokyo?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the weather in a given city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string",
              "description": "The city to get the weather for"
            }
          },
          "required": ["city"]
        }
      }
    }
  ],
  "stream": false 
}'
```

##### Response

```json
{
  "model": "llama3.2",
  "created_at": "2025-07-07T20:32:53.844124Z",
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "function": {
          "name": "get_weather",
          "arguments": {
            "city": "Tokyo"
          }
        },
      }
    ]
  },
  "done_reason": "stop",
  "done": true,
  "total_duration": 3244883583,
  "load_duration": 2969184542,
  "prompt_eval_count": 169,
  "prompt_eval_duration": 141656333,
  "eval_count": 18,
  "eval_duration": 133293625
}
```

768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
#### Chat request (Structured outputs)

##### Request

```shell
curl -X POST http://localhost:11434/api/chat -H "Content-Type: application/json" -d '{
  "model": "llama3.1",
  "messages": [{"role": "user", "content": "Ollama is 22 years old and busy saving the world. Return a JSON object with the age and availability."}],
  "stream": false,
  "format": {
    "type": "object",
    "properties": {
      "age": {
        "type": "integer"
      },
      "available": {
        "type": "boolean"
      }
    },
    "required": [
      "age",
      "available"
    ]
  },
  "options": {
    "temperature": 0
  }
}'
```

##### Response

```json
{
  "model": "llama3.1",
  "created_at": "2024-12-06T00:46:58.265747Z",
  "message": { "role": "assistant", "content": "{\"age\": 22, \"available\": false}" },
  "done_reason": "stop",
  "done": true,
  "total_duration": 2254970291,
  "load_duration": 574751416,
  "prompt_eval_count": 34,
  "prompt_eval_duration": 1502000000,
  "eval_count": 12,
  "eval_duration": 175000000
}
```

Matt Williams's avatar
Matt Williams committed
816
817
818
819
820
#### Chat request (With History)

Send a chat message with a conversation history. You can use this same approach to start the conversation using multi-shot or chain-of-thought prompting.

##### Request
Bruce MacDonald's avatar
Bruce MacDonald committed
821
822

```shell
823
curl http://localhost:11434/api/chat -d '{
824
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    },
    {
      "role": "assistant",
      "content": "due to rayleigh scattering."
    },
    {
      "role": "user",
      "content": "how is that different than mie scattering?"
    }
  ]
}'
```

Matt Williams's avatar
Matt Williams committed
842
##### Response
Bruce MacDonald's avatar
Bruce MacDonald committed
843
844
845
846
847

A stream of JSON objects is returned:

```json
{
848
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
849
850
  "created_at": "2023-08-04T08:52:19.385406455-07:00",
  "message": {
Robin Glauser's avatar
Robin Glauser committed
851
    "role": "assistant",
Bruce MacDonald's avatar
Bruce MacDonald committed
852
853
854
855
856
857
858
859
860
861
    "content": "The"
  },
  "done": false
}
```

Final response:

```json
{
862
  "model": "llama3.2",
Bruce MacDonald's avatar
Bruce MacDonald committed
863
864
  "created_at": "2023-08-04T19:22:45.499127Z",
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
865
866
867
868
869
870
  "total_duration": 8113331500,
  "load_duration": 6396458,
  "prompt_eval_count": 61,
  "prompt_eval_duration": 398801000,
  "eval_count": 468,
  "eval_duration": 7701267000
Bruce MacDonald's avatar
Bruce MacDonald committed
871
872
873
}
```

874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954

#### Chat request (With history, with tools)

##### Request

```shell
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    {
      "role": "user",
      "content": "what is the weather in Toronto?"
    },
    // the message from the model appended to history
    {
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "function": {
            "name": "get_temperature",
            "arguments": {
              "city": "Toronto"
            }
          },
        }
      ]
    },
    // the tool call result appended to history
    {
      "role": "tool",
      "content": "11 degrees celsius",
      "tool_name": "get_temperature",
    }
  ],
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the weather in a given city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string",
              "description": "The city to get the weather for"
            }
          },
          "required": ["city"]
        }
      }
    }
  ]
}'
```

##### Response

```json
{
  "model": "llama3.2",
  "created_at": "2025-07-07T20:43:37.688511Z",
  "message": {
    "role": "assistant",
    "content": "The current temperature in Toronto is 11°C."
  },
  "done_reason": "stop",
  "done": true,
  "total_duration": 890771750,
  "load_duration": 707634750,
  "prompt_eval_count": 94,
  "prompt_eval_duration": 91703208,
  "eval_count": 11,
  "eval_duration": 90282125
}

```


Matt Williams's avatar
Matt Williams committed
955
956
957
#### Chat request (with images)

##### Request
958

Veit Heller's avatar
Veit Heller committed
959
Send a chat message with images. The images should be provided as an array, with the individual images encoded in Base64.
960
961
962

```shell
curl http://localhost:11434/api/chat -d '{
Matt Williams's avatar
Matt Williams committed
963
  "model": "llava",
964
965
966
967
968
  "messages": [
    {
      "role": "user",
      "content": "what is in this image?",
      "images": ["iVBORw0KGgoAAAANSUhEUgAAAG0AAABmCAYAAADBPx+VAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAA3VSURBVHgB7Z27r0zdG8fX743i1bi1ikMoFMQloXRpKFFIqI7LH4BEQ+NWIkjQuSWCRIEoULk0gsK1kCBI0IhrQVT7tz/7zZo888yz1r7MnDl7z5xvsjkzs2fP3uu71nNfa7lkAsm7d++Sffv2JbNmzUqcc8m0adOSzZs3Z+/XES4ZckAWJEGWPiCxjsQNLWmQsWjRIpMseaxcuTKpG/7HP27I8P79e7dq1ars/yL4/v27S0ejqwv+cUOGEGGpKHR37tzJCEpHV9tnT58+dXXCJDdECBE2Ojrqjh071hpNECjx4cMHVycM1Uhbv359B2F79+51586daxN/+pyRkRFXKyRDAqxEp4yMlDDzXG1NPnnyJKkThoK0VFd1ELZu3TrzXKxKfW7dMBQ6bcuWLW2v0VlHjx41z717927ba22U9APcw7Nnz1oGEPeL3m3p2mTAYYnFmMOMXybPPXv2bNIPpFZr1NHn4HMw0KRBjg9NuRw95s8PEcz/6DZELQd/09C9QGq5RsmSRybqkwHGjh07OsJSsYYm3ijPpyHzoiacg35MLdDSIS/O1yM778jOTwYUkKNHWUzUWaOsylE00MyI0fcnOwIdjvtNdW/HZwNLGg+sR1kMepSNJXmIwxBZiG8tDTpEZzKg0GItNsosY8USkxDhD0Rinuiko2gfL/RbiD2LZAjU9zKQJj8RDR0vJBR1/Phx9+PHj9Z7REF4nTZkxzX4LCXHrV271qXkBAPGfP/atWvu/PnzHe4C97F48eIsRLZ9+3a3f/9+87dwP1JxaF7/3r17ba+5l4EcaVo0lj3SBq5kGTJSQmLWMjgYNei2GPT1MuMqGTDEFHzeQSP2wi/jGnkmPJ/nhccs44jvDAxpVcxnq0F6eT8h4ni/iIWpR5lPyA6ETkNXoSukvpJAD3AsXLiwpZs49+fPn5ke4j10TqYvegSfn0OnafC+Tv9ooA/JPkgQysqQNBzagXY55nO/oa1F7qvIPWkRL12WRpMWUvpVDYmxAPehxWSe8ZEXL20sadYIozfmNch4QJPAfeJgW3rNsnzphBKNJM2KKODo1rVOMRYik5ETy3ix4qWNI81qAAirizgMIc+yhTytx0JWZuNI03qsrgWlGtwjoS9XwgUhWGyhUaRZZQNNIEwCiXD16tXcAHUs79co0vSD8rrJCIW98pzvxpAWyyo3HYwqS0+H0BjStClcZJT5coMm6D2LOF8TolGJtK9fvyZpyiC5ePFi9nc/oJU4eiEP0jVoAnHa9wyJycITMP78+eMeP37sXrx44d6+fdt6f82aNdkx1pg9e3Zb5W+RSRE+n+VjksQWifvVaTKFhn5O8my63K8Qabdv33b379/PiAP//vuvW7BggZszZ072/+TJk91YgkafPn166zXB1rQHFvouAWHq9z3SEevSUerqCn2/dDCeta2jxYbr69evk4MHDyY7d+7MjhMnTiTPnz9Pfv/+nfQT2ggpO2dMF8cghuoM7Ygj5iWCqRlGFml0QC/ftGmTmzt3rmsaKDsgBSPh0/8yPeLLBihLkOKJc0jp8H8vUzcxIA1k6QJ/c78tWEyj5P3o4u9+jywNPdJi5rAH9x0KHcl4Hg570eQp3+vHXGyrmEeigzQsQsjavXt38ujRo44LQuDDhw+TW7duRS1HGgMxhNXHgflaNTOsHyKvHK5Ijo2jbFjJBQK9YwFd6RVMzfgRBmEfP37suBBm/p49e1qjEP2mwTViNRo0VJWH1deMXcNK08uUjVUu7s/zRaL+oLNxz1bpANco4npUgX4G2eFbpDFyQoQxojBCpEGSytmOH8qrH5Q9vuzD6ofQylkCUmh8DBAr+q8JCyVNtWQIidKQE9wNtLSQnS4jDSsxNHogzFuQBw4cyM61UKVsjfr3ooBkPSqqQHesUPWVtzi9/vQi1T+rJj7WiTz4Pt/l3LxUkr5P2VYZaZ4URpsE+st/dujQoaBBYokbrz/8TJNQYLSonrPS9kUaSkPeZyj1AWSj+d+VBoy1pIWVNed8P0Ll/ee5HdGRhrHhR5GGN0r4LGZBaj8oFDJitBTJzIZgFcmU0Y8ytWMZMzJOaXUSrUs5RxKnrxmbb5YXO9VGUhtpXldhEUogFr3IzIsvlpmdosVcGVGXFWp2oU9kLFL3dEkSz6NHEY1sjSRdIuDFWEhd8KxFqsRi1uM/nz9/zpxnwlESONdg6dKlbsaMGS4EHFHtjFIDHwKOo46l4TxSuxgDzi+rE2jg+BaFruOX4HXa0Nnf1lwAPufZeF8/r6zD97WK2qFnGjBxTw5qNGPxT+5T/r7/7RawFC3j4vTp09koCxkeHjqbHJqArmH5UrFKKksnxrK7FuRIs8STfBZv+luugXZ2pR/pP9Ois4z+TiMzUUkUjD0iEi1fzX8GmXyuxUBRcaUfykV0YZnlJGKQpOiGB76x5GeWkWWJc3mOrK6S7xdND+W5N6XyaRgtWJFe13GkaZnKOsYqGdOVVVbGupsyA/l7emTLHi7vwTdirNEt0qxnzAvBFcnQF16xh/TMpUuXHDowhlA9vQVraQhkudRdzOnK+04ZSP3DUhVSP61YsaLtd/ks7ZgtPcXqPqEafHkdqa84X6aCeL7YWlv6edGFHb+ZFICPlljHhg0bKuk0CSvVznWsotRu433alNdFrqG45ejoaPCaUkWERpLXjzFL2Rpllp7PJU2a/v7Ab8N05/9t27Z16KUqoFGsxnI9EosS2niSYg9SpU6B4JgTrvVW1flt1sT+0ADIJU2maXzcUTraGCRaL1Wp9rUMk16PMom8QhruxzvZIegJjFU7LLCePfS8uaQdPny4jTTL0dbee5mYokQsXTIWNY46kuMbnt8Kmec+LGWtOVIl9cT1rCB0V8WqkjAsRwta93TbwNYoGKsUSChN44lgBNCoHLHzquYKrU6qZ8lolCIN0Rh6cP0Q3U6I6IXILYOQI513hJaSKAorFpuHXJNfVlpRtmYBk1Su1obZr5dnKAO+L10Hrj3WZW+E3qh6IszE37F6EB+68mGpvKm4eb9bFrlzrok7fvr0Kfv727dvWRmdVTJHw0qiiCUSZ6wCK+7XL/AcsgNyL74DQQ730sv78Su7+t/A36MdY0sW5o40ahslXr58aZ5HtZB8GH64m9EmMZ7FpYw4T6QnrZfgenrhFxaSiSGXtPnz57e9TkNZLvTjeqhr734CNtrK41L40sUQckmj1lGKQ0rC37x544r8eNXRpnVE3ZZY7zXo8NomiO0ZUCj2uHz58rbXoZ6gc0uA+F6ZeKS/jhRDUq8MKrTho9fEkihMmhxtBI1DxKFY9XLpVcSkfoi8JGnToZO5sU5aiDQIW716ddt7ZLYtMQlhECdBGXZZMWldY5BHm5xgAroWj4C0hbYkSc/jBmggIrXJWlZM6pSETsEPGqZOndr2uuuR5rF169a2HoHPdurUKZM4CO1WTPqaDaAd+GFGKdIQkxAn9RuEWcTRyN2KSUgiSgF5aWzPTeA/lN5rZubMmR2bE4SIC4nJoltgAV/dVefZm72AtctUCJU2CMJ327hxY9t7EHbkyJFseq+EJSY16RPo3Dkq1kkr7+q0bNmyDuLQcZBEPYmHVdOBiJyIlrRDq41YPWfXOxUysi5fvtyaj+2BpcnsUV/oSoEMOk2CQGlr4ckhBwaetBhjCwH0ZHtJROPJkyc7UjcYLDjmrH7ADTEBXFfOYmB0k9oYBOjJ8b4aOYSe7QkKcYhFlq3QYLQhSidNmtS2RATwy8YOM3EQJsUjKiaWZ+vZToUQgzhkHXudb/PW5YMHD9yZM2faPsMwoc7RciYJXbGuBqJ1UIGKKLv915jsvgtJxCZDubdXr165mzdvtr1Hz5LONA8jrUwKPqsmVesKa49S3Q4WxmRPUEYdTjgiUcfUwLx589ySJUva3oMkP6IYddq6HMS4o55xBJBUeRjzfa4Zdeg56QZ43LhxoyPo7Lf1kNt7oO8wWAbNwaYjIv5lhyS7kRf96dvm5Jah8vfvX3flyhX35cuX6HfzFHOToS1H4BenCaHvO8pr8iDuwoUL7tevX+b5ZdbBair0xkFIlFDlW4ZknEClsp/TzXyAKVOmmHWFVSbDNw1l1+4f90U6IY/q4V27dpnE9bJ+v87QEydjqx/UamVVPRG+mwkNTYN+9tjkwzEx+atCm/X9WvWtDtAb68Wy9LXa1UmvCDDIpPkyOQ5ZwSzJ4jMrvFcr0rSjOUh+GcT4LSg5ugkW1Io0/SCDQBojh0hPlaJdah+tkVYrnTZowP8iq1F1TgMBBauufyB33x1v+NWFYmT5KmppgHC+NkAgbmRkpD3yn9QIseXymoTQFGQmIOKTxiZIWpvAatenVqRVXf2nTrAWMsPnKrMZHz6bJq5jvce6QK8J1cQNgKxlJapMPdZSR64/UivS9NztpkVEdKcrs5alhhWP9NeqlfWopzhZScI6QxseegZRGeg5a8C3Re1Mfl1ScP36ddcUaMuv24iOJtz7sbUjTS4qBvKmstYJoUauiuD3k5qhyr7QdUHMeCgLa1Ear9NquemdXgmum4fvJ6w1lqsuDhNrg1qSpleJK7K3TF0Q2jSd94uSZ60kK1e3qyVpQK6PVWXp2/FC3mp6jBhKKOiY2h3gtUV64TWM6wDETRPLDfSakXmH3w8g9Jlug8ZtTt4kVF0kLUYYmCCtD/DrQ5YhMGbA9L3ucdjh0y8kOHW5gU/VEEmJTcL4Pz/f7mgoAbYkAAAAAElFTkSuQmCC"]
Jeffrey Morgan's avatar
Jeffrey Morgan committed
969
    }
970
971
972
973
  ]
}'
```

Matt Williams's avatar
Matt Williams committed
974
975
976
977
978
979
980
981
982
983
984
985
##### Response

```json
{
  "model": "llava",
  "created_at": "2023-12-13T22:42:50.203334Z",
  "message": {
    "role": "assistant",
    "content": " The image features a cute, little pig with an angry facial expression. It's wearing a heart on its shirt and is waving in the air. This scene appears to be part of a drawing or sketching project.",
    "images": null
  },
  "done": true,
Jeffrey Morgan's avatar
Jeffrey Morgan committed
986
987
988
989
990
991
  "total_duration": 1668506709,
  "load_duration": 1986209,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 359682000,
  "eval_count": 83,
  "eval_duration": 1303285000
Matt Williams's avatar
Matt Williams committed
992
993
994
}
```

995
996
997
998
999
1000
#### Chat request (Reproducible outputs)

##### Request

```shell
curl http://localhost:11434/api/chat -d '{
1001
  "model": "llama3.2",
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
  "messages": [
    {
      "role": "user",
      "content": "Hello!"
    }
  ],
  "options": {
    "seed": 101,
    "temperature": 0
  }
}'
```

##### Response

```json
{
1019
  "model": "llama3.2",
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
  "created_at": "2023-12-12T14:13:43.416799Z",
  "message": {
    "role": "assistant",
    "content": "Hello! How are you today?"
  },
  "done": true,
  "total_duration": 5191566416,
  "load_duration": 2154458,
  "prompt_eval_count": 26,
  "prompt_eval_duration": 383809000,
  "eval_count": 298,
  "eval_duration": 4799921000
}
```

Michael Yang's avatar
Michael Yang committed
1035
1036
1037
1038
#### Chat request (with tools)

##### Request

1039
```shell
Michael Yang's avatar
Michael Yang committed
1040
curl http://localhost:11434/api/chat -d '{
1041
  "model": "llama3.2",
Michael Yang's avatar
Michael Yang committed
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
  "messages": [
    {
      "role": "user",
      "content": "What is the weather today in Paris?"
    }
  ],
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The location to get the weather for, e.g. San Francisco, CA"
            },
            "format": {
              "type": "string",
              "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location", "format"]
        }
      }
    }
  ]
}'
```

##### Response

```json
{
1080
  "model": "llama3.2",
Michael Yang's avatar
Michael Yang committed
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
  "created_at": "2024-07-22T20:33:28.123648Z",
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "function": {
          "name": "get_current_weather",
          "arguments": {
            "format": "celsius",
            "location": "Paris, FR"
          }
        }
      }
    ]
  },
  "done_reason": "stop",
  "done": true,
  "total_duration": 885095291,
  "load_duration": 3753500,
  "prompt_eval_count": 122,
  "prompt_eval_duration": 328493000,
  "eval_count": 33,
  "eval_duration": 552222000
}
```

1108
1109
1110
1111
1112
1113
#### Load a model

If the messages array is empty, the model will be loaded into memory.

##### Request

1114
```shell
1115
curl http://localhost:11434/api/chat -d '{
1116
  "model": "llama3.2",
1117
1118
1119
1120
1121
  "messages": []
}'
```

##### Response
1122

1123
1124
```json
{
1125
  "model": "llama3.2",
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
  "created_at":"2024-09-12T21:17:29.110811Z",
  "message": {
    "role": "assistant",
    "content": ""
  },
  "done_reason": "load",
  "done": true
}
```

#### Unload a model

If the messages array is empty and the `keep_alive` parameter is set to `0`, a model will be unloaded from memory.

##### Request

1142
```shell
1143
curl http://localhost:11434/api/chat -d '{
1144
  "model": "llama3.2",
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
  "messages": [],
  "keep_alive": 0
}'
```

##### Response

A single JSON object is returned:

```json
{
1156
  "model": "llama3.2",
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
  "created_at":"2024-09-12T21:33:17.547535Z",
  "message": {
    "role": "assistant",
    "content": ""
  },
  "done_reason": "unload",
  "done": true
}
```

1167
## Create a Model
Matt Williams's avatar
Matt Williams committed
1168

1169
```
1170
1171
1172
POST /api/create
```

1173
1174
1175
1176
1177
1178
Create a model from:
 * another model;
 * a safetensors directory; or
 * a GGUF file.

If you are creating a model from a safetensors directory or from a GGUF file, you must [create a blob](#create-a-blob) for each of the files and then use the file name and SHA256 digest associated with each blob in the `files` field.
1179

1180
### Parameters
Matt Williams's avatar
Matt Williams committed
1181

Patrick Devine's avatar
Patrick Devine committed
1182
- `model`: name of the model to create
1183
1184
1185
1186
1187
1188
1189
1190
- `from`: (optional) name of an existing model to create the new model from
- `files`: (optional) a dictionary of file names to SHA256 digests of blobs to create the model from
- `adapters`: (optional) a dictionary of file names to SHA256 digests of blobs for LORA adapters
- `template`: (optional) the prompt template for the model
- `license`: (optional) a string or list of strings containing the license or licenses for the model
- `system`: (optional) a string containing the system prompt for the model
- `parameters`: (optional) a dictionary of parameters for the model (see [Modelfile](./modelfile.md#valid-parameters-and-values) for a list of parameters)
- `messages`: (optional) a list of message objects used to create a conversation
1191
- `stream`: (optional) if `false` the response will be returned as a single response object, rather than a stream of objects
Patrick Devine's avatar
Patrick Devine committed
1192
1193
1194
1195
1196
1197
1198
1199
1200
- `quantize` (optional): quantize a non-quantized (e.g. float16) model

#### Quantization types

| Type | Recommended |
| --- | :-: |
| q4_K_M | * |
| q4_K_S | |
| q8_0 | * |
Matt Williams's avatar
Matt Williams committed
1201

1202
1203
### Examples

Matt Williams's avatar
Matt Williams committed
1204
1205
#### Create a new model

1206
Create a new model from an existing model.
Matt Williams's avatar
Matt Williams committed
1207
1208

##### Request
Matt Williams's avatar
Matt Williams committed
1209

Matt Williams's avatar
Matt Williams committed
1210
```shell
1211
curl http://localhost:11434/api/create -d '{
Patrick Devine's avatar
Patrick Devine committed
1212
  "model": "mario",
1213
1214
  "from": "llama3.2",
  "system": "You are Mario from Super Mario Bros."
1215
}'
Matt Williams's avatar
Matt Williams committed
1216
1217
```

Matt Williams's avatar
Matt Williams committed
1218
##### Response
Matt Williams's avatar
Matt Williams committed
1219

Patrick Devine's avatar
Patrick Devine committed
1220
A stream of JSON objects is returned:
Matt Williams's avatar
Matt Williams committed
1221

1222
```json
Matt Williams's avatar
Matt Williams committed
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
{"status":"reading model metadata"}
{"status":"creating system layer"}
{"status":"using already created layer sha256:22f7f8ef5f4c791c1b03d7eb414399294764d7cc82c7e94aa81a1feb80a983a2"}
{"status":"using already created layer sha256:8c17c2ebb0ea011be9981cc3922db8ca8fa61e828c5d3f44cb6ae342bf80460b"}
{"status":"using already created layer sha256:7c23fb36d80141c4ab8cdbb61ee4790102ebd2bf7aeff414453177d4f2110e5d"}
{"status":"using already created layer sha256:2e0493f67d0c8c9c68a8aeacdf6a38a2151cb3c4c1d42accf296e19810527988"}
{"status":"using already created layer sha256:2759286baa875dc22de5394b4a925701b1896a7e3f8e53275c36f75a877a82c9"}
{"status":"writing layer sha256:df30045fe90f0d750db82a058109cecd6d4de9c90a3d75b19c09e5f64580bb42"}
{"status":"writing layer sha256:f18a68eb09bf925bb1b669490407c1b1251c5db98dc4d3d81f3088498ea55690"}
{"status":"writing manifest"}
{"status":"success"}
Matt Williams's avatar
Matt Williams committed
1234
1235
```

Patrick Devine's avatar
Patrick Devine committed
1236
1237
1238
1239
1240
1241
1242
1243
#### Quantize a model

Quantize a non-quantized model.

##### Request

```shell
curl http://localhost:11434/api/create -d '{
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1244
1245
  "model": "llama3.2:quantized",
  "from": "llama3.2:3b-instruct-fp16",
Patrick Devine's avatar
Patrick Devine committed
1246
1247
1248
1249
1250
1251
1252
1253
  "quantize": "q4_K_M"
}'
```

##### Response

A stream of JSON objects is returned:

1254
```json
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1255
1256
1257
1258
1259
1260
1261
{"status":"quantizing F16 model to Q4_K_M","digest":"0","total":6433687776,"completed":12302}
{"status":"quantizing F16 model to Q4_K_M","digest":"0","total":6433687776,"completed":6433687552}
{"status":"verifying conversion"}
{"status":"creating new layer sha256:fb7f4f211b89c6c4928ff4ddb73db9f9c0cfca3e000c3e40d6cf27ddc6ca72eb"}
{"status":"using existing layer sha256:966de95ca8a62200913e3f8bfbf84c8494536f1b94b49166851e76644e966396"}
{"status":"using existing layer sha256:fcc5a6bec9daf9b561a68827b67ab6088e1dba9d1fa2a50d7bbcc8384e0a265d"}
{"status":"using existing layer sha256:a70ff7e570d97baaf4e62ac6e6ad9975e04caa6d900d3742d37698494479e0cd"}
Patrick Devine's avatar
Patrick Devine committed
1262
1263
1264
1265
1266
{"status":"using existing layer sha256:56bb8bd477a519ffa694fc449c2413c6f0e1d3b1c88fa7e3c9d88d3ae49d4dcb"}
{"status":"writing manifest"}
{"status":"success"}
```

1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
#### Create a model from GGUF

Create a model from a GGUF file. The `files` parameter should be filled out with the file name and SHA256 digest of the GGUF file you wish to use. Use [/api/blobs/:digest](#push-a-blob) to push the GGUF file to the server before calling this API.


##### Request

```shell
curl http://localhost:11434/api/create -d '{
  "model": "my-gguf-model",
  "files": {
    "test.gguf": "sha256:432f310a77f4650a88d0fd59ecdd7cebed8d684bafea53cbff0473542964f0c3"
  }
}'
```

##### Response

A stream of JSON objects is returned:

1287
```json
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
{"status":"parsing GGUF"}
{"status":"using existing layer sha256:432f310a77f4650a88d0fd59ecdd7cebed8d684bafea53cbff0473542964f0c3"}
{"status":"writing manifest"}
{"status":"success"}
```


#### Create a model from a Safetensors directory

The `files` parameter should include a dictionary of files for the safetensors model which includes the file names and SHA256 digest of each file. Use [/api/blobs/:digest](#push-a-blob) to first push each of the files to the server before calling this API. Files will remain in the cache until the Ollama server is restarted.
Patrick Devine's avatar
Patrick Devine committed
1298

1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
##### Request

```shell
curl http://localhost:11434/api/create -d '{
  "model": "fred",
  "files": {
    "config.json": "sha256:dd3443e529fb2290423a0c65c2d633e67b419d273f170259e27297219828e389",
    "generation_config.json": "sha256:88effbb63300dbbc7390143fbbdd9d9fa50587b37e8bfd16c8c90d4970a74a36",
    "special_tokens_map.json": "sha256:b7455f0e8f00539108837bfa586c4fbf424e31f8717819a6798be74bef813d05",
    "tokenizer.json": "sha256:bbc1904d35169c542dffbe1f7589a5994ec7426d9e5b609d07bab876f32e97ab",
    "tokenizer_config.json": "sha256:24e8a6dc2547164b7002e3125f10b415105644fcf02bf9ad8b674c87b1eaaed6",
    "model.safetensors": "sha256:1ff795ff6a07e6a68085d206fb84417da2f083f68391c2843cd2b8ac6df8538f"
  }
}'
```

##### Response

A stream of JSON objects is returned:

```shell
{"status":"converting model"}
{"status":"creating new layer sha256:05ca5b813af4a53d2c2922933936e398958855c44ee534858fcfd830940618b6"}
{"status":"using autodetected template llama3-instruct"}
{"status":"using existing layer sha256:56bb8bd477a519ffa694fc449c2413c6f0e1d3b1c88fa7e3c9d88d3ae49d4dcb"}
{"status":"writing manifest"}
{"status":"success"}
```

## Check if a Blob Exists
Michael Yang's avatar
Michael Yang committed
1329
1330
1331
1332
1333

```shell
HEAD /api/blobs/:digest
```

1334
Ensures that the file blob (Binary Large Object) used with create a model exists on the server. This checks your Ollama server and not ollama.com.
Matt Williams's avatar
Matt Williams committed
1335

1336
### Query Parameters
Michael Yang's avatar
Michael Yang committed
1337
1338
1339

- `digest`: the SHA256 digest of the blob

1340
### Examples
Michael Yang's avatar
Michael Yang committed
1341

1342
#### Request
Michael Yang's avatar
Michael Yang committed
1343
1344
1345
1346
1347

```shell
curl -I http://localhost:11434/api/blobs/sha256:29fdb92e57cf0827ded04ae6461b5931d01fa595843f55d36f5b275a52087dd2
```

1348
#### Response
Michael Yang's avatar
Michael Yang committed
1349
1350
1351

Return 200 OK if the blob exists, 404 Not Found if it does not.

1352
## Push a Blob
Michael Yang's avatar
Michael Yang committed
1353

1354
```
Michael Yang's avatar
Michael Yang committed
1355
1356
1357
POST /api/blobs/:digest
```

1358
Push a file to the Ollama server to create a "blob" (Binary Large Object).
Michael Yang's avatar
Michael Yang committed
1359

1360
### Query Parameters
Michael Yang's avatar
Michael Yang committed
1361
1362
1363

- `digest`: the expected SHA256 digest of the file

1364
### Examples
Michael Yang's avatar
Michael Yang committed
1365

1366
#### Request
Michael Yang's avatar
Michael Yang committed
1367

Michael Yang's avatar
Michael Yang committed
1368
```shell
1369
curl -T model.gguf -X POST http://localhost:11434/api/blobs/sha256:29fdb92e57cf0827ded04ae6461b5931d01fa595843f55d36f5b275a52087dd2
Michael Yang's avatar
Michael Yang committed
1370
1371
```

1372
#### Response
Michael Yang's avatar
Michael Yang committed
1373

Matt Williams's avatar
Matt Williams committed
1374
Return 201 Created if the blob was successfully created, 400 Bad Request if the digest used is not expected.
Michael Yang's avatar
Michael Yang committed
1375

1376
## List Local Models
Matt Williams's avatar
Matt Williams committed
1377

1378
```
1379
GET /api/tags
Matt Williams's avatar
Matt Williams committed
1380
1381
```

1382
List models that are available locally.
Matt Williams's avatar
Matt Williams committed
1383

1384
1385
1386
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1387

Matt Williams's avatar
Matt Williams committed
1388
```shell
1389
1390
curl http://localhost:11434/api/tags
```
Matt Williams's avatar
Matt Williams committed
1391

1392
#### Response
Matt Williams's avatar
Matt Williams committed
1393

1394
1395
A single JSON object will be returned.

1396
```json
Matt Williams's avatar
Matt Williams committed
1397
1398
1399
{
  "models": [
    {
1400
1401
1402
1403
1404
      "name": "deepseek-r1:latest",
      "model": "deepseek-r1:latest",
      "modified_at": "2025-05-10T08:06:48.639712648-07:00",
      "size": 4683075271,
      "digest": "0a8c266910232fd3291e71e5ba1e058cc5af9d411192cf88b6d30e92b6e73163",
Matt Williams's avatar
Matt Williams committed
1405
      "details": {
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1406
        "parent_model": "",
Matt Williams's avatar
Matt Williams committed
1407
        "format": "gguf",
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1408
1409
1410
1411
1412
1413
        "family": "qwen2",
        "families": [
          "qwen2"
        ],
        "parameter_size": "7.6B",
        "quantization_level": "Q4_K_M"
Matt Williams's avatar
Matt Williams committed
1414
      }
1415
1416
    },
    {
1417
1418
1419
1420
1421
      "name": "llama3.2:latest",
      "model": "llama3.2:latest",
      "modified_at": "2025-05-04T17:37:44.706015396-07:00",
      "size": 2019393189,
      "digest": "a80c4f17acd55265feec403c7aef86be0c25983ab279d83f3bcd3abbcb5b8b72",
Matt Williams's avatar
Matt Williams committed
1422
      "details": {
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1423
        "parent_model": "",
Matt Williams's avatar
Matt Williams committed
1424
1425
        "format": "gguf",
        "family": "llama",
Daniel Hiltgen's avatar
Daniel Hiltgen committed
1426
1427
1428
1429
1430
        "families": [
          "llama"
        ],
        "parameter_size": "3.2B",
        "quantization_level": "Q4_K_M"
Matt Williams's avatar
Matt Williams committed
1431
      }
Matt Williams's avatar
Matt Williams committed
1432
1433
    }
  ]
Matt Williams's avatar
Matt Williams committed
1434
1435
1436
}
```

Matt Williams's avatar
Matt Williams committed
1437
1438
## Show Model Information

1439
```
Matt Williams's avatar
Matt Williams committed
1440
1441
1442
POST /api/show
```

1443
Show information about a model including details, modelfile, template, parameters, license, system prompt.
Matt Williams's avatar
Matt Williams committed
1444
1445
1446

### Parameters

Patrick Devine's avatar
Patrick Devine committed
1447
- `model`: name of the model to show
1448
- `verbose`: (optional) if set to `true`, returns full data for verbose response fields
Matt Williams's avatar
Matt Williams committed
1449

1450
1451
1452
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1453

1454
```shell
Matt Williams's avatar
Matt Williams committed
1455
curl http://localhost:11434/api/show -d '{
1456
  "model": "llava"
Matt Williams's avatar
Matt Williams committed
1457
}'
Matt Williams's avatar
Matt Williams committed
1458
```
Matt Williams's avatar
Matt Williams committed
1459

1460
#### Response
Matt Williams's avatar
Matt Williams committed
1461

1462
```json5
Matt Williams's avatar
Matt Williams committed
1463
{
睡觉型学渣's avatar
睡觉型学渣 committed
1464
  "modelfile": "# Modelfile generated by \"ollama show\"\n# To build a new Modelfile based on this one, replace the FROM line with:\n# FROM llava:latest\n\nFROM /Users/matt/.ollama/models/blobs/sha256:200765e1283640ffbd013184bf496e261032fa75b99498a9613be4e94d63ad52\nTEMPLATE \"\"\"{{ .System }}\nUSER: {{ .Prompt }}\nASSISTANT: \"\"\"\nPARAMETER num_ctx 4096\nPARAMETER stop \"\u003c/s\u003e\"\nPARAMETER stop \"USER:\"\nPARAMETER stop \"ASSISTANT:\"",
1465
1466
  "parameters": "num_keep                       24\nstop                           \"<|start_header_id|>\"\nstop                           \"<|end_header_id|>\"\nstop                           \"<|eot_id|>\"",
  "template": "{{ if .System }}<|start_header_id|>system<|end_header_id|>\n\n{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>\n\n{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>\n\n{{ .Response }}<|eot_id|>",
1467
  "details": {
1468
    "parent_model": "",
1469
    "format": "gguf",
Matt Williams's avatar
Matt Williams committed
1470
    "family": "llama",
1471
1472
1473
1474
    "families": [
      "llama"
    ],
    "parameter_size": "8.0B",
1475
    "quantization_level": "Q4_0"
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
  },
  "model_info": {
    "general.architecture": "llama",
    "general.file_type": 2,
    "general.parameter_count": 8030261248,
    "general.quantization_version": 2,
    "llama.attention.head_count": 32,
    "llama.attention.head_count_kv": 8,
    "llama.attention.layer_norm_rms_epsilon": 0.00001,
    "llama.block_count": 32,
    "llama.context_length": 8192,
    "llama.embedding_length": 4096,
    "llama.feed_forward_length": 14336,
    "llama.rope.dimension_count": 128,
    "llama.rope.freq_base": 500000,
    "llama.vocab_size": 128256,
    "tokenizer.ggml.bos_token_id": 128000,
    "tokenizer.ggml.eos_token_id": 128009,
    "tokenizer.ggml.merges": [],            // populates if `verbose=true`
    "tokenizer.ggml.model": "gpt2",
    "tokenizer.ggml.pre": "llama-bpe",
    "tokenizer.ggml.token_type": [],        // populates if `verbose=true`
    "tokenizer.ggml.tokens": []             // populates if `verbose=true`
1499
1500
1501
1502
1503
  },
  "capabilities": [
    "completion",
    "vision"
  ],
Matt Williams's avatar
Matt Williams committed
1504
1505
1506
1507
1508
}
```

## Copy a Model

1509
```
1510
POST /api/copy
Matt Williams's avatar
Matt Williams committed
1511
```
1512

1513
Copy a model. Creates a model with another name from an existing model.
Matt Williams's avatar
Matt Williams committed
1514

1515
1516
1517
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1518

Matt Williams's avatar
Matt Williams committed
1519
```shell
1520
curl http://localhost:11434/api/copy -d '{
1521
  "source": "llama3.2",
1522
  "destination": "llama3-backup"
Matt Williams's avatar
Matt Williams committed
1523
1524
1525
}'
```

1526
#### Response
1527

Matt Williams's avatar
Matt Williams committed
1528
Returns a 200 OK if successful, or a 404 Not Found if the source model doesn't exist.
1529

Matt Williams's avatar
Matt Williams committed
1530
## Delete a Model
Matt Williams's avatar
Matt Williams committed
1531

1532
```
1533
DELETE /api/delete
Matt Williams's avatar
Matt Williams committed
1534
1535
```

1536
Delete a model and its data.
Matt Williams's avatar
Matt Williams committed
1537

1538
### Parameters
Matt Williams's avatar
Matt Williams committed
1539

Patrick Devine's avatar
Patrick Devine committed
1540
- `model`: model name to delete
Matt Williams's avatar
Matt Williams committed
1541

1542
1543
1544
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1545

Matt Williams's avatar
Matt Williams committed
1546
```shell
1547
curl -X DELETE http://localhost:11434/api/delete -d '{
Patrick Devine's avatar
Patrick Devine committed
1548
  "model": "llama3:13b"
Matt Williams's avatar
Matt Williams committed
1549
1550
1551
}'
```

1552
#### Response
1553

Matt Williams's avatar
Matt Williams committed
1554
Returns a 200 OK if successful, 404 Not Found if the model to be deleted doesn't exist.
1555

1556
## Pull a Model
Matt Williams's avatar
Matt Williams committed
1557

1558
```
1559
1560
1561
POST /api/pull
```

Matt Williams's avatar
Matt Williams committed
1562
Download a model from the ollama library. Cancelled pulls are resumed from where they left off, and multiple calls will share the same download progress.
Matt Williams's avatar
Matt Williams committed
1563

1564
### Parameters
Matt Williams's avatar
Matt Williams committed
1565

Patrick Devine's avatar
Patrick Devine committed
1566
- `model`: name of the model to pull
Matt Williams's avatar
Matt Williams committed
1567
- `insecure`: (optional) allow insecure connections to the library. Only use this if you are pulling from your own library during development.
1568
- `stream`: (optional) if `false` the response will be returned as a single response object, rather than a stream of objects
Matt Williams's avatar
Matt Williams committed
1569

1570
1571
1572
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1573

Matt Williams's avatar
Matt Williams committed
1574
```shell
1575
curl http://localhost:11434/api/pull -d '{
Patrick Devine's avatar
Patrick Devine committed
1576
  "model": "llama3.2"
1577
}'
Matt Williams's avatar
Matt Williams committed
1578
1579
```

1580
#### Response
1581

1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
If `stream` is not specified, or set to `true`, a stream of JSON objects is returned:

The first object is the manifest:

```json
{
  "status": "pulling manifest"
}
```

Then there is a series of downloading responses. Until any of the download is completed, the `completed` key may not be included. The number of files to be downloaded depends on the number of layers specified in the manifest.

1594
```json
Matt Williams's avatar
Matt Williams committed
1595
{
1596
1597
  "status": "downloading digestname",
  "digest": "digestname",
1598
  "total": 2142590208,
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
  "completed": 241970
}
```

After all the files are downloaded, the final responses are:

```json
{
    "status": "verifying sha256 digest"
}
{
    "status": "writing manifest"
}
{
    "status": "removing any unused layers"
}
{
    "status": "success"
}
```

if `stream` is set to false, then the response is a single JSON object:

```json
{
  "status": "success"
Matt Williams's avatar
Matt Williams committed
1625
1626
}
```
1627

Matt Williams's avatar
Matt Williams committed
1628
1629
## Push a Model

1630
```
Matt Williams's avatar
Matt Williams committed
1631
1632
1633
1634
1635
1636
1637
POST /api/push
```

Upload a model to a model library. Requires registering for ollama.ai and adding a public key first.

### Parameters

Patrick Devine's avatar
Patrick Devine committed
1638
- `model`: name of the model to push in the form of `<namespace>/<model>:<tag>`
1639
- `insecure`: (optional) allow insecure connections to the library. Only use this if you are pushing to your library during development.
1640
- `stream`: (optional) if `false` the response will be returned as a single response object, rather than a stream of objects
Matt Williams's avatar
Matt Williams committed
1641

1642
1643
1644
### Examples

#### Request
Matt Williams's avatar
Matt Williams committed
1645
1646

```shell
1647
curl http://localhost:11434/api/push -d '{
Patrick Devine's avatar
Patrick Devine committed
1648
  "model": "mattw/pygmalion:latest"
Matt Williams's avatar
Matt Williams committed
1649
1650
1651
}'
```

1652
#### Response
1653

1654
If `stream` is not specified, or set to `true`, a stream of JSON objects is returned:
Matt Williams's avatar
Matt Williams committed
1655
1656

```json
1657
{ "status": "retrieving manifest" }
1658
```
Matt Williams's avatar
Matt Williams committed
1659
1660
1661
1662
1663

and then:

```json
{
1664
1665
1666
  "status": "starting upload",
  "digest": "sha256:bc07c81de745696fdf5afca05e065818a8149fb0c77266fb584d9b2cba3711ab",
  "total": 1928429856
Matt Williams's avatar
Matt Williams committed
1667
1668
1669
1670
1671
1672
1673
}
```

Then there is a series of uploading responses:

```json
{
1674
1675
1676
1677
  "status": "starting upload",
  "digest": "sha256:bc07c81de745696fdf5afca05e065818a8149fb0c77266fb584d9b2cba3711ab",
  "total": 1928429856
}
Matt Williams's avatar
Matt Williams committed
1678
1679
1680
1681
1682
1683
1684
1685
1686
```

Finally, when the upload is complete:

```json
{"status":"pushing manifest"}
{"status":"success"}
```

1687
1688
1689
If `stream` is set to `false`, then the response is a single JSON object:

```json
1690
{ "status": "success" }
1691
1692
```

Matt Williams's avatar
Matt Williams committed
1693
1694
## Generate Embeddings

1695
```
royjhan's avatar
royjhan committed
1696
POST /api/embed
1697
1698
1699
1700
1701
1702
1703
```

Generate embeddings from a model

### Parameters

- `model`: name of model to generate embeddings from
royjhan's avatar
royjhan committed
1704
- `input`: text or list of text to generate embeddings for
1705

1706
1707
Advanced parameters:

royjhan's avatar
royjhan committed
1708
- `truncate`: truncates the end of each input to fit within context length. Returns error if `false` and context length is exceeded. Defaults to `true`
1709
- `options`: additional model parameters listed in the documentation for the [Modelfile](./modelfile.md#valid-parameters-and-values) such as `temperature`
Bruce MacDonald's avatar
Bruce MacDonald committed
1710
- `keep_alive`: controls how long the model will stay loaded into memory following the request (default: `5m`)
1711

1712
1713
1714
### Examples

#### Request
1715

Matt Williams's avatar
Matt Williams committed
1716
```shell
royjhan's avatar
royjhan committed
1717
curl http://localhost:11434/api/embed -d '{
Jeffrey Morgan's avatar
Jeffrey Morgan committed
1718
  "model": "all-minilm",
royjhan's avatar
royjhan committed
1719
  "input": "Why is the sky blue?"
1720
1721
1722
}'
```

1723
#### Response
1724
1725
1726

```json
{
royjhan's avatar
royjhan committed
1727
1728
1729
1730
  "model": "all-minilm",
  "embeddings": [[
    0.010071029, -0.0017594862, 0.05007221, 0.04692972, 0.054916814,
    0.008599704, 0.105441414, -0.025878139, 0.12958129, 0.031952348
royjhan's avatar
royjhan committed
1731
1732
1733
1734
  ]],
  "total_duration": 14143917,
  "load_duration": 1019500,
  "prompt_eval_count": 8
royjhan's avatar
royjhan committed
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
}
```

#### Request (Multiple input)

```shell
curl http://localhost:11434/api/embed -d '{
  "model": "all-minilm",
  "input": ["Why is the sky blue?", "Why is the grass green?"]
}'
```

#### Response

```json
{
  "model": "all-minilm",
  "embeddings": [[
    0.010071029, -0.0017594862, 0.05007221, 0.04692972, 0.054916814,
    0.008599704, 0.105441414, -0.025878139, 0.12958129, 0.031952348
  ],[
    -0.0098027075, 0.06042469, 0.025257962, -0.006364387, 0.07272725,
    0.017194884, 0.09032035, -0.051705178, 0.09951512, 0.09072481
  ]]
Costa Alexoglou's avatar
Costa Alexoglou committed
1759
1760
}
```
royjhan's avatar
royjhan committed
1761
1762

## List Running Models
1763
```
royjhan's avatar
royjhan committed
1764
1765
1766
1767
1768
1769
1770
1771
GET /api/ps
```

List models that are currently loaded into memory.

#### Examples

### Request
Jeffrey Morgan's avatar
Jeffrey Morgan committed
1772

royjhan's avatar
royjhan committed
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
```shell
curl http://localhost:11434/api/ps
```

#### Response

A single JSON object will be returned.

```json
{
  "models": [
    {
      "name": "mistral:latest",
      "model": "mistral:latest",
      "size": 5137025024,
      "digest": "2ae6f6dd7a3dd734790bbbf58b8909a606e0e7e97e94b7604e0aa7ae4490e6d8",
      "details": {
        "parent_model": "",
        "format": "gguf",
        "family": "llama",
        "families": [
          "llama"
        ],
        "parameter_size": "7.2B",
        "quantization_level": "Q4_0"
      },
      "expires_at": "2024-06-04T14:38:31.83753-07:00",
      "size_vram": 5137025024
    }
  ]
}
Jeffrey Morgan's avatar
Jeffrey Morgan committed
1804
```
royjhan's avatar
royjhan committed
1805
1806
1807
1808
1809

## Generate Embedding

> Note: this endpoint has been superseded by `/api/embed`

1810
```
royjhan's avatar
royjhan committed
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
POST /api/embeddings
```

Generate embeddings from a model

### Parameters

- `model`: name of model to generate embeddings from
- `prompt`: text to generate embeddings for

Advanced parameters:

- `options`: additional model parameters listed in the documentation for the [Modelfile](./modelfile.md#valid-parameters-and-values) such as `temperature`
- `keep_alive`: controls how long the model will stay loaded into memory following the request (default: `5m`)

### Examples

#### Request

```shell
curl http://localhost:11434/api/embeddings -d '{
  "model": "all-minilm",
  "prompt": "Here is an article about llamas..."
}'
```

#### Response

```json
{
  "embedding": [
    0.5670403838157654, 0.009260174818336964, 0.23178744316101074, -0.2916173040866852, -0.8924556970596313,
    0.8785552978515625, -0.34576427936553955, 0.5742510557174683, -0.04222835972905159, -0.137906014919281
  ]
}
Jeffrey Morgan's avatar
Jeffrey Morgan committed
1846
```
1847
1848
1849

## Version

1850
```
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
GET /api/version
```

Retrieve the Ollama version

### Examples

#### Request

```shell
curl http://localhost:11434/api/version
```

#### Response

```json
{
  "version": "0.5.1"
}
```