Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ollama
Commits
371d4e5d
Commit
371d4e5d
authored
Aug 08, 2023
by
Jeffrey Morgan
Browse files
docs: fix invalid json in `api.md`
parent
1f78e409
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
11 additions
and
10 deletions
+11
-10
docs/api.md
docs/api.md
+11
-10
No files found.
docs/api.md
View file @
371d4e5d
...
@@ -62,27 +62,28 @@ A stream of JSON objects:
...
@@ -62,27 +62,28 @@ A stream of JSON objects:
The final response in the stream also includes additional data about the generation:
The final response in the stream also includes additional data about the generation:
-
`total_duration`
: time spent generating the response
-
`load_duration`
: time spent in nanoseconds loading the model
-
`sample_count`
: number of samples generated
-
`sample_duration`
: time spent generating samples
-
`prompt_eval_count`
: number of tokens in the prompt
-
`prompt_eval_duration`
: time spent in nanoseconds evaluating the prompt
-
`eval_count`
: number of tokens the response
-
`eval_duration`
: time in nanoseconds spent generating the response
To calculate how fast the response is generated in tokens per second (token/s), divide
`eval_count`
/
`eval_duration`
.
```
json
```
json
{
{
"model"
:
"llama2:7b"
,
"model"
:
"llama2:7b"
,
"created_at"
:
"2023-08-04T19:22:45.499127Z"
,
"created_at"
:
"2023-08-04T19:22:45.499127Z"
,
"done"
:
true
,
"done"
:
true
,
//
total
time
in
nanoseconds
spent
generating
the
response
"total_duration"
:
5589157167
,
"total_duration"
:
5589157167
,
//
time
spent
in
nanoseconds
loading
the
model
"load_duration"
:
3013701500
,
"load_duration"
:
3013701500
,
//
Sample:
how
fast
tokens
were
sampled
"sample_count"
:
114
,
"sample_count"
:
114
,
"sample_duration"
:
81442000
,
"sample_duration"
:
81442000
,
//
Prompt
stats:
how
fast
the
prompt
was
evaluated
"prompt_eval_count"
:
46
,
"prompt_eval_count"
:
46
,
"prompt_eval_duration"
:
1160282000
,
"prompt_eval_duration"
:
1160282000
,
//
Eval
stats:
how
fast
tokens
were
generated
by
the
model
"eval_count"
:
113
,
"eval_count"
:
113
,
"eval_duration"
:
1325948000
"eval_duration"
:
1325948000
}
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment