Unverified Commit 1d0ebe67 authored by Jay Nakrani's avatar Jay Nakrani Committed by GitHub
Browse files

Document response stream chunk delimiter. (#632)

Document response stream chunk delimiter.
parent a1b2d95f
...@@ -23,6 +23,10 @@ Model names follow a `model:tag` format. Some examples are `orca-mini:3b-q4_1` a ...@@ -23,6 +23,10 @@ Model names follow a `model:tag` format. Some examples are `orca-mini:3b-q4_1` a
All durations are returned in nanoseconds. All durations are returned in nanoseconds.
### Streaming responses
Certain endpoints stream responses as JSON objects delineated with the newline (`\n`) character.
## Generate a completion ## Generate a completion
```shell ```shell
......
...@@ -610,6 +610,7 @@ func streamResponse(c *gin.Context, ch chan any) { ...@@ -610,6 +610,7 @@ func streamResponse(c *gin.Context, ch chan any) {
return false return false
} }
// Delineate chunks with new-line delimiter
bts = append(bts, '\n') bts = append(bts, '\n')
if _, err := w.Write(bts); err != nil { if _, err := w.Write(bts); err != nil {
log.Printf("streamResponse: w.Write failed with %s", err) log.Printf("streamResponse: w.Write failed with %s", err)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment