Skip to content

GitLab

Unverified Commit a897e833 authored by Bruce MacDonald's avatar Bruce MacDonald Committed by GitHub
Browse files

do not cache prompt (#2018) 

- prompt cache causes inferance to hang after some time
parent eef50acc
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 1 deletion
llm/dyn_ext_server.go
View file @ a897e833
...@@ -181,7 +181,6 @@ func (llm *dynExtServer) Predict(ctx context.Context, predict PredictOpts, fn fu ...@@ -181,7 +181,6 @@ func (llm *dynExtServer) Predict(ctx context.Context, predict PredictOpts, fn fu
"seed": predict.Options.Seed, "seed": predict.Options.Seed,
"stop": predict.Options.Stop, "stop": predict.Options.Stop,
"image_data": imageData, "image_data": imageData,
"cache_prompt": true,
} }
if predict.Format == "json" { if predict.Format == "json" {
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment