Skip to content

GitLab

Unverified Commit 38616920 authored by Michael Yang's avatar Michael Yang Committed by GitHub
Browse files

update runtime options (#864)

parent 0d6342a8
Show whitespace changes
Inline Side-by-side
Showing with 4 additions and 0 deletions
server/routes.go
View file @ 38616920
...@@ -112,6 +112,10 @@ func load(ctx context.Context, workDir string, model *Model, reqOpts map[string] ...@@ -112,6 +112,10 @@ func load(ctx context.Context, workDir string, model *Model, reqOpts map[string]
loaded.Options = &opts loaded.Options = &opts
} }
// update options for the loaded llm
// TODO(mxyng): this isn't thread safe, but it should be fine for now
loaded.runner.SetOptions(opts)
loaded.expireAt = time.Now().Add(sessionDuration) loaded.expireAt = time.Now().Add(sessionDuration)
if loaded.expireTimer == nil { if loaded.expireTimer == nil {
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment