• Jesse Gross's avatar
    server: Don't clear cmd when closing a server · 03408f34
    Jesse Gross authored
    Close can be called on an LLM server if the runner subprocess dies.
    However, the Ollama scheduler code may not know about this yet and
    still try to access it. In this case, it is important that 'cmd'
    is still available as it is used to check on the status of the
    subprocess. If this happens, Kill may be called twice on the subprocess -
    that is fine.
    
    In addition, model unloading may race with new accesses, so we should
    hold a lock around this. This may result in the model being reloaded
    after the first close call - this is also fine as close will be called
    again later.
    03408f34
server.go 32.4 KB