Skip to content

GitLab

Switch branch/tag
Find file BlameHistoryPermalink
  • Jesse Gross's avatar
    sched: Lift parallel restriction for multimodal models except mllama · 6cd56687
    Jesse Gross authored 
    The Go runner does not have a problem with supporting parallel
requests for most multimodal models. Now that we won't be potentially
falling back to server.cpp, this restriction can be lifted.

However, the new mllama model can't support parallel requests, so we
will need to keep a restriction for that.
    6cd56687
sched.go 29 KB