sched: Lift parallel restriction for multimodal models except mllama
The Go runner does not have a problem with supporting parallel requests for most multimodal models. Now that we won't be potentially falling back to server.cpp, this restriction can be lifted. However, the new mllama model can't support parallel requests, so we will need to keep a restriction for that.
Showing
Please register or sign in to comment