Skip to content

GitLab

"git@developer.sourcefind.cn:OpenDAS/vision.git" did not exist on "622bb2699d281c244f452bcb6a529fa897cae2bd"
Unverified Commit ea0fdaed authored by Michael Yang's avatar Michael Yang Committed by GitHub
Browse files

Merge pull request #4320 from ollama/mxyng/phi2-mem 

add phi2 mem
parents bb6fd022 1eb382da
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 1 deletion
llm/ggml.go
View file @ ea0fdaed
...@@ -329,7 +329,10 @@ func (llm GGML) GraphSize(context, batch uint64) (partialOffload, fullOffload ui ...@@ -329,7 +329,10 @@ func (llm GGML) GraphSize(context, batch uint64) (partialOffload, fullOffload ui
4*batch*(1+4*embedding+context+context*heads), 4*batch*(1+4*embedding+context+context*heads),
) )
partialOffload = 4*batch*(2*embedding+vocab) + embedding*vocab*105/128 partialOffload = max(
4*batch*(2*embedding+vocab)+embedding*vocab*105/128,
4*batch*(2+3*embedding+context+context*heads),
)
case "stablelm": case "stablelm":
fullOffload = 4 * batch * (context*(1+heads) + 3*embedding + 2) fullOffload = 4 * batch * (context*(1+heads) + 3*embedding + 2)
partialOffload = max( partialOffload = max(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment