Skip to content

GitLab

Unverified Commit fc8e1086 authored by Michael Yang's avatar Michael Yang Committed by GitHub
Browse files

Merge pull request #3496 from ollama/mxyng/cmd-r-graph 

add command-r graph estimate
parents c5d5c4a9 01f77ae2
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
llm/ggml.go
View file @ fc8e1086
...@@ -327,7 +327,7 @@ func (llm GGML) GraphSize(context, batch int) (int64, bool) { ...@@ -327,7 +327,7 @@ func (llm GGML) GraphSize(context, batch int) (int64, bool) {
} }
switch llm.KV().Architecture() { switch llm.KV().Architecture() {
case "gemma": case "gemma", "command-r":
return 4 * int64(batch) * int64(embeddingLength+uint64(vocabLength)), true return 4 * int64(batch) * int64(embeddingLength+uint64(vocabLength)), true
case "phi2": case "phi2":
return max( return max(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment