Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ollama
Commits
a3fcecf9
"...text-generation-inference.git" did not exist on "c7e570e59de7dc47c35acb1b14d5fe31b123b234"
Commit
a3fcecf9
authored
Nov 20, 2023
by
Jeffrey Morgan
Browse files
only set `main_gpu` if value > 0 is provided
parent
df07e4a0
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
1 deletion
+4
-1
llm/llama.go
llm/llama.go
+4
-1
No files found.
llm/llama.go
View file @
a3fcecf9
...
...
@@ -339,11 +339,14 @@ func newLlama(model string, adapters []string, runners []ModelRunner, numLayers
"--model"
,
model
,
"--ctx-size"
,
fmt
.
Sprintf
(
"%d"
,
opts
.
NumCtx
),
"--batch-size"
,
fmt
.
Sprintf
(
"%d"
,
opts
.
NumBatch
),
"--main-gpu"
,
fmt
.
Sprintf
(
"%d"
,
opts
.
MainGPU
),
"--n-gpu-layers"
,
fmt
.
Sprintf
(
"%d"
,
numGPU
),
"--embedding"
,
}
if
opts
.
MainGPU
>
0
{
params
=
append
(
params
,
"--main-gpu"
,
fmt
.
Sprintf
(
"%d"
,
opts
.
MainGPU
))
}
if
opts
.
RopeFrequencyBase
>
0
{
params
=
append
(
params
,
"--rope-freq-base"
,
fmt
.
Sprintf
(
"%f"
,
opts
.
RopeFrequencyBase
))
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment