Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
ollama
Commits
557110d0
Unverified
Commit
557110d0
authored
Jan 13, 2024
by
Jeffrey Morgan
Committed by
GitHub
Jan 13, 2024
Browse files
Disable `mmap` with lora layers (#1985)
parent
288ef8ff
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
7 additions
and
3 deletions
+7
-3
llm/ext_server/ext_server.cpp
llm/ext_server/ext_server.cpp
+7
-3
No files found.
llm/ext_server/ext_server.cpp
View file @
557110d0
...
@@ -47,11 +47,15 @@ void llama_server_init(ext_server_params *sparams, ext_server_resp_t *err) {
...
@@ -47,11 +47,15 @@ void llama_server_init(ext_server_params *sparams, ext_server_resp_t *err) {
params
.
model
=
sparams
->
model
;
params
.
model
=
sparams
->
model
;
}
}
if
(
sparams
->
lora_adapters
!=
NULL
)
{
for
(
ext_server_lora_adapter
*
la
=
sparams
->
lora_adapters
;
la
!=
NULL
;
for
(
ext_server_lora_adapter
*
la
=
sparams
->
lora_adapters
;
la
!=
NULL
;
la
=
la
->
next
)
{
la
=
la
->
next
)
{
params
.
lora_adapter
.
push_back
(
std
::
make_tuple
(
la
->
adapter
,
la
->
scale
));
params
.
lora_adapter
.
push_back
(
std
::
make_tuple
(
la
->
adapter
,
la
->
scale
));
}
}
params
.
use_mmap
=
false
;
}
if
(
sparams
->
mmproj
!=
NULL
)
{
if
(
sparams
->
mmproj
!=
NULL
)
{
params
.
mmproj
=
std
::
string
(
sparams
->
mmproj
);
params
.
mmproj
=
std
::
string
(
sparams
->
mmproj
);
}
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment