Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
Lmdeploy
Commits
e0c7f51b
Unverified
Commit
e0c7f51b
authored
Jun 25, 2023
by
tpoisonooo
Committed by
GitHub
Jun 25, 2023
Browse files
fix(deploy.py): qkv no bias assertion
parent
1b7151c1
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
llmdeploy/serve/fastertransformer/deploy.py
llmdeploy/serve/fastertransformer/deploy.py
+1
-1
No files found.
llmdeploy/serve/fastertransformer/deploy.py
View file @
e0c7f51b
...
@@ -245,7 +245,7 @@ def deploy_llama(model_name: str, model_path: str, tokenizer_path: str,
...
@@ -245,7 +245,7 @@ def deploy_llama(model_name: str, model_path: str, tokenizer_path: str,
print
(
f
'layers.
{
i
}
.attention.w_qkv.
{
t
}
'
,
qkv
.
shape
)
print
(
f
'layers.
{
i
}
.attention.w_qkv.
{
t
}
'
,
qkv
.
shape
)
model_params
[
f
'layers.
{
i
}
.attention.w_qkv.
{
t
}
'
]
=
qkv
model_params
[
f
'layers.
{
i
}
.attention.w_qkv.
{
t
}
'
]
=
qkv
assert
num_layer
==
i
,
f
'miss matched layers:
{
num_layer
}
vs
{
i
}
'
assert
i
==
0
or
num_layer
==
i
,
f
'miss matched layers:
{
num_layer
}
vs
{
i
}
'
return
export
(
model_name
,
num_layer
,
norm_eps
,
model_params
,
return
export
(
model_name
,
num_layer
,
norm_eps
,
model_params
,
tokenizer_path
,
triton_models_path
,
tp
)
tokenizer_path
,
triton_models_path
,
tp
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment