Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
text-generation-inference
Commits
648ea064
Unverified
Commit
648ea064
authored
Oct 19, 2023
by
star
Committed by
GitHub
Oct 19, 2023
Browse files
fix: EETQLinear with bias in layers.py (#1176)
parent
9179605e
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
4 deletions
+1
-4
server/text_generation_server/utils/layers.py
server/text_generation_server/utils/layers.py
+1
-4
No files found.
server/text_generation_server/utils/layers.py
View file @
648ea064
...
...
@@ -155,10 +155,7 @@ class EETQLinear(nn.Module):
device
=
weight
.
device
weight
=
torch
.
t
(
weight
).
contiguous
().
cpu
()
weight
,
scale
=
quant_weights
(
weight
,
torch
.
int8
,
False
)
if
bias
:
bias
=
weights
.
get_tensor
(
f
"
{
prefix
}
.bias"
)
else
:
bias
=
None
self
.
weight
=
weight
.
cuda
(
device
)
self
.
scale
=
scale
.
cuda
(
device
)
self
.
bias
=
bias
.
cuda
(
device
)
if
bias
is
not
None
else
None
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment