Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
AutoAWQ
Commits
133dd7a7
Commit
133dd7a7
authored
Sep 21, 2023
by
Casper
Browse files
Fix usage of engine
parent
fbeea40b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
awq/modules/fused/attn.py
awq/modules/fused/attn.py
+1
-1
No files found.
awq/modules/fused/attn.py
View file @
133dd7a7
...
...
@@ -219,7 +219,7 @@ class QuantAttentionFused(nn.Module):
xv
=
xv
.
view
((
bsz
,)
+
self
.
attention_shapes
[
"single_xv_view"
])
past_key_value
=
(
xk
,
xv
)
if
use_cache
else
None
attention_weight
=
awq
_inference_engine
.
single_query_attention
(
attention_weight
=
ft
_inference_engine
.
single_query_attention
(
xq
,
# query
xk
,
# key
xv
,
# value
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment