Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
text-generation-inference
Commits
eb188324
Commit
eb188324
authored
Feb 26, 2025
by
xuxzh1
🎱
Browse files
Update kv_cache.py
parent
aa6b01c6
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
server/text_generation_server/layers/attention/kv_cache.py
server/text_generation_server/layers/attention/kv_cache.py
+1
-1
No files found.
server/text_generation_server/layers/attention/kv_cache.py
View file @
eb188324
...
@@ -215,7 +215,7 @@ def paged_reshape_and_cache(
...
@@ -215,7 +215,7 @@ def paged_reshape_and_cache(
raise
ImportError
(
raise
ImportError
(
f
"Could not import vllm paged attention. Make sure your installation is correct. Complete error:
{
e
}
"
f
"Could not import vllm paged attention. Make sure your installation is correct. Complete error:
{
e
}
"
)
)
ops
.
reshape_and_cache
(
key
,
value
,
key_cache
,
value_cache
,
slots
,
"auto"
,
1.0
)
ops
.
reshape_and_cache
(
key
,
value
,
key_cache
,
value_cache
,
slots
,
"auto"
,
1.0
,
1.0
)
elif
SYSTEM
==
"ipex"
:
elif
SYSTEM
==
"ipex"
:
import
intel_extension_for_pytorch
as
ipex
import
intel_extension_for_pytorch
as
ipex
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment