Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
1a493a24
Commit
1a493a24
authored
Nov 20, 2024
by
zhuwenwen
Browse files
remove xops
parent
fc7db442
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
5 additions
and
3 deletions
+5
-3
tests/kernels/test_prefix_prefill.py
tests/kernels/test_prefix_prefill.py
+5
-3
No files found.
tests/kernels/test_prefix_prefill.py
View file @
1a493a24
...
@@ -4,14 +4,16 @@ import time
...
@@ -4,14 +4,16 @@ import time
import
pytest
import
pytest
import
torch
import
torch
from
xformers
import
ops
as
xops
from
xformers.ops.fmha.attn_bias
import
BlockDiagonalCausalFromBottomRightMask
from
vllm.attention.backends.xformers
import
_make_alibi_bias
from
vllm.attention.ops.prefix_prefill
import
context_attention_fwd
from
vllm.attention.ops.prefix_prefill
import
context_attention_fwd
from
vllm.utils
import
is_hip
from
vllm.utils
import
is_hip
from
vllm.utils
import
STR_DTYPE_TO_TORCH_DTYPE
,
seed_everything
from
vllm.utils
import
STR_DTYPE_TO_TORCH_DTYPE
,
seed_everything
if
not
is_hip
():
from
xformers
import
ops
as
xops
from
xformers.ops.fmha.attn_bias
import
BlockDiagonalCausalFromBottomRightMask
from
vllm.attention.backends.xformers
import
_make_alibi_bias
NUM_HEADS
=
[
64
]
NUM_HEADS
=
[
64
]
NUM_QUERIES_PER_KV
=
[
1
,
8
,
64
]
NUM_QUERIES_PER_KV
=
[
1
,
8
,
64
]
HEAD_SIZES
=
[
128
,
96
,
24
]
HEAD_SIZES
=
[
128
,
96
,
24
]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment