Skip to content

GitLab

Unverified Commit c53711bd authored by Ning Xie's avatar Ning Xie Committed by GitHub
Browse files

[MISC] correct copy_blocks src_to_dists param type (#19696) 


Signed-off-by: default avatarAndy Xie <andy.xning@gmail.com>
parent dac8cc49
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
vllm/attention/ops/ipex_attn.py
View file @ c53711bd
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
from typing import Dict, List, Optional, Tuple from typing import List, Optional, Tuple
try: try:
import intel_extension_for_pytorch.llm.modules as ipex_modules import intel_extension_for_pytorch.llm.modules as ipex_modules
...@@ -120,7 +120,7 @@ class _PagedAttention: ...@@ -120,7 +120,7 @@ class _PagedAttention:
@staticmethod @staticmethod
def copy_blocks( def copy_blocks(
kv_caches: List[torch.Tensor], kv_caches: List[torch.Tensor],
src_to_dists: Dict[int, List[int]], src_to_dists: torch.Tensor,
*args, *args,
) -> None: ) -> None:
key_caches = [kv_cache[0] for kv_cache in kv_caches] key_caches = [kv_cache[0] for kv_cache in kv_caches]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment