Skip to content

GitLab

You need to sign in or sign up before continuing.
Unverified Commit 02543b54 authored by sogalin's avatar sogalin Committed by GitHub
Browse files

Fix misusing the "_is_cuda". (#7091)

parent 25a6a9aa
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
python/sglang/srt/mem_cache/memory_pool.py
View file @ 02543b54
......@@ -267,7 +267,7 @@ class MHATokenToKVPool(KVCache):
self.layer_transfer_counter = None
self.device_module = torch.get_device_module(self.device)
self.alt_stream = self.device_module.Stream() if is_cuda else None
self.alt_stream = self.device_module.Stream() if _is_cuda else None
k_size, v_size = self.get_kv_size_bytes()
logger.info(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment