Skip to content

GitLab

Unverified Commit e5c67150 authored by HAI's avatar HAI Committed by GitHub
Browse files

Fix core (MI300X) with --enable-overlap (#2048)

parent 023d0a73
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
python/sglang/srt/managers/schedule_batch.py
View file @ e5c67150
...@@ -588,14 +588,14 @@ class ScheduleBatch: ...@@ -588,14 +588,14 @@ class ScheduleBatch:
) )
if not decoder_out_cache_loc: if not decoder_out_cache_loc:
self.out_cache_loc = torch.empty(0, dtype=torch.int32).to( self.out_cache_loc = torch.zeros(0, dtype=torch.int32).to(
self.device, non_blocking=True self.device, non_blocking=True
) )
else: else:
self.out_cache_loc = torch.cat(decoder_out_cache_loc) self.out_cache_loc = torch.cat(decoder_out_cache_loc)
if not encoder_out_cache_loc: if not encoder_out_cache_loc:
self.encoder_out_cache_loc = torch.empty(0, dtype=torch.int32).to( self.encoder_out_cache_loc = torch.zeros(0, dtype=torch.int32).to(
self.device, non_blocking=True self.device, non_blocking=True
) )
else: else:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment