Unverified Commit cea96a01 authored by Rui Qiao's avatar Rui Qiao Committed by GitHub
Browse files

[Bugfix] Fix sync_and_slice_intermediate_tensors (#21537)


Signed-off-by: default avatarRui Qiao <ruisearch42@gmail.com>
parent 2eddd437
......@@ -1270,7 +1270,7 @@ class GPUModelRunner(LoRAModelRunnerMixin, KVConnectorModelRunnerMixin):
if sync_self:
assert intermediate_tensors is not None
for k, v in intermediate_tensors.items():
is_scattered = "residual" and is_residual_scattered
is_scattered = k == "residual" and is_residual_scattered
copy_len = num_tokens // tp if is_scattered else \
num_tokens
self.intermediate_tensors[k][:copy_len].copy_(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment