[Core][MM] Use non-blocking CPU-GPU copy of multimodal data (#28141)

Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>

[Core][MM] Use non-blocking CPU-GPU copy of multimodal data (#28141)
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
80679f10 · Lukas Geiger · GitHub · 43ecd0a9 · 80679f10
Unverified Commit 80679f10 authored Nov 06, 2025 by Lukas Geiger Committed by GitHub Nov 06, 2025
Hide whitespace changes
Inline Side-by-side

Showing with 3 additions and 1 deletion

vllm/multimodal/utils.py vllm/multimodal/utils.py +3 -1

No files found.
--- a/vllm/multimodal/utils.py
+++ b/vllm/multimodal/utils.py
@@ -444,7 +444,9 @@ def group_mm_kwargs_by_modality(

            if device is not None:
                mm_kwargs_group = json_map_leaves(
-                    lambda x: x.to(device=device) if isinstance(x, torch.Tensor) else x,
+                    lambda x: x.to(device=device, non_blocking=True)
+                    if isinstance(x, torch.Tensor)
+                    else x,
                    mm_kwargs_group,
                )
        else: