Skip to content

GitLab

Unverified Commit 4e33a7ea authored by Naman Lalit's avatar Naman Lalit Committed by GitHub
Browse files

[Bugfix] Optimize CpuGpuBuffer initialization (#25447) 


Signed-off-by: default avatarNaman Lalit <nl2688@nyu.edu>
parent dc48ba0c
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/v1/utils.py
View file @ 4e33a7ea
...@@ -117,7 +117,7 @@ class CpuGpuBuffer: ...@@ -117,7 +117,7 @@ class CpuGpuBuffer:
dtype=dtype, dtype=dtype,
device="cpu", device="cpu",
pin_memory=pin_memory) pin_memory=pin_memory)
self.gpu = self.cpu.to(device) self.gpu = torch.zeros_like(self.cpu, device=device)
self.np: np.ndarray self.np: np.ndarray
# To keep type hints simple (avoiding generics and subclasses), we # To keep type hints simple (avoiding generics and subclasses), we
# only conditionally create the numpy array attribute. This can cause # only conditionally create the numpy array attribute. This can cause
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment