Skip to content

GitLab

Unverified Commit f8977c23 authored by Chenyaaang's avatar Chenyaaang Committed by GitHub
Browse files

Fix an error in dummy weight loading for quantization models (#18855) 


Signed-off-by: default avatarChenyaaang <chenyangli@google.com>
parent f274581f
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/model_executor/model_loader/weight_utils.py
View file @ f8977c23
...@@ -696,7 +696,7 @@ def initialize_dummy_weights( ...@@ -696,7 +696,7 @@ def initialize_dummy_weights(
# Note: We avoid using torch.rank_like as it doesn't currently # Note: We avoid using torch.rank_like as it doesn't currently
# support the generator argument. # support the generator argument.
param.copy_((high - low) * param.copy_((high - low) *
torch.rand(*param.shape, torch.rand(param.shape,
generator=generator, generator=generator,
dtype=param.dtype, dtype=param.dtype,
layout=param.layout, layout=param.layout,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment