Skip to content

GitLab

Unverified Commit 73b3de79 authored by Xinyu Yang's avatar Xinyu Yang Committed by GitHub
Browse files

explicitly del state (#784)

parent d1744376
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
vllm/model_executor/weight_utils.py
View file @ 73b3de79
......@@ -76,6 +76,8 @@ def hf_model_weights_iterator(
state = torch.load(bin_file, map_location="cpu")
for name, param in state.items():
yield name, param
del state
torch.cuda.empty_cache()
def load_tensor_parallel_weights(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment