Skip to content

GitLab

"src/git@developer.sourcefind.cn:OpenDAS/lmdeploy.git" did not exist on "5b9e454aefeccad8bff4113079e8b92e65ad0c0b"
Unverified Commit 73b3de79 authored by Xinyu Yang's avatar Xinyu Yang Committed by GitHub
Browse files

explicitly del state (#784)

parent d1744376
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
vllm/model_executor/weight_utils.py
View file @ 73b3de79
...@@ -76,6 +76,8 @@ def hf_model_weights_iterator( ...@@ -76,6 +76,8 @@ def hf_model_weights_iterator(
state = torch.load(bin_file, map_location="cpu") state = torch.load(bin_file, map_location="cpu")
for name, param in state.items(): for name, param in state.items():
yield name, param yield name, param
del state
torch.cuda.empty_cache()
def load_tensor_parallel_weights( def load_tensor_parallel_weights(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment