Unverified Commit 716e6827 authored by b8zhong's avatar b8zhong Committed by GitHub
Browse files

[Fix] Add undefined `update_tensor_inplace` function (#6307)

parent 84b30d9e
...@@ -146,6 +146,10 @@ def requantize_with_max_scale( ...@@ -146,6 +146,10 @@ def requantize_with_max_scale(
return max_w_scale, weight return max_w_scale, weight
def update_tensor_inplace(old: torch.Tensor, new: torch.Tensor) -> None:
old.copy_(new)
# Adapted from https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/layers/quantization/utils/layer_utils.py # Adapted from https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/layers/quantization/utils/layer_utils.py
# Newly generated tensors need to replace existing tensors that are # Newly generated tensors need to replace existing tensors that are
# already registered as parameters by vLLM (and won't be freed) # already registered as parameters by vLLM (and won't be freed)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment