Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
norm
vllm
Commits
e946260c
Unverified
Commit
e946260c
authored
Nov 19, 2023
by
twaka
Committed by
GitHub
Nov 18, 2023
Browse files
use get_tensor in safe_open (#1696)
parent
edb30558
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
8 deletions
+3
-8
vllm/model_executor/weight_utils.py
vllm/model_executor/weight_utils.py
+3
-8
No files found.
vllm/model_executor/weight_utils.py
View file @
e946260c
...
@@ -243,8 +243,8 @@ def hf_model_weights_iterator(
...
@@ -243,8 +243,8 @@ def hf_model_weights_iterator(
for
st_file
in
hf_weights_files
:
for
st_file
in
hf_weights_files
:
with
safe_open
(
st_file
,
framework
=
"pt"
)
as
f
:
with
safe_open
(
st_file
,
framework
=
"pt"
)
as
f
:
for
name
in
f
.
keys
():
for
name
in
f
.
keys
():
param
=
f
.
get_
slice
(
name
)
param
=
f
.
get_
tensor
(
name
)
yield
name
,
convert_pyslice_to_tensor
(
param
)
yield
name
,
param
else
:
else
:
for
bin_file
in
hf_weights_files
:
for
bin_file
in
hf_weights_files
:
state
=
torch
.
load
(
bin_file
,
map_location
=
"cpu"
)
state
=
torch
.
load
(
bin_file
,
map_location
=
"cpu"
)
...
@@ -265,12 +265,7 @@ def convert_pyslice_to_tensor(x: Any) -> torch.Tensor:
...
@@ -265,12 +265,7 @@ def convert_pyslice_to_tensor(x: Any) -> torch.Tensor:
tensor first.
tensor first.
"""
"""
if
not
isinstance
(
x
,
torch
.
Tensor
):
if
not
isinstance
(
x
,
torch
.
Tensor
):
try
:
x
=
x
[:]
x
=
x
[:]
except
IndexError
:
# IndexError happens when the tensor is empty.
# transformer.h.0.attn.masked_bias is empty in some gpt2 models.
return
torch
.
Tensor
()
return
x
return
x
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment