Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
4cf256ae
Unverified
Commit
4cf256ae
authored
Jul 15, 2024
by
youkaichao
Committed by
GitHub
Jul 15, 2024
Browse files
[misc][distributed] fix pp missing layer condition (#6446)
parent
64fdc08c
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
1 deletion
+4
-1
vllm/model_executor/models/utils.py
vllm/model_executor/models/utils.py
+4
-1
No files found.
vllm/model_executor/models/utils.py
View file @
4cf256ae
...
...
@@ -83,7 +83,10 @@ def get_pp_missing_layer_names(model: torch.nn.Module) -> List[str]:
missing_layer_names
=
[]
for
name
,
module
in
model
.
named_modules
():
if
isinstance
(
module
,
PPMissingLayer
):
missing_layer_names
.
append
(
name
)
# NOTE: the trailing dot is used to match the prefix of the layer.
# without the dot, we could match a layer that is not missing,
# e.g., 'encoder.layer.1' would match 'encoder.layer.11'
missing_layer_names
.
append
(
name
+
'.'
)
_model_to_pp_missing_layer_names
[
model_id
]
=
missing_layer_names
return
missing_layer_names
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment