Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
4d70732e
Commit
4d70732e
authored
Jan 19, 2026
by
王敏
Browse files
[fix]解决gpt oss nn moe权重加载出错
parent
ae0dc0ac
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
11 additions
and
4 deletions
+11
-4
vllm/model_executor/models/gpt_oss.py
vllm/model_executor/models/gpt_oss.py
+11
-4
No files found.
vllm/model_executor/models/gpt_oss.py
View file @
4d70732e
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
import
os
from
collections.abc
import
Iterable
from
typing
import
Optional
...
...
@@ -253,6 +254,7 @@ class GptOssModel(nn.Module):
make_empty_intermediate_tensors_factory
(
[
"hidden_states"
,
"residual"
],
self
.
config
.
hidden_size
))
self
.
aux_hidden_state_layers
=
tuple
[
int
,
...]()
self
.
use_nn_moe
=
int
(
os
.
environ
.
get
(
'MOE_NN'
,
1
))
==
1
def
get_input_embeddings
(
self
,
input_ids
:
torch
.
Tensor
)
->
torch
.
Tensor
:
return
self
.
embedding
(
input_ids
)
...
...
@@ -522,9 +524,11 @@ class GptOssModel(nn.Module):
narrow_weight
=
weight
[
ep_rank_start
:
ep_rank_end
,
...]
else
:
narrow_weight
=
weight
[:,
:,
2
*
tp_rank_start
:
2
*
tp_rank_end
]
2
*
tp_rank_start
:
2
*
tp_rank_end
]
if
not
self
.
use_nn_moe
:
narrow_weight
=
narrow_weight
.
permute
(
0
,
2
,
1
).
contiguous
()
narrow_weight
=
narrow_weight
.
permute
(
0
,
2
,
1
).
contiguous
()
param
=
params_dict
[
name
]
param
.
copy_
(
narrow_weight
)
...
...
@@ -536,9 +540,12 @@ class GptOssModel(nn.Module):
narrow_weight
=
weight
[
ep_rank_start
:
ep_rank_end
,
...]
else
:
narrow_weight
=
weight
[:,
tp_rank_start
:
tp_rank_end
,
:]
narrow_weight
=
narrow_weight
.
permute
(
0
,
2
,
1
).
contiguous
()
param
=
params_dict
[
name
]
if
not
self
.
use_nn_moe
:
narrow_weight
=
narrow_weight
.
permute
(
0
,
2
,
1
).
contiguous
()
param
=
params_dict
[
name
]
param
.
copy_
(
narrow_weight
)
loaded_params
.
add
(
name
)
continue
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment