Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
norm
vllm
Commits
5c4471ef
Commit
5c4471ef
authored
May 15, 2024
by
zhuwenwen
Browse files
ignore linear_method layout
parent
47c04371
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
0 deletions
+3
-0
vllm/model_executor/model_loader.py
vllm/model_executor/model_loader.py
+3
-0
No files found.
vllm/model_executor/model_loader.py
View file @
5c4471ef
...
@@ -65,6 +65,9 @@ def get_model(model_config: ModelConfig, device_config: DeviceConfig,
...
@@ -65,6 +65,9 @@ def get_model(model_config: ModelConfig, device_config: DeviceConfig,
f
"
{
supported_dtypes
}
"
)
f
"
{
supported_dtypes
}
"
)
linear_method
=
quant_config
.
get_linear_method
()
linear_method
=
quant_config
.
get_linear_method
()
if
linear_method
!=
None
:
os
.
environ
[
'LLAMA_NN'
]
=
'0'
with
_set_default_torch_dtype
(
model_config
.
dtype
):
with
_set_default_torch_dtype
(
model_config
.
dtype
):
# Create a model instance.
# Create a model instance.
# The weights will be initialized as empty tensors.
# The weights will be initialized as empty tensors.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment