Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
norm
vllm
Commits
5c4471ef
"vscode:/vscode.git/clone" did not exist on "ebdbfde0cee9d6adca2c0508f1a664c13d3cd65a"
Commit
5c4471ef
authored
May 15, 2024
by
zhuwenwen
Browse files
ignore linear_method layout
parent
47c04371
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
0 deletions
+3
-0
vllm/model_executor/model_loader.py
vllm/model_executor/model_loader.py
+3
-0
No files found.
vllm/model_executor/model_loader.py
View file @
5c4471ef
...
...
@@ -65,6 +65,9 @@ def get_model(model_config: ModelConfig, device_config: DeviceConfig,
f
"
{
supported_dtypes
}
"
)
linear_method
=
quant_config
.
get_linear_method
()
if
linear_method
!=
None
:
os
.
environ
[
'LLAMA_NN'
]
=
'0'
with
_set_default_torch_dtype
(
model_config
.
dtype
):
# Create a model instance.
# The weights will be initialized as empty tensors.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment