Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
AutoAWQ
Commits
e205548d
Commit
e205548d
authored
Sep 20, 2023
by
Casper Hansen
Browse files
Move import up
parent
2290bdb2
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
2 deletions
+1
-2
awq/models/base.py
awq/models/base.py
+1
-2
No files found.
awq/models/base.py
View file @
e205548d
...
...
@@ -8,6 +8,7 @@ from typing import List, Union
from
safetensors.torch
import
save_file
from
awq.modules.act
import
ScaledActivation
from
huggingface_hub
import
snapshot_download
from
awq.quantize.quantizer
import
AwqQuantizer
from
awq.utils.utils
import
simple_dispatch_model
from
transformers.modeling_utils
import
shard_checkpoint
from
awq.modules.linear
import
WQLinear_GEMM
,
WQLinear_GEMV
...
...
@@ -41,8 +42,6 @@ class BaseAWQForCausalLM(nn.Module):
self
.
quant_config
=
quant_config
quant_config
[
"version"
]
=
"GEMM"
if
'version'
not
in
quant_config
.
keys
()
else
quant_config
[
"version"
]
from
awq.quantize.quantizer
import
AwqQuantizer
quantizer
=
AwqQuantizer
(
self
,
self
.
model
,
tokenizer
,
quant_config
[
"w_bit"
],
quant_config
[
"q_group_size"
],
quant_config
[
"version"
],
calib_data
,
split
,
text_column
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment