Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
text-generation-inference
Commits
197c47a3
Unverified
Commit
197c47a3
authored
Jun 21, 2024
by
Daniël de Kok
Committed by
GitHub
Jun 21, 2024
Browse files
Fix `text-generation-server quantize` (#2103)
The subcommand did not work due to some broken imports.
parent
bcb3faa1
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
2 additions
and
2 deletions
+2
-2
server/text_generation_server/cli.py
server/text_generation_server/cli.py
+1
-1
server/text_generation_server/layers/gptq/quantize.py
server/text_generation_server/layers/gptq/quantize.py
+1
-1
No files found.
server/text_generation_server/cli.py
View file @
197c47a3
...
...
@@ -316,7 +316,7 @@ def quantize(
logger_level
=
logger_level
,
json_output
=
json_output
,
)
from
text_generation_server.
util
s.gptq.quantize
import
quantize
from
text_generation_server.
layer
s.gptq.quantize
import
quantize
quantize
(
model_id
=
model_id
,
...
...
server/text_generation_server/layers/gptq/quantize.py
View file @
197c47a3
...
...
@@ -12,7 +12,7 @@ from huggingface_hub import HfApi
from
accelerate
import
init_empty_weights
from
text_generation_server.utils
import
initialize_torch_distributed
,
Weights
from
text_generation_server.utils.hub
import
weight_files
from
text_generation_server.
util
s.gptq.quant_linear
import
QuantLinear
from
text_generation_server.
layer
s.gptq.quant_linear
import
QuantLinear
from
loguru
import
logger
from
typing
import
Optional
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment