Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
AutoGPTQ
Commits
19f50f89
Commit
19f50f89
authored
Nov 28, 2024
by
yangql
Browse files
Update README.md, test_quant.py files
parent
6d26a146
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
4 additions
and
1 deletion
+4
-1
README.md
README.md
+3
-0
test_quant.py
test_quant.py
+1
-1
No files found.
README.md
View file @
19f50f89
...
@@ -37,6 +37,9 @@ cd dist && pip3 install auto_gptq*
...
@@ -37,6 +37,9 @@ cd dist && pip3 install auto_gptq*
#### 量化模型
#### 量化模型
方式1、
方式1、
下载https://huggingface.co/datasets/llm-wizard/alpaca-gpt4-data-zh
```
shell
```
shell
python test_quant.py
python test_quant.py
```
```
...
...
test_quant.py
View file @
19f50f89
...
@@ -28,7 +28,7 @@ def gptq():
...
@@ -28,7 +28,7 @@ def gptq():
model
=
AutoGPTQForCausalLM
.
from_pretrained
(
model_path
,
quantize_config
)
model
=
AutoGPTQForCausalLM
.
from_pretrained
(
model_path
,
quantize_config
)
# https://huggingface.co/datasets/llm-wizard/alpaca-gpt4-data-zh
# https://huggingface.co/datasets/llm-wizard/alpaca-gpt4-data-zh
file_path
=
'.
.
/alpaca_gpt4_data_zh.json'
file_path
=
'./alpaca_gpt4_data_zh.json'
# file_path = 'oaast_rm_zh.json'
# file_path = 'oaast_rm_zh.json'
messages
=
[]
messages
=
[]
with
open
(
file_path
,
'r'
,
encoding
=
'UTF-8'
)
as
fcc_file
:
with
open
(
file_path
,
'r'
,
encoding
=
'UTF-8'
)
as
fcc_file
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment