Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
AutoAWQ
Commits
d6f623fa
Commit
d6f623fa
authored
Jul 24, 2023
by
Haotian Tang
Browse files
[Minor] Update scripts.
parent
4f3e977c
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
tinychat/scripts/llama2_demo.sh
tinychat/scripts/llama2_demo.sh
+2
-2
No files found.
tinychat/scripts/llama2_demo.sh
View file @
d6f623fa
...
...
@@ -2,13 +2,13 @@ MODEL_PATH=/data/llm/checkpoints/llama2-hf
MODEL_NAME
=
llama-2-7b-chat
# # Perform AWQ search and save search results (we already did it for you):
# mkdir awq_cache
# mkdir
-p
awq_cache
# python -m awq.entry --model_path $MODEL_PATH/$MODEL_NAME \
# --w_bit 4 --q_group_size 128 \
# --run_awq --dump_awq awq_cache/llama-2-7b-chat-w4-g128.pt
# Generate real quantized weights (INT4):
mkdir
quant_cache
mkdir
-p
quant_cache
python
-m
awq.entry
--model_path
$MODEL_PATH
/
$MODEL_NAME
\
--w_bit
4
--q_group_size
128
\
--load_awq
awq_cache/llama-2-7b-chat-w4-g128.pt
\
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment