Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
AutoAWQ
Commits
42c8f57d
Commit
42c8f57d
authored
Jun 02, 2023
by
Sakits
Browse files
update README & demo
parent
f51f5864
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
17 additions
and
8 deletions
+17
-8
README.md
README.md
+9
-6
examples/chat_demo.ipynb
examples/chat_demo.ipynb
+1
-1
examples/llava_demo.ipynb
examples/llava_demo.ipynb
+7
-1
No files found.
README.md
View file @
42c8f57d
...
@@ -14,11 +14,14 @@ The current release supports:
...
@@ -14,11 +14,14 @@ The current release supports:
## Contents
## Contents
-
[
Install
](
#install
)
-
[
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration \[Paper\]
](
#awq-activation-aware-weight-quantization-for-llm-compression-and-acceleration-paper
)
-
[
AWQ Model Zoo
](
#awq-model-zoo
)
-
[
Contents
](
#contents
)
-
[
Examples
](
#examples
)
-
[
Install
](
#install
)
-
[
Usage
](
#usage
)
-
[
AWQ Model Zoo
](
#awq-model-zoo
)
-
[
Reference
](
#reference
)
-
[
Examples
](
#examples
)
-
[
Usage
](
#usage
)
-
[
Reference
](
#reference
)
-
[
Related Projects
](
#related-projects
)
## Install
## Install
...
@@ -81,7 +84,7 @@ python -m awq.entry --model_path /PATH/TO/OPT/opt-6.7b \
...
@@ -81,7 +84,7 @@ python -m awq.entry --model_path /PATH/TO/OPT/opt-6.7b \
--run_awq
--dump_awq
awq_cache/opt-6.7b-w4-g128.pt
--run_awq
--dump_awq
awq_cache/opt-6.7b-w4-g128.pt
```
```
2.
Evaluate the AWQ quantize model on WikiText-2 (simulated pseudo quantization)
2.
Evaluate the AWQ quantize
d
model on WikiText-2 (simulated pseudo quantization)
```
bash
```
bash
python
-m
awq.entry
--model_path
/PATH/TO/OPT/opt-6.7b
\
python
-m
awq.entry
--model_path
/PATH/TO/OPT/opt-6.7b
\
--tasks
wikitext
\
--tasks
wikitext
\
...
...
examples/chat_demo.ipynb
View file @
42c8f57d
...
@@ -22,7 +22,7 @@
...
@@ -22,7 +22,7 @@
"metadata": {},
"metadata": {},
"source": [
"source": [
"In order to run this notebook, you need to install the following packages:\n",
"In order to run this notebook, you need to install the following packages:\n",
"- [AWQ]()\n",
"- [AWQ](
https://github.com/mit-han-lab/llm-awq
)\n",
"- [Pytorch](https://pytorch.org/)\n",
"- [Pytorch](https://pytorch.org/)\n",
"- [Accelerate](https://github.com/huggingface/accelerate)\n",
"- [Accelerate](https://github.com/huggingface/accelerate)\n",
"- [FastChat](https://github.com/lm-sys/FastChat)\n",
"- [FastChat](https://github.com/lm-sys/FastChat)\n",
...
...
examples/llava_demo.ipynb
View file @
42c8f57d
{
{
"cells": [
"cells": [
{
{
"attachments": {},
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {},
"metadata": {},
"source": [
"source": [
...
@@ -8,6 +9,7 @@
...
@@ -8,6 +9,7 @@
]
]
},
},
{
{
"attachments": {},
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {},
"metadata": {},
"source": [
"source": [
...
@@ -15,11 +17,12 @@
...
@@ -15,11 +17,12 @@
]
]
},
},
{
{
"attachments": {},
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {},
"metadata": {},
"source": [
"source": [
"In order to run this notebook, you need to install the following packages:\n",
"In order to run this notebook, you need to install the following packages:\n",
"- [AWQ]()\n",
"- [AWQ](
https://github.com/mit-han-lab/llm-awq
)\n",
"- [Pytorch](https://pytorch.org/)\n",
"- [Pytorch](https://pytorch.org/)\n",
"- [Accelerate](https://github.com/huggingface/accelerate)\n",
"- [Accelerate](https://github.com/huggingface/accelerate)\n",
"- [LLaVA](https://github.com/haotian-liu/LLaVA)\n",
"- [LLaVA](https://github.com/haotian-liu/LLaVA)\n",
...
@@ -66,6 +69,7 @@
...
@@ -66,6 +69,7 @@
]
]
},
},
{
{
"attachments": {},
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {},
"metadata": {},
"source": [
"source": [
...
@@ -104,6 +108,7 @@
...
@@ -104,6 +108,7 @@
]
]
},
},
{
{
"attachments": {},
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {},
"metadata": {},
"source": [
"source": [
...
@@ -125,6 +130,7 @@
...
@@ -125,6 +130,7 @@
]
]
},
},
{
{
"attachments": {},
"cell_type": "markdown",
"cell_type": "markdown",
"metadata": {},
"metadata": {},
"source": [
"source": [
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment