Commit 42c8f57d authored by Sakits's avatar Sakits
Browse files

update README & demo

parent f51f5864
...@@ -14,11 +14,14 @@ The current release supports: ...@@ -14,11 +14,14 @@ The current release supports:
## Contents ## Contents
- [Install](#install) - [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration \[Paper\]](#awq-activation-aware-weight-quantization-for-llm-compression-and-acceleration-paper)
- [AWQ Model Zoo](#awq-model-zoo) - [Contents](#contents)
- [Examples](#examples) - [Install](#install)
- [Usage](#usage) - [AWQ Model Zoo](#awq-model-zoo)
- [Reference](#reference) - [Examples](#examples)
- [Usage](#usage)
- [Reference](#reference)
- [Related Projects](#related-projects)
## Install ## Install
...@@ -81,7 +84,7 @@ python -m awq.entry --model_path /PATH/TO/OPT/opt-6.7b \ ...@@ -81,7 +84,7 @@ python -m awq.entry --model_path /PATH/TO/OPT/opt-6.7b \
--run_awq --dump_awq awq_cache/opt-6.7b-w4-g128.pt --run_awq --dump_awq awq_cache/opt-6.7b-w4-g128.pt
``` ```
2. Evaluate the AWQ quantize model on WikiText-2 (simulated pseudo quantization) 2. Evaluate the AWQ quantized model on WikiText-2 (simulated pseudo quantization)
```bash ```bash
python -m awq.entry --model_path /PATH/TO/OPT/opt-6.7b \ python -m awq.entry --model_path /PATH/TO/OPT/opt-6.7b \
--tasks wikitext \ --tasks wikitext \
......
...@@ -22,7 +22,7 @@ ...@@ -22,7 +22,7 @@
"metadata": {}, "metadata": {},
"source": [ "source": [
"In order to run this notebook, you need to install the following packages:\n", "In order to run this notebook, you need to install the following packages:\n",
"- [AWQ]()\n", "- [AWQ](https://github.com/mit-han-lab/llm-awq)\n",
"- [Pytorch](https://pytorch.org/)\n", "- [Pytorch](https://pytorch.org/)\n",
"- [Accelerate](https://github.com/huggingface/accelerate)\n", "- [Accelerate](https://github.com/huggingface/accelerate)\n",
"- [FastChat](https://github.com/lm-sys/FastChat)\n", "- [FastChat](https://github.com/lm-sys/FastChat)\n",
......
{ {
"cells": [ "cells": [
{ {
"attachments": {},
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
...@@ -8,6 +9,7 @@ ...@@ -8,6 +9,7 @@
] ]
}, },
{ {
"attachments": {},
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
...@@ -15,11 +17,12 @@ ...@@ -15,11 +17,12 @@
] ]
}, },
{ {
"attachments": {},
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"In order to run this notebook, you need to install the following packages:\n", "In order to run this notebook, you need to install the following packages:\n",
"- [AWQ]()\n", "- [AWQ](https://github.com/mit-han-lab/llm-awq)\n",
"- [Pytorch](https://pytorch.org/)\n", "- [Pytorch](https://pytorch.org/)\n",
"- [Accelerate](https://github.com/huggingface/accelerate)\n", "- [Accelerate](https://github.com/huggingface/accelerate)\n",
"- [LLaVA](https://github.com/haotian-liu/LLaVA)\n", "- [LLaVA](https://github.com/haotian-liu/LLaVA)\n",
...@@ -66,6 +69,7 @@ ...@@ -66,6 +69,7 @@
] ]
}, },
{ {
"attachments": {},
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
...@@ -104,6 +108,7 @@ ...@@ -104,6 +108,7 @@
] ]
}, },
{ {
"attachments": {},
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
...@@ -125,6 +130,7 @@ ...@@ -125,6 +130,7 @@
] ]
}, },
{ {
"attachments": {},
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment