Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
lm-evaluation-harness
Commits
1d8f782f
Commit
1d8f782f
authored
Jul 04, 2023
by
gk
Browse files
auto-gptq -> gptq and an import warning
parent
18d842ab
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
10 additions
and
4 deletions
+10
-4
README.md
README.md
+2
-2
lm_eval/models/huggingface.py
lm_eval/models/huggingface.py
+7
-1
setup.py
setup.py
+1
-1
No files found.
README.md
View file @
1d8f782f
...
@@ -44,10 +44,10 @@ To install additional multilingual tokenization and text segmentation packages,
...
@@ -44,10 +44,10 @@ To install additional multilingual tokenization and text segmentation packages,
pip
install
-e
".[multilingual]"
pip
install
-e
".[multilingual]"
```
```
To support loading GPTQ quantized models, install the package with the
`
auto-
gptq`
extra:
To support loading GPTQ quantized models, install the package with the
`gptq`
extra:
```
bash
```
bash
pip
install
-e
".[
auto-
gptq]"
pip
install
-e
".[gptq]"
```
```
## Basic Usage
## Basic Usage
...
...
lm_eval/models/huggingface.py
View file @
1d8f782f
...
@@ -168,7 +168,13 @@ class HFLM(LM):
...
@@ -168,7 +168,13 @@ class HFLM(LM):
**
model_kwargs
,
**
model_kwargs
,
)
)
else
:
else
:
from
auto_gptq
import
AutoGPTQForCausalLM
try
:
from
auto_gptq
import
AutoGPTQForCausalLM
except
ModuleNotFoundError
:
raise
Exception
(
"Tried to load auto_gptq, but auto-gptq is not installed "
,
"please install auto-gptq via pip install lm-eval[gptq] or pip install -e .[gptq]"
,
)
self
.
_model
=
AutoGPTQForCausalLM
.
from_quantized
(
self
.
_model
=
AutoGPTQForCausalLM
.
from_quantized
(
pretrained
,
pretrained
,
...
...
setup.py
View file @
1d8f782f
...
@@ -55,7 +55,7 @@ setuptools.setup(
...
@@ -55,7 +55,7 @@ setuptools.setup(
"promptsource"
:
[
"promptsource"
:
[
"promptsource @ git+https://github.com/bigscience-workshop/promptsource.git#egg=promptsource"
"promptsource @ git+https://github.com/bigscience-workshop/promptsource.git#egg=promptsource"
],
],
"
auto-
gptq"
:
[
"auto-gptq[triton] @ git+https://github.com/PanQiWei/AutoGPTQ"
],
"gptq"
:
[
"auto-gptq[triton] @ git+https://github.com/PanQiWei/AutoGPTQ"
],
"anthropic"
:
[
"anthropic"
],
"anthropic"
:
[
"anthropic"
],
},
},
)
)
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment