Skip to content

GitLab

Unverified Commit 3f9b7ab9 authored by Kyle Sayers's avatar Kyle Sayers Committed by GitHub
Browse files

[Doc] Update examples to remove SparseAutoModelForCausalLM (#12062) 


Signed-off-by: default avatarKyle Sayers <kylesayrs@gmail.com>
parent ad34c0df
Hide whitespace changes
Inline Side-by-side
Showing with 8 additions and 10 deletions
docs/source/features/quantization/fp8.md
View file @ 3f9b7ab9
......@@ -54,16 +54,15 @@ The quantization process involves three main steps:
### 1. Loading the Model
Use `SparseAutoModelForCausalLM`, which wraps `AutoModelForCausalLM`, for saving and loading quantized models:
Load your model and tokenizer using the standard `transformers` AutoModel classes:
```python
from llmcompressor.transformers import SparseAutoModelForCausalLM
from transformers import AutoTokenizer
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_ID = "meta-llama/Meta-Llama-3-8B-Instruct"
model = SparseAutoModelForCausalLM.from_pretrained(
MODEL_ID, device_map="auto", torch_dtype="auto")
model = AutoModelForCausalLM.from_pretrained(
MODEL_ID, device_map="auto", torch_dtype="auto",
)
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
```
......
docs/source/features/quantization/int8.md
View file @ 3f9b7ab9
......@@ -30,14 +30,13 @@ The quantization process involves four main steps:
### 1. Loading the Model
Use `SparseAutoModelForCausalLM`, which wraps `AutoModelForCausalLM`, for saving and loading quantized models:
Load your model and tokenizer using the standard `transformers` AutoModel classes:
```python
from llmcompressor.transformers import SparseAutoModelForCausalLM
from transformers import AutoTokenizer
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_ID = "meta-llama/Meta-Llama-3-8B-Instruct"
model = SparseAutoModelForCausalLM.from_pretrained(
model = AutoModelForCausalLM.from_pretrained(
MODEL_ID, device_map="auto", torch_dtype="auto",
)
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment