Skip to content

GitLab

Commit ffaaa259 authored by s4rduk4r's avatar s4rduk4r
Browse files

Rely on accelerate.dispatch_model() only

parent 91de3bfa
Show whitespace changes
Inline Side-by-side
Showing with 9 additions and 16 deletions
awq/models/base.py
View file @ ffaaa259
...@@ -168,13 +168,6 @@ class BaseAWQForCausalLM(nn.Module): ...@@ -168,13 +168,6 @@ class BaseAWQForCausalLM(nn.Module):
) )
# Dispath to devices # Dispath to devices
if max_memory is None:
# VRAM only
model = simple_dispatch_model(model, device_map)
if fuse_layers:
self.fuse_layers(model, quant_config)
else:
if fuse_layers: if fuse_layers:
self.fuse_layers(model, quant_config) self.fuse_layers(model, quant_config)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment