Skip to content

GitLab

Unverified Commit 922732b2 authored by Daniël de Kok's avatar Daniël de Kok Committed by GitHub
Browse files

Install Marlin from standalone package (#2320)

parent 583d37a2
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
server/text_generation_server/layers/marlin/gptq.py
View file @ 922732b2
......@@ -223,6 +223,7 @@ class GPTQMarlinLinear(nn.Module):
A_flat.shape[1],
self.is_full_k,
self.qzeros.numel() > 0,
True,
)
C = C.reshape(A.shape[:-1] + (self.scales.shape[1],))
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment