You need to sign in or sign up before continuing.
Unverified Commit 39dd8471 authored by kaixuanliu's avatar kaixuanliu Committed by GitHub
Browse files

for intel xpu case, use MatMul8bitFp even not use ipex (#1728)



* for intel xpu case, use MatMul8bitFp even not use ipex
Signed-off-by: default avatarLiu, Kaixuan <kaixuan.liu@intel.com>

* fix lint issue
Signed-off-by: default avatarLiu, Kaixuan <kaixuan.liu@intel.com>

---------
Signed-off-by: default avatarLiu, Kaixuan <kaixuan.liu@intel.com>
parent a09d05a0
...@@ -8,7 +8,7 @@ import torch ...@@ -8,7 +8,7 @@ import torch
from typing_extensions import deprecated from typing_extensions import deprecated
import bitsandbytes.functional as F import bitsandbytes.functional as F
from bitsandbytes.functional import ipex_cpu, ipex_xpu from bitsandbytes.functional import ipex_cpu
# The inverse transformation for the colTuring and colAmpere format were contributed by Alex Borzunov: # The inverse transformation for the colTuring and colAmpere format were contributed by Alex Borzunov:
# https://github.com/bigscience-workshop/petals/blob/main/src/petals/utils/linear8bitlt_patch.py # https://github.com/bigscience-workshop/petals/blob/main/src/petals/utils/linear8bitlt_patch.py
...@@ -426,7 +426,7 @@ def matmul( ...@@ -426,7 +426,7 @@ def matmul(
state.threshold = threshold state.threshold = threshold
# MatMul8bitLt is slower because no fast kernel for quant/dequant 8bit in CPU/XPU # MatMul8bitLt is slower because no fast kernel for quant/dequant 8bit in CPU/XPU
if state.is_training: if state.is_training:
if (A.device.type == "cpu" and ipex_cpu) or (A.device.type == "xpu" and ipex_xpu): if (A.device.type == "cpu" and ipex_cpu) or (A.device.type == "xpu"):
return MatMul8bitFp.apply(A, B, out, bias, state) return MatMul8bitFp.apply(A, B, out, bias, state)
return MatMul8bitLt.apply(A, B, out, bias, state) return MatMul8bitLt.apply(A, B, out, bias, state)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment