Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
bitsandbytes
Commits
b6383ba1
Commit
b6383ba1
authored
Mar 22, 2023
by
Ji Lin
Browse files
fix a bug in quantize_no_absmax and dequantize_no_absmax with multiple gpus
parent
49a04253
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
0 deletions
+4
-0
bitsandbytes/functional.py
bitsandbytes/functional.py
+4
-0
No files found.
bitsandbytes/functional.py
View file @
b6383ba1
...
@@ -656,9 +656,11 @@ def quantize_no_absmax(A: Tensor, code: Tensor, out: Tensor = None) -> Tensor:
...
@@ -656,9 +656,11 @@ def quantize_no_absmax(A: Tensor, code: Tensor, out: Tensor = None) -> Tensor:
torch.Tensor:
torch.Tensor:
Quantized 8-bit tensor.
Quantized 8-bit tensor.
'''
'''
prev_device
=
pre_call
(
A
.
device
)
if
out
is
None
:
out
=
torch
.
zeros_like
(
A
,
dtype
=
torch
.
uint8
)
if
out
is
None
:
out
=
torch
.
zeros_like
(
A
,
dtype
=
torch
.
uint8
)
is_on_gpu
([
A
,
out
])
is_on_gpu
([
A
,
out
])
lib
.
cquantize
(
get_ptr
(
code
),
get_ptr
(
A
),
get_ptr
(
out
),
ct
.
c_int
(
A
.
numel
()))
lib
.
cquantize
(
get_ptr
(
code
),
get_ptr
(
A
),
get_ptr
(
out
),
ct
.
c_int
(
A
.
numel
()))
post_call
(
prev_device
)
return
out
return
out
...
@@ -683,9 +685,11 @@ def dequantize_no_absmax(A: Tensor, code: Tensor, out: Tensor = None) -> Tensor:
...
@@ -683,9 +685,11 @@ def dequantize_no_absmax(A: Tensor, code: Tensor, out: Tensor = None) -> Tensor:
torch.Tensor:
torch.Tensor:
32-bit output tensor.
32-bit output tensor.
'''
'''
prev_device
=
pre_call
(
A
.
device
)
if
out
is
None
:
out
=
torch
.
zeros_like
(
A
,
dtype
=
torch
.
float32
)
if
out
is
None
:
out
=
torch
.
zeros_like
(
A
,
dtype
=
torch
.
float32
)
is_on_gpu
([
code
,
A
,
out
])
is_on_gpu
([
code
,
A
,
out
])
lib
.
cdequantize
(
get_ptr
(
code
),
get_ptr
(
A
),
get_ptr
(
out
),
ct
.
c_int
(
A
.
numel
()))
lib
.
cdequantize
(
get_ptr
(
code
),
get_ptr
(
A
),
get_ptr
(
out
),
ct
.
c_int
(
A
.
numel
()))
post_call
(
prev_device
)
return
out
return
out
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment