Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
d6da8a8f
Unverified
Commit
d6da8a8f
authored
Apr 28, 2025
by
Richard Barnes
Committed by
GitHub
Apr 28, 2025
Browse files
[Bugfix] Fix `numel()` downcast in fused_layernorm_dynamic_per_token_quant.cu (#17316)
parent
b4ac4fa0
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
csrc/quantization/fused_kernels/fused_layernorm_dynamic_per_token_quant.cu
.../fused_kernels/fused_layernorm_dynamic_per_token_quant.cu
+1
-1
No files found.
csrc/quantization/fused_kernels/fused_layernorm_dynamic_per_token_quant.cu
View file @
d6da8a8f
...
...
@@ -96,7 +96,7 @@ void rms_norm_dynamic_per_token_quant_dispatch(
std
::
optional
<
at
::
Tensor
>
const
&
scale_ub
,
std
::
optional
<
at
::
Tensor
>&
residual
)
{
int32_t
hidden_size
=
input
.
size
(
-
1
);
int32_t
num_tokens
=
input
.
numel
()
/
hidden_size
;
auto
num_tokens
=
input
.
numel
()
/
hidden_size
;
dim3
grid
(
num_tokens
);
dim3
block
(
std
::
min
(
hidden_size
,
1024
));
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment