Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
bitsandbytes
Commits
916000c8
Commit
916000c8
authored
Mar 22, 2023
by
Phil Wang
Browse files
fix consistent tabs / spaces
parent
aa9b939e
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
3 deletions
+3
-3
csrc/kernels.cu
csrc/kernels.cu
+3
-3
No files found.
csrc/kernels.cu
View file @
916000c8
...
@@ -1708,9 +1708,9 @@ kOptimizerStatic8bit1StateBlockwise(T* p, T* __restrict__ const g, unsigned char
...
@@ -1708,9 +1708,9 @@ kOptimizerStatic8bit1StateBlockwise(T* p, T* __restrict__ const g, unsigned char
s1_vals
[
j
]
=
(
s1_vals
[
j
]
*
beta1
)
+
g_val
;
s1_vals
[
j
]
=
(
s1_vals
[
j
]
*
beta1
)
+
g_val
;
break
;
break
;
case
LION
:
case
LION
:
// here, using gvals[j] to store the gradient smoothed by beta1
// here, using gvals[j] to store the gradient smoothed by beta1
// then update the momentum state1, to make sure the order is correct
// then update the momentum state1, to make sure the order is correct
g_vals
[
j
]
=
lr
*
sgn
(((
float
)
s1_vals
[
j
])
*
beta1
+
((
1.0
f
-
beta1
)
*
g_val
));
g_vals
[
j
]
=
lr
*
sgn
(((
float
)
s1_vals
[
j
])
*
beta1
+
((
1.0
f
-
beta1
)
*
g_val
));
s1_vals
[
j
]
=
s1_vals
[
j
]
*
beta2
+
((
1.0
f
-
beta2
)
*
g_val
);
s1_vals
[
j
]
=
s1_vals
[
j
]
*
beta2
+
((
1.0
f
-
beta2
)
*
g_val
);
break
;
break
;
case
RMSPROP
:
case
RMSPROP
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment