Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
tsoc
hg-misc-tools
Commits
06fe2294
Commit
06fe2294
authored
Jan 31, 2026
by
one
Browse files
Fix vortex attention interface by adding dropout parameters and updating function signature
parent
1516fed0
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
24 additions
and
7 deletions
+24
-7
evo2/vortex_fix.patch
evo2/vortex_fix.patch
+24
-7
No files found.
evo2/vortex_fix.patch
View file @
06fe2294
...
@@ -37,8 +37,8 @@
...
@@ -37,8 +37,8 @@
+ q,
+ q,
+ kv,
+ kv,
+ self.drop.p if self.training else 0.0,
+ self.drop.p if self.training else 0.0,
+
softmax_scale=None
,
+
causal=causal
,
+
causal=Fals
e,
+
softmax_scale=self.softmax_scal
e,
+ alibi_slopes=self.alibi_slopes,
+ alibi_slopes=self.alibi_slopes,
+ window_size=self.window_size,
+ window_size=self.window_size,
+ deterministic=self.deterministic,
+ deterministic=self.deterministic,
...
@@ -60,3 +60,20 @@
...
@@ -60,3 +60,20 @@
q,
q,
k,
k,
v,
v,
@@ -72,6 +72,9 @@
softcap,
return_softmax,
None,
+ False,
+ None,
+ 0.0,
)
return out, softmax_lse, S_dmask, rng_state
@@ -1624,5 +1627,6 @@
softcap,
rotary_interleaved,
num_splits,
+ None,
)
return (out, softmax_lse) if return_softmax_lse else out
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment