Unverified Commit 6a2a16e9 authored by cao lei's avatar cao lei Committed by GitHub
Browse files

fix typo (#974)

parent 5bf20196
...@@ -233,7 +233,7 @@ def attention_ref( ...@@ -233,7 +233,7 @@ def attention_ref(
window_size: (int, int), left and right window size window_size: (int, int), left and right window size
upcast: whether to cast all inputs to fp32, do all computation in fp32, then cast upcast: whether to cast all inputs to fp32, do all computation in fp32, then cast
output back to fp16/bf16. output back to fp16/bf16.
reorder_ops: whether to change the order of operations (scaling k instead of scaling k, etc.) reorder_ops: whether to change the order of operations (scaling k instead of scaling q, etc.)
without changing the math. This is to estimate the numerical error from operation without changing the math. This is to estimate the numerical error from operation
reordering. reordering.
Output: Output:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment