GitLab

Fix y and y_grad data vector load size for flash attention

Remove repeated definition in local scope

Use Flash-Attention-2 method to do output accumulation for infer/forward

Update in D0 shuffled loading to support bigger KPerBlock size

Add bias with shuffle for flash attention fwd

fix flash attention bwd output d0 grad

mqa/gqa inference

Grouped Query Attention/Multi Query Attention

fix mha bwd  IsSupportedArgument

Two tiny udpates

reinterpret_cast to const char* in dumpBufferToFile to be compatible with both const and non-const input pointers

Two tiny updates