GitLab

Documentation says default is 0.1, but the code has attention_dropout default at 0.0

To avoid import error if one doesn't have rotary_emb installed

Recasting query and key after rotary_emb()