transformer_engine/pytorch/attention.py · efd4b62a0c3b32672e48db26736a8fd1ae07a8cd · OpenDAS / TransformerEngine

[PyTorch] Fix bug in micro batched inference with rotary embeddings (#536) · 37b3b7a7

Fabian Joswig authored Dec 15, 2023



[fix] fixed micro batched inference with RoPE
Signed-off-by: Fabian Joswig <fabian.joswig@deepl.com>
Co-authored-by: cyanguwa <8636796+cyanguwa@users.noreply.github.com>

37b3b7a7

attention.py 138 KB

Replace attention.py