examples/language/llama2/pretrain.py · dd2c28a32352de45675ab13e72049a7f2a57e364 · OpenDAS / ColossalAI · GitLab

Find file Blame History Permalink

[shardformer]fix flash attention, when mask is casual, just don't unpad it (#5084) · aae49663
flybird11111 authored Nov 22, 2023
```
* fix flash attn

* fix

fix
```
aae49663

pretrain.py 12.9 KB