flash_attn/models/gpt.py · 25d6b1dbcb60d9a07a0092f87663ded87be0138e · gaoqiong / flash-attention · GitLab

Find file Blame History Permalink

handle uneven heads across ranks when combining state_dicts; resolves #467 (#468) · 25d6b1db
Xuechen Li authored Aug 20, 2023
```
* q

* add comment.
```
25d6b1db

gpt.py 45 KB