Skip to content

GitLab

"qa/L0_jax_distributed_unittest/test.sh" did not exist on "b48403861b651572bdd39e4a17c24f5c0930370a"
Switch branch/tag
Find file BlameHistoryPermalink
  • zhujian's avatar
    [PyTorch] Support FA3 for MLA and with CP (#1907) · c334fc46
    zhujian authored 
    

feature(FA3,MLA,CP):
1. Update FA3 to commit-id 3ba6f82 (tag 2.8.0.post2 with compile error fixed), PR-1604 support hdimQK != hdimV backward
2. Update get_attention_backend method because FA3 support MLA now
3. Add CP MLA support for FA3
4. Add unit tests for FA3 MLA CP
5. Update attention doc
Signed-off-by: default avatarzhujian <zhujian.whu.cs@gmail.com>
    c334fc46
attention.ipynb 38 KB