GitLab

* [conda] fixed colossalai dependency version

* polish code

Co-authored-by: zhuww <zhuww@sugon.com>

Co-authored-by: zhuww <zhuww@sugon.com>

* optimize code for bwd performance and refine code.

* refine README to add test SW version

* fix sampler of ddp and add dap

* add end to end unit test for training

* modify unit test datapath

* lower the bar for cuda kernel

* skip unit test of end-to-end train on 3080

* update to alphafold 2.3 dataset

* fix uniprot dataset

* add habana

* add mask

* fix mask in outer_product_mean

* add dap

* add hmp

* merge training code

* add chunk for inference

* fix extra-msa stack for training

* support ddp in training

* fix inference bugs

* code refactoring for habana

* support hmp training

* enable all inference and train on Gaudi/Gaudi2 with optimized perf with latest base (#139)

* enable all inference and train on Gaudi/Gaudi2 with optimized perf

* refine code to adapt new base

* refine code to fix issues in code review
Co-authored-by: habanachina <habanachina@habana.ai>
Co-authored-by: Leo Zhao <48052473+LeoZhao-Habana@users.noreply.github.com>
Co-authored-by: habanachina <habanachina@habana.ai>

* add metrics for training

* modify log loss format

- Added argument and save by default
- Include all tags in output file for multimer preset

* support training, fix some inplace func of nn

* fix some merge issue
Co-authored-by: shenggan <csg19971016@gmail.com>

* add triton flash attention

* add fallfack for flash attention

* add pytest skip mask for attention kernel

* fix multimer bug

* fix bug

* remove print

* fix index

* fix layernorm length

* update test

* improve template code

* remove print

* polish layernorm

* fix multimer communication in evoformer

* Update cuda version in readme

* Update README.md

* Update README.md

* Update README.md

* fix multimer bug

* fix bug

* remove print

* fix index

* fix layernorm length

* update test

* improve template code

* remove print

* polish layernorm

* update readme and setup, not finished

* update readme and dockerfile
Co-authored-by: lc_pro <gyang_lu@foxmail.com>

* Move template modification from nn to fastnn (#97)

* update out product mean test

* polish out product mean test and update linear test

* update test for layernorm, use both kernel

* support test without triton

* polish layernorm

* use current code

* move template modification to fastnn, restore template in nn

* re-organize fastnn

* update evoformer unit test

* Align results and update unit tests (#98)

* update out product mean test

* polish out product mean test and update linear test

* update test for layernorm, use both kernel

* support test without triton

* polish layernorm

* use current code

* move template modification to fastnn, restore template in nn

* re-organize fastnn

* update evoformer unit test

* update evoformer stack test

* update test

* update msa_att_row

* update msa_att_col

* update evoformer and evo-stack

* update evoformer

* update extramsa

* move model loading out of the loop

* finish template test

* update test

* Move template modification from nn to fastnn (#84)

* update out product mean test

* polish out product mean test and update linear test

* update test for layernorm, use both kernel

* support test without triton

* polish layernorm

* use current code

* move template modification to fastnn, restore template in nn

* re-organize fastnn

* update evoformer unit test

* move model out of function

* only test inference

* remove cache in build

* update test inference

* restore changes

* restore build changes

* update inference and evoformer stack

* fix some bug

* update test

* update evoformer stack test

* update test

* update test

* fix test

* update test

* update test

* update input embedder

* update embedder

* reset core

* update test

* support template multimer in inject_nn

* modify requirements, fix CI container

* fix format error

* remove torch1.11 kernel cache

* add cache back

* add CI, update Dockerfile

* remove useless loop in inference, add some comments to Attention

* update inference test and CI

* fix path

* add pytest for test env

* add einops install

* add cuda cache, loose the condition

* fixed bugs in softmax kernel and update unittest for softmax

* remove redundancy mask for softmax grad

* test both cuda/triton kernel in unittest

* update out product mean test

* polish out product mean test and update linear test

* update test for layernorm, use both kernel

* support test without triton

* polish layernorm

* use current code

Can get stable results now

* Update environment.yml

torch==1.11.1+cu113 is not supported in pip channel now

* update other version

* add torch version check

if torch version > 1.11.1, turn on tf32