GitLab

* zero optim state_dict takes only_rank_0

* fix unit test

* [fx]add torchaudio test

* [fx]add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test

* [fx] add torchaudio test and test patches

* Delete ~

* [fx] add patches and patches test

* [fx] add patches and patches test

* [fx] fix patches

* [fx] fix rnn patches

* [fx] fix rnn patches

* [fx] fix rnn patches

* [fx] fix rnn patches

* [fx] merge upstream

* [fx] fix import errors

* impl nvme optimizer

* update cpu adam

* add unit test

* update hybrid adam

* update docstr

* add TODOs

* update CI

* fix CI

* fix CI

* fix CI path

* fix CI path

* fix CI path

* fix install tensornvme

* fix CI

* fix CI path

* fix CI env variables

* test CI

* test CI

* fix CI

* fix nvme optim __del__

* fix adam __del__

* fix nvme optim

* fix CI env variables

* fix nvme optim import

* test CI

* test CI

* fix CI

* [fx] added activation checkpointing annotation

* polish code

* polish code

[colotensor] add megatron initialization for gpt2

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx] refactor tracer to trace complete graph

* add comments and solve conflicts.

* [fx] tested the complete workflow for auto-parallel

* polish code

* polish code

* polish code

* process group supports getting ranks in group

* chunk mgr receives a process group

* update unit test

* fix unit tests

* [utils] integrated colotensor with lazy init context

* polish code

* polish code

* polish code

* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [fx] add balanced policy v2

* add unittest

* add test and fix bugs

* add functions back

* add comments

* [fx] added apex normalization to patched modules

* remove unused imports

* make it faster

* [hotfix] remove circle import