GitLab

Co-authored-by: github-actions <github-actions@github.com>

* [autoparallel] add non_split linear stategy

* polish

* [fx] metainfo class for auto parallel

* [fx] add unit test for linear metainfo

* [fx] fix bwd param for linear

* [fx] modify unit test

* [fx] modify unit test

* [fx] modify import

* [fx] modify import

* [fx] modify import

* [fx] move meta profiler to auto parallel

* [fx] add conv metainfo class

* [fx] restore profiler

* [fx] restore meta profiler

* [autoparallel] modify unit test

* [fx] modify unit test

* [autoparallel] add batchnorm metainfo class

* [autoparallel] fix batchnorm unit test function declaration

* [fx] restore profiler

* [fx] add relu metainfo class

* [fx] restore profiler

* [autoparallel] modify metainfo input

* [autoparallel] add pooling metainfo

* [autoparallel] add F.linear metainfo generator

* [autoparallel] add binary elementwise metainfo

* [fx] recover profiler

* [autoparallel] fix forward memory calculation

* [autoparallel] modify constants.py

* [autoparallel] remove redundant print

* [autoparallel] add F.conv metainfo

* [autoparallel] linear fix

* [autoparallel] complete gpt block searching

* fix test

* adapt scheduler for Topo

* remoove comment

* fix set input
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>

* [fx] metainfo class for auto parallel

* [fx] add unit test for linear metainfo

* [fx] fix bwd param for linear

* [fx] modify unit test

* [fx] modify unit test

* [fx] modify import

* [fx] modify import

* [fx] modify import

* [fx] move meta profiler to auto parallel

* [fx] add conv metainfo class

* [fx] restore profiler

* [fx] restore meta profiler

* [autoparallel] modify unit test

* [fx] modify unit test

* [autoparallel] add batchnorm metainfo class

* [autoparallel] fix batchnorm unit test function declaration

* [fx] restore profiler

* [fx] add relu metainfo class

* [fx] restore profiler

* [autoparallel] modify metainfo input

* [autoparallel] add pooling metainfo

* [autoparallel] add F.linear metainfo generator

* [autoparallel] add binary elementwise metainfo

* [fx] recover profiler

* [autoparallel] fix forward memory calculation

* [autoparallel] modify constants.py

* [autoparallel] remove redundant print

* use Topo class to rewrite DAG

* polish code

* polish code

* polish code

* add comment

* add else to unended if
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>

* [autoparallel] adapt solver with self attention

* polish code

* [cli] updated installation cheheck with more inforamtion

* polish code

* polish code

* [gemini] fix init bugs for modules

* fix bugs

* [setup] supported conda-installed torch

* polish code

* [zero] fix testing parameters

* [gemini] add arguments

* add docstrings

* [Gemini] more tests for Gemini

* polish code

* [testing] fix testing models

* roll back

* [zero] fix memory leak for zero2

* [zero] test gradient accumulation

* [zero] remove grad clip test

* add DAG to split_module

* add comment

* add test case for DAG

* remove print

* add DAG middleware in scheduler

* add test case for scheduler

* remove break

* recover old lifecycle
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>