GitLab

[autockpt] provide option for activation checkpoint search in SPMD solver

* [autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline

* [autoparallel] using fwd_time and bwd_time instead of fwd_flop and bwd_flop

* [autoparallel] specifycomm nodes' memory cost in construct chain

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] bypass metainfo when available and modify BCAST_FUNC_OP

* [autockpt] make it work.

* [autockpt] linearize / merge shape-consistency nodes.

* [autockpt] considering parameter and optimizer weights.

* [hotfix] pass a parameter.

* add benchmark

* merge common func

* add total and avg tflops
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>

* [autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline

* [autoparallel] using fwd_time and bwd_time instead of fwd_flop and bwd_flop

* [autoparallel] specifycomm nodes' memory cost in construct chain

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] fix wrong runtime apply calculation

* [autoparallel] fix wrong runtime apply calculation

* [autockpt] make it work.

* [autockpt] linearize / merge shape-consistency nodes.

* [autockpt] considering parameter and optimizer weights.

Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>

* [autockpt] make it work.

* [autockpt] linearize / merge shape-consistency nodes.

* [autoparallel] gpt2 autoparallel examples

* polish code

* polish code

* move to cpu to avoid dead lock

* get output by offsets
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>

* [autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline

* [autoparallel] using fwd_time and bwd_time instead of fwd_flop and bwd_flop

* [autoparallel] specifycomm nodes' memory cost in construct chain

[autoparallel] align the data_ptr with the old version of auto activation checkpoint pipeline (#2261)

* [autoparallel] hook node meta on graph nodes for checkpoint solver

* [autoparallel] polish code

* [autoparallel] restore some node handlers

* colossalai/auto_parallel/passes/meta_info_prop.py

* [autoparallel] remove some unused import

* [autoparallel] hook bwd_mem_out

* [autoparallel] patch torch.flatten

* [doc] updated the stable diffussion on docker usage

* polish doc

* [builder] builder for scaled_upper_triang_masked_softmax

* add missing files

* fix a bug

* polish code

* [example] diffusion install from docker

Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>