GitLab

Ds v0.9.2 rocm

See merge request dcutoolkit/deeplearing/deepspeed!3

Ds v0.9.2 rocm

See merge request dcutoolkit/deeplearing/deepspeed!2

Ds v0.8.2 rocm

See merge request aicomponent/deepspeed!3

modify error

See merge request aicomponent/deepspeed!2

Ds v0.8.2 rocm, support torch1.13 for the hipify change

See merge request aicomponent/deepspeed!1

Merge branch 'deepspeed-0.6.3-rocm' of http://10.0.100.3/dcutoolkit/deeplearing/deepspeed into deepspeed-0.6.3-rocm
version modify

Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

This test has been giving us trouble for a bit, seeing nondeterministic failures, skipping for now to not break out CI. Need to revisit soon though.

* zero.Init() clarification

clarify that if `model.half()` can't fit into gpu memory `zero.Init()` is a must.

this proposal is via @samyam's clarification shared elsewhere.

Thank you.

* style

* add clarity

* style
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

Bumps [kramdown](https://github.com/gettalong/kramdown) from 2.3.0 to 2.3.1.
- [Release notes](https://github.com/gettalong/kramdown/releases)
- [Changelog](https://github.com/gettalong/kramdown/blob/master/doc/news.page)
- [Commits](https://github.com/gettalong/kramdown/commits

)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

security alert related to older kramdown version

* Fix zero stage2 cpu_offload when some model trainable parameters skipped in training, as in https://github.com/microsoft/DeepSpeed/issues/707



As some model trainable parameters skipped in training,
their backward hooks in self.create_reduce_and_remove_grad_hooks() will not run, 
so they have no norm_for_param_grads

* Trim space

* Trim space
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>