Commits · e67dab92a995cb51b8280dbed668a0c9aea0dff7 · OpenDAS / ColossalAI

02 Dec, 2021 1 commit
- add some details in vit-b16 example (#43) (#44) · e67dab92
  ver217 authored Dec 02, 2021
  
  e67dab92
29 Nov, 2021 1 commit
- add explanation for ViT example (#35) (#36) · 2528adc6
  binmakeswell authored Nov 29, 2021
  
  2528adc6
18 Nov, 2021 2 commits

add an example of ViT-B/16 and remove w_norm clipping in LAMB (#29) · dbe62c67
ver217 authored Nov 18, 2021

dbe62c67

Support TP-compatible Torch AMP and Update trainer API (#27) · 3defa32a

Frank Lee authored Nov 18, 2021



* Add gradient accumulation, fix lr scheduler

* fix FP16 optimizer and adapted torch amp with tensor parallel (#18)

* fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes

* fixed trainer

* Revert "fixed trainer"

This reverts commit 2e0b0b76990e8d4e337add483d878c0f61cf5097.

* improved consistency between trainer, engine and schedule (#23)
Co-authored-by: 1SAA <c2h214748@gmail.com>
Co-authored-by: 1SAA <c2h214748@gmail.com>
Co-authored-by: ver217 <lhx0217@gmail.com>

3defa32a

28 Oct, 2021 1 commit
- Migrated project · 404ecbdc
  zbian authored Oct 28, 2021
  
  404ecbdc