GitLab

* fix ema

* add assert

---------
Co-authored-by: jixh <jixh@dp.tech>

* change to torchrun

* change ubuntu 20.04

* change format

* update

* fix build error

* use latest

* fall back

* upgrade nvcr

* fallback

* fall back cu117

* add example

* remove duplicate cmd

* fix urllib

---------
Co-authored-by: jixh <jixh@dp.tech>
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>

* add wandb support

* code clean

allow --keep-last-checkpoints == 0
Co-authored-by: zhougm <zhougm@dp.com>

* fix log

* fix log

* valid with ema

* save ema model

* update ema

* rm model_fp32

* rm ordered names

* load ema

* emapy

* Update unicore/trainer.py
Co-authored-by: Guolin Ke <guolin.ke@outlook.com>

* Update docker.yml

* py version >= 3.7

* rename docker image

* update docker file

* support softmax for large columns

* more tests

flatten memories

* try

* fix bugs

* code clean

* support mask in softmax

* code clean

* check for shapes