GitLab

* Fix gradient accumulation

Add ``is_scaled_loss`` flag to support both scaled / unscaled loss
Fix ``test_grad_accum`` and``test_set_num_gradients_to_accumulate``

* Add a method to scale grad for grad_accum using unscaled loss

- Revert the changes in `step` method
- Add a method `scale_grad_by_num_grads_to_accum`to handle gradient accumulation using unscaled loss more explicitly
- Add gradient tests

* Implement _compute_corr_mean_between_grads

* Improve tests and comments

* Use ubuntu-20.04 instead of latest

Use ubuntu-20.04 to fix the `arch x64 not found` issue
[Version 3.10 with arch x64 not found actions/setup-python#401](https://github.com/actions/setup-python/issues/401)

* Switch flake8 from gitlab to github

Flake8 was moved to Github
See discussions https://www.reddit.com/r/Python/comments/yvfww8/flake8_took_down_the_gitlab_repository_in_favor/

* Fix scikit-learn package

* Update PyTorch versions

* Resolve comments from Min

* Minor fix

* Disable broken tests for new versions of PyTorch

* [chore] move to 3.8/3.9/3.10

* move to 1.11 for stable version

* fixed version check

* fixed cache file names

* update the precommit flow too
Co-authored-by: Min Xu <min.xu.public@gmail.com>

* [skip ci] first commit to automate release process

* empty commit

* fix syntax

* fix next_version value

* fixing more syntax

* remove uses

* fix

* fixed path in setup.py

* trying a basic example

* adding branch

* change release to name

* adding first step

* remove push trigger

* change order in ON section

* modifying manual workflow

* adding fairscale release workflow

* removing unused workflows

* replacing values with secrets

* fixing __version__ in __init__.py

* cleanup

* restoring import statement

* adding a manual workflow

* add push

* fix syntax

* adding a new workflow

* renaming file

* cleanup yaml files

* [skip ci] removing pyproject edits

* adding pre-commit files

* applying pre-commit to all files

* adding no-strict-optional argument to mypy in circle ci config

* fix typo

* updating python versions

* [skip ci] remove extra args

* adding python 3.9

* [skip ci] set pre-commit version in requirements-dev.txt

* set CACHE_VERSION

* move linters from circleci to github actions

* update python version

* update python version in benchmarks_2

* moving to python 3.9.7

* v0.1.6

* precise skip, only if agent has only cpu

* adding fairseq's gha
* adding py3.9, removing submodules
* yaml linting