GitLab

Decreased moe tests;

Added FFNExperts and ViTMoE model

fixed padding index issue for vocab parallel embedding layers; updated 3D linear to be compatible with examples in the tutorial

added branch context;
added vocab parallel layers;
moved split_batch from load_batch to tensor parallel embedding layers;
updated gpt model;
updated unit test cases;
fixed few collective communicator bugs

Co-authored-by: github-actions <github-actions@github.com>

* Update GitHub action and pre-commit settings

* Update GitHub action and pre-commit settings (#198)

* add pytorch hooks
fix #175

* remove licenses in src code

* add gpu memory tracer

* replacing print with logger in ophooks.

* added examples as submodule

* update submodule url

* remove examples folder

* added examples as submodule

* update .gitmodules

* remove benchmark folder

* added benchmark submodule

* update .gitmodules