GitLab

https://github.com/laekov/fastmoe/issues/63

* Added default weight initializations to FMoELinear and NoisyGate

* Following torch's naming convention

the old impl raised error "too many values to unpack (expected 1)"

* use single variable instead of vector in c functions

* expert count kernel

* remove all lists

* fix old tests

- mask some tensors of tokens for fmoe forward
- pass a list of expert classes to specify what experts in what order want to use

Bias now being calculated directly in MOELinear layer. Added corresponding CUDA changes. Updated forward and backward functions of MOELinear