GitLab

The LICENSE file says everything is 3-clause BSD, which is what we want,
but at some point the Apache license was added to the top of some files
and that proliferated. This commit removes the Apache license from any
files that we own the copyright to.

Also updates the copyright year and removes the unnessary coding=utf-8
line.

Remove deprecated destination argument to state_dict functions and make all arguments keyword to avoid warnings.

remove redundant linear layer class definition

add fuse_gradient_accumulation attribute to weights for simple targetting

reflect feedback and clean up the codes

arg change

Adds some backward compatibility code so old inference code still works.

- Accumulate encoder hidden state gradient to handle skip connection
- Correctly compute the number of layers in encoder / decoder for T5 model
- Ensure e weights are initialized the same way in embeddings
- Synchronize embedding gradients across encoder and decoder for T5 model
- Support for checkpoint loading and saving