GitLab

Also merged in some changed from apex

Clean up licensing.

See merge request ADLR/megatron-lm!451

The LICENSE file says everything is 3-clause BSD, which is what we want,
but at some point the Apache license was added to the top of some files
and that proliferated. This commit removes the Apache license from any
files that we own the copyright to.

Also updates the copyright year and removes the unnessary coding=utf-8
line.

Update state_dict arguments for recent PyTorch versions.

See merge request ADLR/megatron-lm!432

Memory safety checks were incorrect for the tokens_to_generate=0 case

See merge request ADLR/megatron-lm!447

fixed grad scalar warning for bf16

See merge request ADLR/megatron-lm!442

fixed grad scalar warning so it only prints it for fp16

See merge request ADLR/megatron-lm!441

Timing levels

See merge request ADLR/megatron-lm!436

fix a bug for size mismatch

See merge request ADLR/megatron-lm!438

support for all mask in fused kernel + avoiding inplace operation in bwd pass

See merge request ADLR/megatron-lm!435

added a flag to be able to switch between pytorch and ring exchange p2p

See merge request ADLR/megatron-lm!434

Remove old merge tool.

See merge request ADLR/megatron-lm!433

Remove deprecated destination argument to state_dict functions and make all arguments keyword to avoid warnings.

Adding checks for total number of tokens to keep server from crashing

See merge request ADLR/megatron-lm!428

Implements the top_p decay and top_p bound parameters so from the Factual Sampling work

See merge request ADLR/megatron-lm!423

Distributed optimizer readme section.

See merge request ADLR/megatron-lm!429