-
Jared Casper authored
Perform distributed optimizer's all-gather in param dtype (instead of grad dtype) See merge request ADLR/megatron-lm!448
fc7f4f03
Perform distributed optimizer's all-gather in param dtype (instead of grad dtype) See merge request ADLR/megatron-lm!448