GitLab

added across rank sync for checkpoint iteration laod, fixed type for timing, and validation iterations

hidden_size % attention_heads == 0 is handled above when dealing with kv_channels.

Adding check for decoder sequence length.