GitLab

hidden_size % attention_heads == 0 is handled above when dealing with kv_channels.

Adding check for decoder sequence length.

Fix deadlock when get_num_microbatches() < pipeline-parallel size (don't try to measure pipeline stall)