GitLab

remove completed sentences from batch and allow batching uneven lengths (with fixes to make padded sequences work correctly in all models)

This reverts commit b2e119c209363e6ff6d2878a69c7d1a507a2e9be.

Improve dataloader speed and deprecate concept of batch_offset (use --sample-without-replacement instead)

- process only last time step during generation
- cache keys and values
- dont apply masking during generation