Skip to content

GitLab

Switch branch/tag
Find file BlameHistoryPermalink
  • lexhuismans's avatar
    [T5] Add 3D attention mask to T5 model (2) (#9643) (#11197) · 91cf2915
    lexhuismans authored 
    * Add 3D attention mask to T5 model (#9643)

Added code for 3D attention mask in T5 model. Similar to BERT model.

* Add test for 3D attention mask

Added test for 3D attention mask: test_decoder_model_past_with_3d_attn_mask()
3D attention mask of the shape [Batch_size, Seq_length, Seq_length] both for
attention mask and decoder attention mask. Test is passing.
    91cf2915
test_modeling_t5.py 55.1 KB