src/transformers/tokenization_marian.py · 9a687ebb7750edee86047e0f243c332effdf86b6 · chenpangpang / transformers · GitLab

Find file Blame History Permalink

[Marian Fixes] prevent predicting pad_token_id before softmax, support... · 9a687ebb
Sam Shleifer authored May 13, 2020
```
[Marian Fixes] prevent predicting pad_token_id before softmax, support language codes, name multilingual models (#4290)
```
9a687ebb

tokenization_marian.py 7.19 KB