tests/models/plbart/test_tokenization_plbart.py · 38bff8c84f81a7aec58ac1f0d6463de2584f4a52 · chenpangpang / transformers

"examples/pytorch/text-generation/run_generation.py" did not exist on "a75c64d80c76c3dc71f735d9197a4a601847e0cd"

Update Special Language Tokens for PLBART (#19980) · 149483b2

Jordan Clive authored Nov 21, 2022



* Update Special Language Tokens for PLBART

* fix format

* making mapping for language codes and updating tests:

* fix format

* fix consistency

* add assert to both tokenizer tests.

* fix format

* Update src/transformers/models/plbart/tokenization_plbart.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* improvin readability, setting self.tgt_lang

* fixing

* readability
Co-authored-by: jordiclive <jordiclive19@imperial.ac.uk>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

149483b2

test_tokenization_plbart.py 12.8 KB

Replace test_tokenization_plbart.py