tests/models/plbart/test_tokenization_plbart.py · 149483b25297236fb3e82db1a108c9e4f1ee42b5 · chenpangpang / transformers

"Doc/vscode:/vscode.git/clone" did not exist on "5ace2971c5a6fcbd39a038d46c3f61136a21bf46"

Update Special Language Tokens for PLBART (#19980) · 149483b2

Jordan Clive authored Nov 21, 2022



* Update Special Language Tokens for PLBART

* fix format

* making mapping for language codes and updating tests:

* fix format

* fix consistency

* add assert to both tokenizer tests.

* fix format

* Update src/transformers/models/plbart/tokenization_plbart.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* improvin readability, setting self.tgt_lang

* fixing

* readability
Co-authored-by: jordiclive <jordiclive19@imperial.ac.uk>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

149483b2

test_tokenization_plbart.py 12.8 KB

Replace test_tokenization_plbart.py