tests/test_modeling_marian.py · 3487be75ef8f0d51c51208f266644fc04a947085 · chenpangpang / transformers

[Marian] documentation and AutoModel support (#4152) · 3487be75

Sam Shleifer authored May 10, 2020

- MarianSentencepieceTokenizer - > MarianTokenizer
- Start using unk token.
- add docs page
- add better generation params to MarianConfig
- more conversion utilities

3487be75

test_modeling_marian.py 8.32 KB

Replace test_modeling_marian.py