Commits · 3487be75ef8f0d51c51208f266644fc04a947085 · chenpangpang / transformers

10 May, 2020 1 commit

[Marian] documentation and AutoModel support (#4152) · 3487be75

Sam Shleifer authored May 10, 2020

- MarianSentencepieceTokenizer - > MarianTokenizer
- Start using unk token.
- add docs page
- add better generation params to MarianConfig
- more conversion utilities

3487be75