Skip to content

GitLab

Switch branch/tag
Find file BlameHistoryPermalink
  • Anthony MOI's avatar
    Various tokenizers fixes (#5558) · 5787e4c1
    Anthony MOI authored 
    * BertTokenizerFast - Do not specify strip_accents by default

* Bump tokenizers to new version

* Add test for AddedToken serialization
    5787e4c1
setup.py 6.11 KB