"tests/auto/test_modeling_auto.py" did not exist on "00aa9dbca29dcf0e3a624354ef5c80a8e5226339"
  • Anthony MOI's avatar
    Various tokenizers fixes (#5558) · 5787e4c1
    Anthony MOI authored
    * BertTokenizerFast - Do not specify strip_accents by default
    
    * Bump tokenizers to new version
    
    * Add test for AddedToken serialization
    5787e4c1
test_tokenization_common.py 75.2 KB