• Leon Derczynski's avatar
    MobileBERT tokenizer tests (#16896) · 4a419d49
    Leon Derczynski authored
    
    
    * unhardcode pretrained model path, make it a class var
    
    * add tests for mobilebert tokenizer
    
    * allow tempfiles for vocab & merge similarity test to autodelete
    
    * add explanatory comments
    
    * remove unused imports, let make style do its.. thing
    
    * remove inheritance and use BERT tok tests for MobileBERT
    
    * Update tests/mobilebert/test_tokenization_mobilebert.py
    Co-authored-by: default avatarSaulLu <55560583+SaulLu@users.noreply.github.com>
    
    * amend class names, remove unused import, add fix for mobilebert's hub pathname
    
    * unhardcode pretrained model path, make it a class var
    
    * add tests for mobilebert tokenizer
    
    * allow tempfiles for vocab & merge similarity test to autodelete
    
    * add explanatory comments
    
    * remove unused imports, let make style do its.. thing
    
    * remove inheritance and use BERT tok tests for MobileBERT
    
    * Update tests/mobilebert/test_tokenization_mobilebert.py
    Co-authored-by: default avatarSaulLu <55560583+SaulLu@users.noreply.github.com>
    
    * amend class names, remove unused import, add fix for mobilebert's hub pathname
    
    * amend paths for model tests being in models/ subdir of /tests
    
    * explicitly rm test from prev path
    Co-authored-by: default avatarSaulLu <55560583+SaulLu@users.noreply.github.com>
    4a419d49
test_tokenization_mobilebert.py 14.1 KB