• Forrest Iandola's avatar
    SqueezeBERT architecture (#7083) · 02ef825b
    Forrest Iandola authored
    * configuration_squeezebert.py
    
    thin wrapper around bert tokenizer
    
    fix typos
    
    wip sb model code
    
    wip modeling_squeezebert.py. Next step is to get the multi-layer-output interface working
    
    set up squeezebert to use BertModelOutput when returning results.
    
    squeezebert documentation
    
    formatting
    
    allow head mask that is an array of [None, ..., None]
    
    docs
    
    docs cont'd
    
    path to vocab
    
    docs and pointers to cloud files (WIP)
    
    line length and indentation
    
    squeezebert model cards
    
    formatting of model cards
    
    untrack modeling_squeezebert_scratchpad.py
    
    update aws paths to vocab and config files
    
    get rid of stub of NSP code, and advise users to pretrain with mlm only
    
    fix rebase issues
    
    redo rebase of modeling_auto.py
    
    fix issues with code formatting
    
    more code format auto-fixes
    
    move squeezebert before bert in tokenization_auto.py and modeling_auto.py because squeezebert inherits from bert
    
    tests for squeezebert modeling and tokenization
    
    fix typo
    
    move squeezebert before bert in modeling_auto.py to fix inheritance problem
    
    disable test_head_masking, since squeezebert doesn't yet implement head masking
    
    fix issues exposed by the test_modeling_squeezebert.py
    
    fix an issue exposed by test_tokenization_squeezebert.py
    
    fix issue exposed by test_modeling_squeezebert.py
    
    auto generated code style improvement
    
    issue that we inherited from modeling_xxx.py: SqueezeBertForMaskedLM.forward() calls self.cls(), but there is no self.cls, and I think the goal was actually to call self.lm_head()
    
    update copyright
    
    resolve failing 'test_hidden_states_output' and remove unused encoder_hidden_states and encoder_attention_mask
    
    docs
    
    add integration test. rename squeezebert-mnli --> squeezebert/squeezebert-mnli
    
    autogenerated formatting tweaks
    
    integrate feedback from patrickvonplaten and sgugger to programming style and documentation strings
    
    * tiny change to order of imports
    02ef825b
README.md 25.3 KB