"megatron/model/gpt_model.py" did not exist on "456f17280fcc25eb6bb3d9de7f9cad170b7b98d9"
  • Suraj Patil's avatar
    support new marian models (#15831) · ba21001f
    Suraj Patil authored
    * support not sharing embeddings
    
    * update modeling
    
    * update tokenizer
    
    * fix conversion script
    
    * always use self.shared
    
    * boom boom
    
    * begin tests
    
    * update tests
    
    * fix resize_decoder_token_embeddings
    
    * address Patrick's comments
    
    * style
    
    * update conversion script
    
    * fix conversion script
    
    * fix tokenizer
    
    * better name target vocab
    
    * add integration test for tokenizer with two vocabs
    
    * style
    
    * address Patrick's comments
    
    * add integration test for model
    ba21001f
test_tokenization_marian.py 8.46 KB