• Peng-Jen Chen's avatar
    Make learned positional embedding optional · 39264559
    Peng-Jen Chen authored
    Summary:
    - Add learned positional embedding binary flag to masked LM model.
    - Add base arch config for masked LM model which sets all the binary parameters to False. Otherwise some of the binary flag parameters will always be override by config in `xlm_architecture` (e.g. encoder_learned_pos)
    
    Reviewed By: liezl200
    
    Differential Revision: D15054487
    
    fbshipit-source-id: d78827f352b9160a89c9dc4f45b9fce15a2f234d
    39264559
transformer.py 43 KB