Pytorch RoBERTa

docs/source/model_doc/roberta.rst

 RoBERTa
 ----------------------------------------------------
 
-``RobertaConfig``
+The RoBERTa model was proposed in `RoBERTa: A Robustly Optimized BERT Pretraining Approach`_
+by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer,
+Veselin Stoyanov. It is based on Google's BERT model released in 2018.
+
+It builds on BERT and modifies key hyperparameters, removing the next-sentence pretraining
+objective and training with much larger mini-batches and learning rates.
+
+This implementation is the same as BertModel with a tiny embeddings tweak as well as a setup for Roberta pretrained
+models.
+
+RobertaConfig
 ~~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.RobertaConfig
     :members:
 
 
-``RobertaTokenizer``
+RobertaTokenizer
 ~~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.RobertaTokenizer
     :members:
 
 
-``RobertaModel``
+RobertaModel
 ~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.RobertaModel
     :members:
 
 
-``RobertaForMaskedLM``
+RobertaForMaskedLM
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.RobertaForMaskedLM
     :members:
 
 
-``RobertaForSequenceClassification``
+RobertaForSequenceClassification
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.RobertaForSequenceClassification
@@ -42,21 +52,21 @@ RobertaForTokenClassification
 .. autoclass:: transformers.RobertaForTokenClassification
     :members:
 
-``TFRobertaModel``
+TFRobertaModel
 ~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.TFRobertaModel
     :members:
 
 
-``TFRobertaForMaskedLM``
+TFRobertaForMaskedLM
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.TFRobertaForMaskedLM
     :members:
 
 
-``TFRobertaForSequenceClassification``
+TFRobertaForSequenceClassification
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. autoclass:: transformers.TFRobertaForSequenceClassification

src/transformers/modeling_bert.py

@@ -592,7 +592,7 @@ BERT_INPUTS_DOCSTRING = r"""
             Mask to nullify selected heads of the self-attention modules.
             Mask values selected in ``[0, 1]``:
             :obj:`1` indicates the head is **not masked**, :obj:`0` indicates the head is **masked**.
-        input_embeds (:obj:`torch.FloatTensor` of shape :obj:`(batch_size, sequence_length, hidden_size)`, `optional`, defaults to :obj:`None`):
+        inputs_embeds (:obj:`torch.FloatTensor` of shape :obj:`(batch_size, sequence_length, hidden_size)`, `optional`, defaults to :obj:`None`):
             Optionally, instead of passing :obj:`input_ids` you can choose to directly pass an embedded representation.
             This is useful if you want more control over how to convert `input_ids` indices into associated vectors
             than the model's internal embedding lookup matrix.