Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
3c6efd0c
Commit
3c6efd0c
authored
Dec 17, 2019
by
erenup
Browse files
updated usage example in modeling_roberta for question and answering
parent
a1faaf99
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
10 additions
and
7 deletions
+10
-7
transformers/modeling_roberta.py
transformers/modeling_roberta.py
+10
-7
No files found.
transformers/modeling_roberta.py
View file @
3c6efd0c
...
@@ -585,13 +585,16 @@ class RobertaForQuestionAnswering(BertPreTrainedModel):
...
@@ -585,13 +585,16 @@ class RobertaForQuestionAnswering(BertPreTrainedModel):
list of ``torch.FloatTensor`` (one for each layer) of shape ``(batch_size, num_heads, sequence_length, sequence_length)``:
list of ``torch.FloatTensor`` (one for each layer) of shape ``(batch_size, num_heads, sequence_length, sequence_length)``:
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads.
Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads.
Examples::
Examples::
tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
tokenizer = RobertaTokenizer.from_pretrained('roberta-large')
model = RobertaForMultipleChoice.from_pretrained('roberta-base')
model = RobertaForQuestionAnswering.from_pretrained('roberta-large')
input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute")).unsqueeze(0) # Batch size 1
question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"
start_positions = torch.tensor([1])
input_ids = tokenizer.encode(question, text)
end_positions = torch.tensor([3])
start_scores, end_scores = model(torch.tensor([input_ids]))
outputs = model(input_ids, start_positions=start_positions, end_positions=end_positions)
all_tokens = tokenizer.convert_ids_to_tokens(input_ids)
loss, start_scores, end_scores = outputs[:2]
print(' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1]))
# a nice puppet
# Note: 'roberta-large' model can not produce the right answer above. Waiting for 'roberta-large-finetuned-squad'
to be uploaded.
"""
"""
config_class
=
RobertaConfig
config_class
=
RobertaConfig
pretrained_model_archive_map
=
ROBERTA_PRETRAINED_MODEL_ARCHIVE_MAP
pretrained_model_archive_map
=
ROBERTA_PRETRAINED_MODEL_ARCHIVE_MAP
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment