Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
d000195e
Commit
d000195e
authored
Dec 17, 2019
by
erenup
Browse files
add comment for example_index and unique_id in single process
parent
3c6efd0c
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
3 additions
and
3 deletions
+3
-3
transformers/data/processors/squad.py
transformers/data/processors/squad.py
+1
-1
transformers/modeling_roberta.py
transformers/modeling_roberta.py
+2
-2
No files found.
transformers/data/processors/squad.py
View file @
d000195e
...
...
@@ -225,7 +225,7 @@ def squad_convert_example_to_features(example, max_seq_length,
span
[
'token_type_ids'
],
cls_index
,
p_mask
.
tolist
(),
example_index
=
0
,
example_index
=
0
,
# Can not set unique_id and example_index here. They will be set after multiple processing.
unique_id
=
0
,
paragraph_len
=
span
[
'paragraph_len'
],
token_is_max_context
=
span
[
"token_is_max_context"
],
...
...
transformers/modeling_roberta.py
View file @
d000195e
...
...
@@ -592,8 +592,8 @@ class RobertaForQuestionAnswering(BertPreTrainedModel):
start_scores, end_scores = model(torch.tensor([input_ids]))
all_tokens = tokenizer.convert_ids_to_tokens(input_ids)
print(' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1]))
#
a nice puppet
#
Note: 'roberta-large' model can not produce the right answer above. Waiting for 'roberta-large-finetuned-squad'
a nice puppet
Note: 'roberta-large' model can not produce the right answer above. Waiting for 'roberta-large-finetuned-squad'
to be uploaded.
"""
config_class
=
RobertaConfig
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment