Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
ModelZoo
ResNet50_tensorflow
Commits
730548d4
Commit
730548d4
authored
Nov 23, 2020
by
Allen Wang
Committed by
A. Unique TensorFlower
Nov 23, 2020
Browse files
Internal changes.
PiperOrigin-RevId: 343893692
parent
8bc64372
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
0 deletions
+4
-0
official/nlp/data/create_xlnet_pretraining_data.py
official/nlp/data/create_xlnet_pretraining_data.py
+4
-0
No files found.
official/nlp/data/create_xlnet_pretraining_data.py
View file @
730548d4
...
@@ -477,6 +477,10 @@ def _convert_tokens_to_instances(
...
@@ -477,6 +477,10 @@ def _convert_tokens_to_instances(
+
[
1
]
*
b_length
+
[
1
]
+
[
2
])
+
[
1
]
*
b_length
+
[
1
]
+
[
2
])
boundary_indices
=
_get_boundary_indices
(
tokenizer
=
tokenizer
,
boundary_indices
=
_get_boundary_indices
(
tokenizer
=
tokenizer
,
data
=
data
)
data
=
data
)
assert
len
(
data
)
==
seq_length
assert
len
(
segment_ids
)
==
seq_length
assert
len
(
boundary_indices
)
>
0
# pylint: disable=g-explicit-length-test
instances
.
append
(
TrainingInstance
(
instances
.
append
(
TrainingInstance
(
data
=
data
,
data
=
data
,
segment_ids
=
segment_ids
,
segment_ids
=
segment_ids
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment