Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
6494479f
Unverified
Commit
6494479f
authored
Jul 29, 2024
by
Fanli Lin
Committed by
GitHub
Jul 29, 2024
Browse files
make `p_mask` a numpy array before passing to `select_starts_ends` (#32076)
* fix * bug fix * refine * fix
parent
535fe78b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
src/transformers/pipelines/document_question_answering.py
src/transformers/pipelines/document_question_answering.py
+1
-1
No files found.
src/transformers/pipelines/document_question_answering.py
View file @
6494479f
...
@@ -378,7 +378,7 @@ class DocumentQuestionAnsweringPipeline(ChunkPipeline):
...
@@ -378,7 +378,7 @@ class DocumentQuestionAnsweringPipeline(ChunkPipeline):
# p_mask: mask with 1 for token than cannot be in the answer (0 for token which can be in an answer)
# p_mask: mask with 1 for token than cannot be in the answer (0 for token which can be in an answer)
# We put 0 on the tokens from the context and 1 everywhere else (question and special tokens)
# We put 0 on the tokens from the context and 1 everywhere else (question and special tokens)
# This logic mirrors the logic in the question_answering pipeline
# This logic mirrors the logic in the question_answering pipeline
p_mask
=
[[
tok
!=
1
for
tok
in
encoding
.
sequence_ids
(
span_id
)]
for
span_id
in
range
(
num_spans
)]
p_mask
=
np
.
array
(
[[
tok
!=
1
for
tok
in
encoding
.
sequence_ids
(
span_id
)]
for
span_id
in
range
(
num_spans
)]
)
for
span_idx
in
range
(
num_spans
):
for
span_idx
in
range
(
num_spans
):
if
self
.
framework
==
"pt"
:
if
self
.
framework
==
"pt"
:
span_encoding
=
{
k
:
torch
.
tensor
(
v
[
span_idx
:
span_idx
+
1
])
for
(
k
,
v
)
in
encoding
.
items
()}
span_encoding
=
{
k
:
torch
.
tensor
(
v
[
span_idx
:
span_idx
+
1
])
for
(
k
,
v
)
in
encoding
.
items
()}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment