Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
9d0d1cd3
Commit
9d0d1cd3
authored
Dec 20, 2019
by
Morgan Funtowicz
Browse files
Filter out entity for NER task.
parent
e4baa68d
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
5 deletions
+6
-5
transformers/pipelines.py
transformers/pipelines.py
+6
-5
No files found.
transformers/pipelines.py
View file @
9d0d1cd3
...
@@ -450,11 +450,12 @@ class NerPipeline(Pipeline):
...
@@ -450,11 +450,12 @@ class NerPipeline(Pipeline):
score
=
np
.
exp
(
entities
[
token_start
])
/
np
.
exp
(
entities
[
token_start
]).
sum
(
-
1
,
keepdims
=
True
)
score
=
np
.
exp
(
entities
[
token_start
])
/
np
.
exp
(
entities
[
token_start
]).
sum
(
-
1
,
keepdims
=
True
)
label_idx
=
score
.
argmax
()
label_idx
=
score
.
argmax
()
answer
+=
[{
if
label_idx
>
0
:
'word'
:
words
[
idx
],
answer
+=
[{
'score'
:
score
[
label_idx
].
item
(),
'word'
:
words
[
idx
],
'entity'
:
self
.
model
.
config
.
id2label
[
label_idx
]
'score'
:
score
[
label_idx
].
item
(),
}]
'entity'
:
self
.
model
.
config
.
id2label
[
label_idx
]
}]
# Update token start
# Update token start
token_start
+=
len
(
list
(
word
))
token_start
+=
len
(
list
(
word
))
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment