Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
88b3a91e
"...git@developer.sourcefind.cn:modelzoo/solov2-pytorch.git" did not exist on "153a4e909bc9283653eb49886d9df88b2bd75ae8"
Unverified
Commit
88b3a91e
authored
Oct 23, 2020
by
Lalit Pagaria
Committed by
GitHub
Oct 23, 2020
Browse files
Handle the case when title is None (#7941)
parent
023f0f37
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
3 deletions
+4
-3
examples/rag/use_own_knowledge_dataset.py
examples/rag/use_own_knowledge_dataset.py
+4
-3
No files found.
examples/rag/use_own_knowledge_dataset.py
View file @
88b3a91e
...
@@ -35,8 +35,9 @@ def split_documents(documents: dict) -> dict:
...
@@ -35,8 +35,9 @@ def split_documents(documents: dict) -> dict:
"""Split documents into passages"""
"""Split documents into passages"""
titles
,
texts
=
[],
[]
titles
,
texts
=
[],
[]
for
title
,
text
in
zip
(
documents
[
"title"
],
documents
[
"text"
]):
for
title
,
text
in
zip
(
documents
[
"title"
],
documents
[
"text"
]):
if
text
is
not
None
:
for
passage
in
split_text
(
text
):
for
passage
in
split_text
(
text
):
titles
.
append
(
title
)
titles
.
append
(
title
if
title
is
not
None
else
""
)
texts
.
append
(
passage
)
texts
.
append
(
passage
)
return
{
"title"
:
titles
,
"text"
:
texts
}
return
{
"title"
:
titles
,
"text"
:
texts
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment