Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
wangsen
MinerU
Commits
11344890
Commit
11344890
authored
Dec 09, 2024
by
icecraft
Browse files
fix: unicode decode error
parent
c5a4150e
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
13 additions
and
1 deletion
+13
-1
magic_pdf/data/data_reader_writer/base.py
magic_pdf/data/data_reader_writer/base.py
+13
-1
No files found.
magic_pdf/data/data_reader_writer/base.py
View file @
11344890
...
...
@@ -48,4 +48,16 @@ class DataWriter(ABC):
path (str): the target file where to write
data (str): the data want to write
"""
self
.
write
(
path
,
data
.
encode
())
def
safe_encode
(
data
:
str
,
method
:
str
):
try
:
bit_data
=
data
.
encode
(
encoding
=
method
,
errors
=
'replace'
)
return
bit_data
,
True
except
:
# noqa
return
None
,
False
for
method
in
[
'utf-8'
,
'ascii'
]:
bit_data
,
flag
=
safe_encode
(
data
,
method
)
if
flag
:
self
.
write
(
path
,
bit_data
)
break
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment