Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
wangsen
MinerU
Commits
71b5024a
Unverified
Commit
71b5024a
authored
Feb 26, 2025
by
Xiaomeng Zhao
Committed by
GitHub
Feb 26, 2025
Browse files
Merge pull request #1787 from myhloli/dev
refactor(magic_pdf): remove bfloat16 support checks and usage
parents
fddf111f
0a246f0f
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
21 deletions
+1
-21
magic_pdf/pdf_parse_union_core_v2.py
magic_pdf/pdf_parse_union_core_v2.py
+1
-21
No files found.
magic_pdf/pdf_parse_union_core_v2.py
View file @
71b5024a
...
...
@@ -338,24 +338,7 @@ def txt_spans_extract_v2(pdf_page, spans, all_bboxes, all_discarded_blocks, lang
def
model_init
(
model_name
:
str
):
from
transformers
import
LayoutLMv3ForTokenClassification
device
=
get_device
()
if
torch
.
cuda
.
is_available
():
device
=
torch
.
device
(
'cuda'
)
if
torch
.
cuda
.
is_bf16_supported
():
supports_bfloat16
=
True
else
:
supports_bfloat16
=
False
elif
str
(
device
).
startswith
(
"npu"
):
import
torch_npu
if
torch_npu
.
npu
.
is_available
():
device
=
torch
.
device
(
'npu'
)
supports_bfloat16
=
False
else
:
device
=
torch
.
device
(
'cpu'
)
supports_bfloat16
=
False
else
:
device
=
torch
.
device
(
'cpu'
)
supports_bfloat16
=
False
device
=
torch
.
device
(
get_device
())
if
model_name
==
'layoutreader'
:
# 检测modelscope的缓存目录是否存在
...
...
@@ -371,9 +354,6 @@ def model_init(model_name: str):
model
=
LayoutLMv3ForTokenClassification
.
from_pretrained
(
'hantian/layoutreader'
)
# 检查设备是否支持 bfloat16
if
supports_bfloat16
:
model
.
bfloat16
()
model
.
to
(
device
).
eval
()
else
:
logger
.
error
(
'model name not allow'
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment