GitLab

Increased the threshold for filling spans in blocks from 0.3 to 0.5 to improve the accuracy of block formation. This change helps refine the grouping of spans into blocks, potentially enhancing the overall structure and readability of the PDF content.

- Combine __is_list_block() and __is_index_block() into a single function __is_list_or_index_block()
- Simplify block type determination logic
- Remove redundant code and improve readability
- Optimize block merging process

Add List and Index to the list of block types being processed in the draw_bbox.py file. This inclusion ensures that these block types are handled similarly to other text-containing blocks, improving the overall document processing accuracy and consistency.

- Add detection for list and index blocks in OCR processing- Implement merging of list and index blocks across pages
- Update block types to include list and index categories
- Adjust text merging logic to handle new block types
- Modify layout drawing to distinguish list and index blocks

- Reintegrate para_split_v3 into the pdf_parse_union_core_v2 process
- Add support for specifying page range in doc_analyze_by_custom_model
- Implement garbage collection and memory cleaning after processing
- Refine image loading from PDF, including handling out-of-range pages

Update README_Windows_CUDA_Acceleration_en_US.md

docs: update CUDA acceleration guides and README content

- Update GPU hardware support information in README.md and README_zh-CN.md
- Enhance CUDA acceleration guides for Ubuntu and Windows
- Modify README_zh-CN.md to reflect changes in GPU requirements and configurations
- Update TODO list to mark semantic reading order as completed

- Update GPU hardware support information in README.md and README_zh-CN.md
- Enhance CUDA acceleration guides for Ubuntu and Windows
- Modify README_zh-CN.md to reflect changes in GPU requirements and configurations
- Update TODO list to mark semantic reading order as completed

docs: add filename to wget command in model download scripts

- Update wget commands in both English and Chinese documentation to specify the filename
- Improve clarity and prevent potential filename conflicts when downloading the scripts

feat(docs): automate model download and configuration

- Add scripts to download models and update configuration file
- Remove manual steps for modifying model paths
- Update documentation for both ModelScope and HuggingFace model downloads
- Improve user experience by automating the entire process

- Add scripts to download models and update configuration file
- Remove manual steps for modifying model paths
- Update documentation for both ModelScope and HuggingFace model downloads
- Improve user experience by automating the entire process

feat(layoutreader): support local model directory and improve model loading

Added a link to the layoutreader repository in the Related Projects sections of both the README.md and README_zh-CN.md files. This addition helps to provide users with more resources and tools related to document layout analysis and processing.

docs: update model download instructions for version 0.9.x and later- Add note about separate download for layoutreader model in version 0.9.x and later
- Include example code for downloading layoutreader model using ModelScope
- Clarify that previous download methods do not support updating to version 0.9.x and later

- Add function to get local LayoutReader model directory- Check and use local model directory if available
- Fall back to online model if local directory not found
- Update model initialization to support local path
- Refactor model loading in singleton class

fix: caption|footnote match algorithm

fix: caption or footnote match algorithm

perf(pdf_extract_kit): conditional memory cleanup based on GPU capacity

- Introduce a conditional memory cleanup step in the PDF extraction process
- Assess available GPU memory before deciding to perform memory cleanup- Log the time taken for garbage collection when it occurs
- This optimization helps to balance performance and resource utilization

feat: add arXiv paper link to header and adjust PDF parsing logic

feat: add arXiv paper link to header and adjust PDF parsing logic- Add arXiv paper link to the header template for easy access to the latest research paper.
- Modify the PDF parsing logic to handle edge cases more accurately, particularly in determining the number of lines in a block based on its height.

refactor(model): improve timing information and performance

- Enhance timing output precision to two decimal places for better readability- Calculate and log document analysis speed in pages per second
- Optimize logging for YOLO and table recognition processes
- Remove unnecessary comments and improve code efficiency

add arxiv url

add arxiv url

feat：add layoutreader to sort blocks

- Insert lines into blocks based on median line height- Calculate block index using line indices median
- Remove virtual line information for table and image blocks
- Enhance line sorting algorithm for different block types
- Add line height calculation function

The clean_memory function has been removed from pdf_parse_union_core_v2.py due to it not being used.
This change streamlines the code and prevents potential confusion regarding its purpose.

Update import statements in `pdf_parse_union_core_v2.py` to directly import
`prepare_inputs`, `boxes2inputs`, and `parse_logits` from `magic_pdf.model.v3.helpers`
instead of from `magic_pdf.model.v3`. This change streamlines the imports, making the
code more readable and maintaining a cleaner approach to modular design.

Adapt import statements in `pdf_parse_union_core_v2.py` to reflect the updated packagestructure, changing from the `magic_pdf.v3.helpers` module to the `magic_pdf.model.v3`
module. This ensures compatibility with the revised directory layout.