GitLab

fix: add init to magic_pdf.config

refactor(ocr): adjust OCR processing parameters

- Lower the Y-axis overlap threshold for merging spans into lines from0.6 to 0.5
- Reduce the unclip ratio for OCR detection from 2.4 to 1.8

refactor(parse_core): improve image and table block handling 

- Split image and table blocks into separate categories
- Add group_id to image and table blocks- Update block processing logic to handle new categories
- Modify layout splitting and span filling to accommodate new block types
- Adjust block indexing and sorting to consider new structures

Feat/new table caption match

- Changed the confidence threshold from0.15 to 0.25 in the DocLayout_YOLO model prediction
- This adjustment aims to improve the accuracy of layout detection by filtering out low-confidence predictions

Feat/data api

feat: add Data api

feat: add support for non-PDF file conversion to PDF

- Implement to_pdf function to convert non-PDF files to PDF format
- Integrate file upload functionality for PDF and image files- Update UI to include file upload component and PDF preview
- Add conversion button and update its functionality to handle new file types

build(docker): add doclayout-yolo dependency

- Add doclayout-yolo==0.0.2 to requirements-docker.txt

build(setup): add doclayout_yolo dependency

- Add doclayout_yolo==0.0.2 to the list of dependencies in setup.py

feat(model): add support for DocLayout-YOLO model

- Add new layout model option: DocLayout-YOLO
- Implement model initialization and prediction for DocLayout-YOLO
- Update configuration options to include new model- Modify existing code to support both LayoutLMv3 and DocLayout-YOLO models
- Update Gradio app to support more Custom Switch

update：update config json

docs:Update the driver requirements on the Ubuntu system.

- Modified the condition to include List and Index block types- This change enhances the function's capability to process different paragraph types

refactor(para): improve paragraph splitting algorithm

- Adjust the threshold for identifying index blocks from 3 lines to 2 lines
- Add a new function __is_list_group to detect if a group of blocks is a list
- Modify the paragraph merging logic to handle list groups differently

- Remove unused parameters parse_type and lang from various functions
- Simplify function calls by removing unnecessary arguments
- Update related files to reflect these changes

refactor(ocr):Increase the dilation factor in OCR to address the issue of word concatenation.

- Remove unused functions such as split_long_words, ocr_mk_mm_markdown_with_para, etc.
- Simplify ocr_mk_markdown_with_para_core_v2 by removing unnecessary language detection and word splitting logic- Remove wordninja dependency from requirements
- Update ocr_model_init to include additional parameters for OCR model configuration

update example files

- Improve reading order with model-based sorting- Add list recognition within text
- Implement table of contents recognition
- Support table recognition
- Enhance code block and geometric shape recognition
- Address known issues in both English and Chinese READMEs

fix(para_split_v3): refine list block detection in paragraph splitting

- Update list block detection logic to require at least 2 numeric start lines
- Ensure the number of numeric start lines matches the number of end lines
- Remove detection of non-border starting lines for simplicity

refactor(para_split_v3): merge list and index block detection