GitLab

fix(parse_pipeline): Resolve post-processing exceptions caused by partial PDFs due to file corruption or non-standard format by forcing a re-print.

- Add digit check for single-character content to avoid adding unnecessary spaces

- Integrate RapidOCR with RapidTable model for table recognition
- Improve memory management for devices with <= 8GB VRAM
- Update table recognition process to use RapidOCR for RapidTable
- Add rapidocr-paddle dependency in setup.py

- Change the default table model from TABLE_MASTER to RAPID_TABLE

- Add RapidTable model support for table recognition
- Update table model configuration and initialization
- Modify table recognition process to use RapidTable when specified
- Add RapidTable dependency to setup.py

- Lower the line count threshold from 316 to 200 to ensure compatibility
- This change aims to prevent potential issues with layoutreader's maximum line support

- Decrease the maximum line count from 512 to 316 for layoutreader

- Implement xycut algorithm to sort blocks when layoutreader fails
- Add recursive_xy_cut function to perform the xycut algorithm- Update pdf_parse_union_core_v2.py to use xycut when layoutreader fails
- Modify draw_bbox.py to handle cases where layoutreader fails to sort blocks

- Remove unused code for copying detection and recognition models
- Simplify OCR model initialization using atom_model_manager
- Delete unnecessary comments and empty lines

- Replace np.array with np.asarray for better performance
- Add image color conversion from RGB to BGR using OpenCV

- Implement __replace_ligatures function to split ligature characters- Integrate ligature replacement into the merge_para_with_text function
- Handle common ligatures such as fi, fl, ff, ffi, and ffl

- Import 're' module for regular expression operations
- Implement HTML minification for 'output_format=html'
- Add 'minify_html' method to remove unnecessary whitespace and format HTML

- Comment out an unused code block in the ppTableModel.py file
- Improve code readability and maintainability by removing unnecessary code

- Update StructTableModel to use the latest struct-eqtable library
- Add support for HTML table extraction in PDF Extract Kit
- Improve error handling and model initialization
- Update dependencies in setup.py for struct-eqtable

Modify line 397 to ensure compatibility with CPU execution, addressing the issue where specifying 'cpu' in config.json still results in a ValueError for expecting a cuda device but getting 'cpu' during demo execution.

- Optimize content stripping and checking logic
- Add special case handling for single-character content
- Adjust spacing rules for different content types

- Add block_height calculation to determine block aspect ratio
- Update list identification condition to include aspect ratio check
- Improve code readability with better formatting and line breaks

feat(list): improve list detection algorithm- Add center_close_num and external_sides_not_close_num variables to analyze line positioning
- Implement new list detection condition for centered lines
- Enhance existing list detection logic with additional checks

fix(list): improve list identification accuracy- Adjust the threshold for determining right-side spacing to 0.26 * block_weight
- Add TODO comment for special list identification with all centered lines- Modify the condition for recognizing short item lists with left alignment
- Update the condition for identifying the end of a list item

- Include InlineEquation in the condition for handling text content
- Remove separate block for InlineEquation processing
- Ensures consistent handling of inline equations and text, improving content formatting

fix(ocr_mkcontent): improve content handling for different languages and equation types- Adjust content formatting for Chinese, Japanese, Korean, and Western languages
- Implement proper spacing rules around inline equations- Remove unnecessary empty lines in paragraph text

- Refactor remove_outside_spans function to filter spans more accurately
- Add image_footnote, index, and list block types to output file documentation
- Update draw_span_bbox to use preproc_blocks instead of para_blocks
- Bump version to 0.9.0

- Update remove_outside_spans function to handle all content types
- Add processing for text and equation spans
- Improve overlap calculation for better accuracy

- Update remove_outside_spans function to handle all content types
- Add processing for text and equation spans
- Improve overlap calculation for better accuracy

- Update remove_outside_spans function to handle all content types
- Add processing for text and equation spans
- Improve overlap calculation for better accuracy

- Add new function `remove_outside_spans` to filter spans based on image and table blocks
- Reorder span processing steps to improve efficiency
- Update imports to include `calculate_overlap_area_in_bbox1_area_ratio`

- Add check for 'image_path' in spans to avoid errors when it's missing
- Update image handling in both paragraph text and content dictionary
- Improve error handling and make the code more robust

- Update image content extraction to iterate through all spans in a block
- Add support for extracting table content from spans within a block
- Handle multiple content types within table spans (latex, html, image)
- Refactor code to be more modular and easier to maintain

- Remove import and usage of StructTableModel- Add support for TableMaster model- Update table model initialization logic to support TableMaster
- Log error and exit if StructEqTable is selected, as it's under upgrade
- Update README files to reflect changes in table parsing capabilities

- Modify the logic for splitting wide blocks exceeding 0.4 page width
- Remove the specific case for blocks exceeding 0.25 page width
- Add comments to explain the reasoning behind different splitting strategies