GitLab

master -> dev

Release 0.10.0

fix(pdf_parse): improve OCR result handling

fix(pdf_parse): improve OCR result handling

- Add null check for OCR results to prevent errors on empty lists
- Enhance robustness of OCR text processing in the magic-pdf project

fix(table): add null check for OCR result in rapid table prediction 

refactor(model): move page total time logging to custom model analysis

- Move page total time logging to doc_analyze_by_custom_model.py
- Remove page total time logging from pdf_extract_kit.py
- Add page_start timing variable to custom model analysis
- Update logger output format for page total time

fix(table): add null check for OCR result in rapid table prediction

- Add a null check for OCR result in the predict method
- Return None values if OCR result is None to prevent further processing

feat(README): update for v0.10.0 

- Introduced hybrid OCR text extraction capabilities in v0.10.0
- Significantly improved parsing performance in complex text distribution scenarios- Combined advantages of accurate content extraction and faster speed in text mode with more precise span/line region recognition in OCR mode
- Updated both English and Chinese README files

refactor(para): improve line stop flag and remove unused debug mode

- Add '-' and '–' to LINE_STOP_FLAG in pdf_parse_union_core_v2.py
- Remove unused debug_mode parameter from para_split function in para_split_v3.py

* delete unused pipeline file

* add json test circle

* add size reduction test case

* add serializable test case

* add invalid json compress test case

* add empty test case

* add special char test case

test: comment out assertions for metascan classify and meta scan tests

- Commented out assertions in test_metascan_classify/test_classify.py
- Commented out assertions in test_metascan_classify/test_meta_scan.py
- This change affects multiple test cases across both test files

fix(pdf_parse): improve line stop flag detection accuracy

- Add an additional condition to the line stop flag check
- Ensure character is to the right of the span's left boundary
- This change helps reduce false positives in line stop detection

fix: use concrete class instead of abstract class

refactor(txt_parse): improve text extraction accuracy with new algorithm

- Implement new text extraction method (txt_spans_extract_v2) to enhance accuracy
- Add character filling in spans for better text reconstruction
- Introduce empty span handling using OCR for missed text
- Optimize span filtering and overlap removal

feat(ocr): improve text detection and OCR accuracy 

# Conflicts:
#	magic_pdf/model/pdf_extract_kit.py

- Update OCR utils to handle different box formats and improve angle calculation
- Modify PDF extraction kit to support OCR option and optimize processing flow
- Enhance PPOCR model to sort and filter detection boxes, improving text splitting accuracy

fix(remove_overlaps_min_spans): optimize overlap detection in OCR span list modification

- Improve logic to skip dropped spans in overlap detection
- Enhance efficiency by avoiding unnecessary comparisons

fix(ocr_mkcontent): improve hyphen handling at line ends

- fix the bug where hyphens in the middle of a line are being discarded

refactor(ocr_dict_merge): add threshold parameter for line merging

- Add threshold parameter to merge_spans_to_line function
- Make threshold configurable for y-axis overlap check
- Improve flexibility and accuracy of line merging algorithm

fix(tools): handle empty language string in common.py

- Check if language string is empty and set it to None
- This prevents potential errors when an empty language string is passed