GitLab

fix: dup classify pdf type

build(deps): update torch and torchvision version requirements

- Specify torch==2.3.1 and torchvision==0.18.1 for Windows CUDA installation
- Add torch and torchvision version constraints in setup.py:
  - torch>=2.2.2,<=2.3.1
  - torchvision>=0.17.2,<=0.18.1
- Update installation instructions in both English and Chinese README files

fix(detect_invalid_chars):fix the stack error caused by multiple memory releases in PyMuPDF

- Uncomment pdfminer.six in requirements.txt
- Specify version 20231228 for pdfminer.six

- Change import paths from paddleocr.ppocr to ppocr for utility functions
- Update import paths for logging and utility modules in ppocr_273_mod.py- Modify import paths for tablemaster_paddle.py to use ppstructure instead of paddleocr.ppstructure

- Replace MuPDF with pdfminer for detecting invalid characters in PDFs
- Uncomment and update the detect_invalid_chars function to use pdfminer
- Update the check_invalid_chars function in pdf_meta_scan.py to use the new implementation

- Change import path for TableSystem from 'ppstructure.table.predict_table' to 'paddleocr.ppstructure.table.predict_table'
- Change import path for init_args from 'ppstructure.utility' to 'paddleocr.ppstructure.utility'

- Modify import paths for paddleocr utilities in ocr_utils.py and ppocr_273_mod.py
- Change from `ppocr.utils.utility` to `paddleocr.ppocr.utils.utility`
- Update related import statements in two files to reflect the new path

- Remove commented-out call to clean_memory() function
- This change simplifies the code by eliminating an unused code snippet

- Import paddle module and disable its signal handler to prevent interference with other components
- This change addresses potential conflicts between PaddlePaddle and other libraries or system signals

- Remove the call to clean_memory() function from pdf_parse_union_core_v2.py
- This change may affect memory usage and needs to be tested to ensure proper functionality

docs(windows): update CUDA installation guide

- Remove specific version requirements for torch and torchvision
- Simplify installation command in both English and Chinese guides
- Delete important note about version compatibility

refactor(magic_pdf): optimize environment setup and dependencies

- Add environment variables to disable albumentations and yolo updates
- Import torchtext and disable deprecation warnings
- Update unimernet to 0.2.2
- Specify ultralytics version as >=8.3.48
- Remove upper version limit for torch

build(deps): update dependency versions

- Update ultralytics to >=8.3.47

fix: unicode decode error

fix: add parse_pdf_type and version

fix(dict2md): add space for inline equations in CJK contexts

- In Chinese, Japanese, and Korean (CJK) languages, no space is needed for line breaks within paragraphs.
- However, if an inline equation is at the end of a line, a space should be added to separate it from the following text.
- This change improves the formatting of documents containing both CJK text and inline equations.

Refactor/add user api

refactor(magic-pdf): optimize model initialization and concurrency control

- Remove concurrency limit logic from app.py
- Update model initialization process in various modules
- Remove unused VRAM check for concurrency limit
- Refactor OCR model initialization in pdf_extract_kit.py
- Update txt_spans_extract_v2 function to use lang parameter instead of ocr_model

refactor(ocr): replace AtomModelSingleton with ocr_model_init for OCR model instantiation

- Remove usage of AtomModelSingleton for OCR model creation
- Add ocr_model_init function to initialize OCR model
- Update OCR model initialization in pdf_extract_kit.py and pdf_parse_union_core_v2.py
- Modify txt_spans_extract_v2 function to accept ocr_model as a parameter
- Update parse_page_core function to use ocr_model instead of lang for OCR processing

refactor(model): implement thread-safe OCR model initialization

- Add threading support for OCR model initialization
- Modify AtomModelSingleton to handle thread-specific instances
- Update PDFExtractKit and PDFParseUnionCoreV2 to use new thread-safe OCR initialization

build(deps): specify minimum version for ultralytics

- Update `ultralytics` dependency to version >= 8.3.43
- This change ensures compatibility with yolov8 for formula detection

refactor(magic_pdf): remove unused threading lock and model initialization code

- Remove threading.Lock import and usage
- Delete unused model initialization comments and code- Simplify OCR model initialization in both pdf_extract_kit.py and pdf_parse_union_core_v2.py