GitLab

- Add NPU support for memory cleaning and model initialization
- Optimize table model initialization and prediction process
- Update memory utils to support NPU
- Add language parameter for table model

- Update unimernet from 0.2.2 to 0.2.3 in requirements-docker.txt and setup.py
- Remove torchtext/eva-decord dependency

- Comment out logging statements for title list, title completion, and length comparison
- Improve code readability and reduce clutter by removing unused debug information

- Implement llm_aided_title function to optimize document titles using LLM
- Update pdf_parse_union_core_v2.py to include title optimization
- Modify ocr_mkcontent.py to use optimized title levels- Add openai SDK dependency in setup.py

feat(llm): add LLM-aided formula and text correction

- Add LLM-aided formula and text correction functionality
- Update config reader to include LLM-aided settings
- Create new LLM-aided processing module
- Update main processing script to incorporate LLM-aided corrections
- Modify download scripts to check for new config version

refactor(pre_proc): improve character overlap handling in spans 

- Remove remove_overlaps_chars function
- Add check_chars_is_overlap_in_span function
- Update span processing logic to handle character overlaps- Improve efficiency and readability of overlap detection

docs: add more method description

feat(demo): add demo script for PDF processing

- Create demo.py script for PDF file processing
- Implement PDF reading, classification, and inference usingOpendatalab's magic_pdf library- Add pipelines for OCR and text modes
- Include result visualization and markdown export

feat(pre_proc): add function to remove overlapping characters in spans

- Implement remove_overlaps_chars function to detect and remove overlapping characters within spans
- Integrate remove_overlaps_chars function into the PDF parsing process
- Improve character-level processing and reduce redundancy in OCR results

fix: skip the char corresponding to invalid bounding boxes

feat(gradio-app): improve PDF conversion and UI functionalities

- Add automatic conversion of uploaded files to PDF
- Update max page slider range and default value- Prevent interaction with PDF preview to avoid errors
- Increase Markdown rendering height for better visibility
- Update file change event handling for PDF conversion
- Modify supported image suffixes for file upload

refactor(magic_pdf): move model config variables

- Move __use_inside_model__ and __model_mode__ from operators/__init__.py to model/__init__.py
- These variables are more appropriately located in the model module since they relate to model configuration

Docs/fix docs

refactor: refactor code

feat(language-detection): add YOLOv11 language detection model

- Add YOLOv11 language detection model for PDF documents
- Implement language detection in PymuDocDataset
- Update app.py to include 'auto' language option
- Create language detection utilities and constants

feat: add get_middle_json method

refactor(magic_pdf): remove YOLO_VERBOSE setting and update YOLOv8 prediction verbosity

- Remove YOLO_VERBOSE environment variable from multiple files
- Set verbose=False in YOLOv8 prediction method to suppress logger output

fix(pdf): improve ligature handling and text extraction