GitLab

- Update PyMuPDF to version <1.25.0
- Update pydantic to version <2.11
- Update transformers to version < 5.0.0
- Remove always_apply parameter from alb.ToGray in image processing

- Remove paddleocr, paddlepaddle, rapidocr-paddle, and rapidocr-onnxruntime from requirements.txt files
- Simplify pip install commands in Dockerfiles
- Remove installation of paddlepaddle-gpu in china and global Dockerfiles
- Update requirements.txt files across all Docker configurations

- Added ftfy package to requirements.txt files for Ascend NPU, China, and Global Docker images
- This change ensures consistent text formatting across all supported platforms

- Update config version to1.2.0
- Refactor model initialization in model_init.py- Update dependencies in requirements.txt files
- Remove unused imports and models
- Add conditional imports for table models

- Update PaddlePaddle from 3.0.0b1 to 3.0.0rc1 in Dockerfiles
- Update RapidOCR dependencies to specific versions in requirements.txt files
- Update requirements for all Docker configurations (china, global, ascend_npu)

- Update rapid-table version from ==0.3.0 to >=1.0.3,<2.0.0 in multiple requirements files
- This change affects Ascend NPU, China, and Global Docker configurations

- Remove doclayout_yolo==0.0.2b1 and doclayout-yolo==0.0.2
- Add doclayout-yolo==0.0.2b1 to all requirements files

- Merge title blocks that are close to each other horizontally
- Adjust line insertion logic for title blocks- Increase image size and decrease confidence threshold for layout detection
- Update DocLayoutYOLO model weights
- Refactor drawing of bounding boxes for different block types

- Set PyMuPDF version to <= 1.24.14 in all requirements files
- Prevent potential compatibility issues with future versions

- Update fast-langdetect from 0.2.0 to >=0.2.3,<0.3.0
- Specify version for rapid_table: ==0.3.0

- Add Dockerfile for global setup with Ubuntu base image
- Add Dockerfile for Huawei NPU setup with Ascend base image
- Update requirements file structure:  - Rename requirements-docker.txt to docker/china/requirements.txt - Add new requirements files for global and Huawei NPU setups
- Install necessary packages and dependencies in both Dockerfiles- Set up virtual environment and install Python packages
- Download models and configure magic-pdf for both setups

- Add openai package to requirements-docker.txt

- Update unimernet from 0.2.2 to 0.2.3 in requirements-docker.txt and setup.py
- Remove torchtext/eva-decord dependency

- Add torch>=2.2.2,<=2.3.1 to requirements-docker.txt- Add torchvision>=0.17.2,<=0.18.1 to requirements-docker.txt

- Add environment variables to disable albumentations and yolo updates
- Import torchtext and disable deprecation warnings
- Update unimernet to 0.2.2
- Specify ultralytics version as >=8.3.48
- Remove upper version limit for torch

- Update the URL for downloading the model setup script in Dockerfile
- Upgrade struct-eqtable to version 0.3.2 and remove pypandoc
- Add new dependencies: einops, accelerate, doclayout_yolo, rapidocr-paddle, and rapid_table

- Add doclayout-yolo==0.0.2 to requirements-docker.txt

- Remove unused functions such as split_long_words, ocr_mk_mm_markdown_with_para, etc.
- Simplify ocr_mk_markdown_with_para_core_v2 by removing unnecessary language detection and word splitting logic- Remove wordninja dependency from requirements
- Update ocr_model_init to include additional parameters for OCR model configuration

update UniMERNet to 0.2.1

Update the paths to model weights and configuration files for the UniMERNet architecture
in both the demo.yaml and model_configs.yaml files. Adjust the mfr_model_init function toreflect the new weight and configuration paths. The changes include specifying more detailed
paths to the unimernet_base directory and changing the weight file extension to .pth.

* Create requirements-docker.txt

* feat: update deps to support rag

* feat: add support to rag, add rag_data_reader api for rag integration

* feat: let user retrieve the filename of the processed file

* feat: add projects demo for rag integrations

---------
Co-authored-by: Xiaomeng Zhao <moe@myhloli.com>
Co-authored-by: icecraft <xurui1@pjlab.org.cn>

* build(docker): update base image to Ubuntu 22.04 and install PaddlePaddleUpgrade the Docker base image from ubuntu:latest to ubuntu:22.04 for improved
performance and stability.

Additionally, integrate PaddlePaddle GPU version 3.0.0b1
into the Docker build for enhanced AI capabilities. The MinIO configuration file has
also been updated to the latest version.

* build(dockerfile): Updated the Dockerfile

* build(Dockerfile): update Dockerfile

* docs(docker): add instructions for quick deployment with Docker

Include Docker-based deployment instructions in the README for both English and
Chinese locales. This update provides users a quick-start guide to using Docker for
deployment, with notes on GPU VRAM requirements and default acceleration features.

* build(docker): Layer the installation of dependencies, downloading the model, and the setup of the program itself.

* build(docker): Layer the installation of dependencies, downloading the model, and the setup of the program itself.

fix(dependencies): remove unnecessary pypandoc and struct-eqtable packages;fix matplotlib>=3.9.1 not support Windows system without compilation environment.

* # add table recognition using struct-eqtable
## Changelog
31/07/20204
- Support table recognition. Table images will be converted into html.

### how to use the new feature:
set the attribute 'table-mode' to 'true' in magic-pdf.json

### caution:
it takes 200s to 500s to convert a single table image using cpu

* # add table recognition using struct-eqtable
## Changelog
31/07/20204
- Support table recognition. Table images will be converted into LaTex.

### how to use the new feature:
set the attribute 'table-mode' to 'true' in magic-pdf.json

### caution:
it takes 200s to 500s to convert a single table image using cpu

* # feat(model inference): add table recognition and convertion to LaTeX

# What's Changed

### New Features

- Add table content recognition, we use weights of [StructEqTable](https://github.com/UniModal4Reasoning/StructEqTable-Deploy) to convert table image to LaTex.

### Instruction

- pip install pypandoc struct-eqtable==0.1.0
- Download [StructEqTable weights](https://huggingface.co/wanderkid/PDF-Extract-Kit/tree/main/models/TabRec

) and put it under models/ directory.
- Edit 'table-mode' value to turn on table recognition function which is turned off by default.
- If you did not download any models before, refer to [how to download models](docs/how_to_download_models_zh_cn.md)。

* add table recognition and convertion to LaTeX

* add table recognition and conversion to LaTeX

* add table recognition and conversion to LaTeX

* add table recognition and conversion to LaTeX

---------
Co-authored-by: liukaiwen <liukaiwen@pjlab.org.cn>

# What's Changed

### New Features

- Add table content recognition, we use weights of [StructEqTable](https://github.com/UniModal4Reasoning/StructEqTable-Deploy) to convert table image to LaTex.

### Instruction

- pip install pypandoc struct-eqtable==0.1.0
- Download [StructEqTable weights](https://huggingface.co/wanderkid/PDF-Extract-Kit/tree/main/models/TabRec) and put it under models/ directory.
- Edit 'table-mode' value to turn on table recognition function which is turned off by default.
- If you did not download any models before, refer to [how to download models](docs/how_to_download_models_zh_cn.md)。