Skip to content

GitLab

".pytest_cache/v/cache/nodeids" did not exist on "8a80947a6269a9760e33485b282b839fe9627385"
Unverified Commit 01054426 authored by Xiaomeng Zhao's avatar Xiaomeng Zhao Committed by GitHub
Browse files

Merge pull request #2082 from opendatalab/dev 

docs(user_guide): update installation guide and CUDA support
parents 579057dd 5c46c791
Show whitespace changes
Inline Side-by-side
Showing with 157 additions and 242 deletions
next_docs/en/user_guide/install/boost_with_cuda.rst
View file @ 01054426
......@@ -28,12 +28,12 @@ NVIDIA drivers are already installed, and you can skip Step 2.
.. note::
``CUDA Version`` should be >= 12.1, If the displayed version number is less than 12.1, please upgrade the driver.
``CUDA Version`` should be >= 12.4, If the displayed version number is less than 12.4, please upgrade the driver.
.. code:: text
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 537.34 Driver Version: 537.34 CUDA Version: 12.2 |
| NVIDIA-SMI 570.133.07 Driver Version: 572.83 CUDA Version: 12.8 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
......@@ -52,7 +52,7 @@ If no driver is installed, use the following command:
.. code:: sh
sudo apt-get update
sudo apt-get install nvidia-driver-545
sudo apt-get install nvidia-driver-570-server
Install the proprietary driver and restart your computer after
installation.
......@@ -80,15 +80,15 @@ Specify Python version 3.10.
.. code:: sh
conda create -n MinerU python=3.10
conda activate MinerU
conda create -n mineru 'python<3.13' -y
conda activate mineru
5. Install Applications
~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
pip install -U magic-pdf[full] --extra-index-url https://wheels.myhloli.com
pip install -U magic-pdf[full]
.. admonition:: Important
:class: tip
......@@ -99,7 +99,7 @@ Specify Python version 3.10.
magic-pdf --version
If the version number is less than 0.7.0, please report the issue.
If the version number is less than 1.3.0, please report the issue.
6. Download Models
~~~~~~~~~~~~~~~~~~
......@@ -126,7 +126,7 @@ Download a sample file from the repository and test it.
.. code:: sh
wget https://github.com/opendatalab/MinerU/raw/master/demo/small_ocr.pdf
wget https://github.com/opendatalab/MinerU/raw/master/demo/pdfs/small_ocr.pdf
magic-pdf -p small_ocr.pdf -o ./output
9. Test CUDA Acceleration
......@@ -150,23 +150,6 @@ to test CUDA acceleration:
magic-pdf -p small_ocr.pdf -o ./output
10. Enable CUDA Acceleration for OCR
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Download ``paddlepaddle-gpu``. Installation will automatically enable
OCR acceleration.
.. code:: sh
python -m pip install paddlepaddle-gpu==3.0.0b1 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/
2. Test OCR acceleration with the following command:
.. code:: sh
magic-pdf -p small_ocr.pdf -o ./output
.. _windows_10_or_11_section:
......@@ -176,11 +159,12 @@ Windows 10/11
1. Install CUDA and cuDNN
~~~~~~~~~~~~~~~~~~~~~~~~~
Required versions: CUDA 11.8 + cuDNN 8.7.0
You need to install a CUDA version that is compatible with torch's requirements. Currently, torch supports CUDA 11.8/12.4/12.6.
- CUDA 11.8 https://developer.nvidia.com/cuda-11-8-0-download-archive
- CUDA 12.4 https://developer.nvidia.com/cuda-12-4-0-download-archive
- CUDA 12.6 https://developer.nvidia.com/cuda-12-6-0-download-archive
- CUDA 11.8: https://developer.nvidia.com/cuda-11-8-0-download-archive
- cuDNN v8.7.0 (November 28th, 2022), for CUDA 11.x:
https://developer.nvidia.com/rdp/cudnn-archive
2. Install Anaconda
~~~~~~~~~~~~~~~~~~~
......@@ -192,19 +176,17 @@ Download link: https://repo.anaconda.com/archive/Anaconda3-2024.06-1-Windows-x86
3. Create an Environment Using Conda
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Python version must be 3.10.
::
conda create -n MinerU python=3.10
conda activate MinerU
conda create -n mineru 'python<3.13' -y
conda activate mineru
4. Install Applications
~~~~~~~~~~~~~~~~~~~~~~~
::
pip install -U magic-pdf[full] --extra-index-url https://wheels.myhloli.com
pip install -U magic-pdf[full]
.. admonition:: Important
:class: tip
......@@ -215,7 +197,7 @@ Python version must be 3.10.
magic-pdf --version
If the version number is less than 0.7.0, please report it in the issues section.
If the version number is less than 1.3.0, please report it in the issues section.
5. Download Models
~~~~~~~~~~~~~~~~~~
......@@ -242,7 +224,7 @@ Download a sample file from the repository and test it.
.. code:: powershell
wget https://github.com/opendatalab/MinerU/raw/master/demo/small_ocr.pdf -O small_ocr.pdf
wget https://github.com/opendatalab/MinerU/raw/master/demo/pdfs/small_ocr.pdf -O small_ocr.pdf
magic-pdf -p small_ocr.pdf -o ./output
8. Test CUDA Acceleration
......@@ -251,23 +233,12 @@ Download a sample file from the repository and test it.
If your graphics card has at least 8GB of VRAM, follow these steps to
test CUDA-accelerated parsing performance.
1. **Overwrite the installation of torch and torchvision** supporting CUDA.
1. **Overwrite the installation of torch and torchvision** supporting CUDA.(Please select the appropriate index-url based on your CUDA version. For more details, refer to the [PyTorch official website](https://pytorch.org/get-started/locally/).)
.. code:: sh
pip install --force-reinstall torch==2.3.1 torchvision==0.18.1 --index-url https://download.pytorch.org/whl/cu118
.. admonition:: Important
:class: tip
❗️Ensure the following versions are specified in the command:
.. code:: sh
torch==2.3.1 torchvision==0.18.1
pip install --force-reinstall torch==2.6.0 torchvision==0.21.1 "numpy<2.0.0" --index-url https://download.pytorch.org/whl/cu124
These are the highest versions we support. Installing higher versions without specifying them will cause the program to fail.
2. **Modify the value of ``"device-mode"``** in the ``magic-pdf.json``
configuration file located in your user directory.
......@@ -283,19 +254,3 @@ test CUDA-accelerated parsing performance.
::
magic-pdf -p small_ocr.pdf -o ./output
9. Enable CUDA Acceleration for OCR
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. **Download paddlepaddle-gpu**, which will automatically enable OCR
acceleration upon installation.
::
pip install paddlepaddle-gpu==2.6.1
2. **Run the following command to test OCR acceleration**:
::
magic-pdf -p small_ocr.pdf -o ./output
next_docs/en/user_guide/install/install.rst
View file @ 01054426
......@@ -41,23 +41,27 @@ Also you can try `online demo <https://www.modelscope.cn/studios/OpenDataLab/Min
<td colspan="3" rowspan="2">Operating System</td>
</tr>
<tr>
<td>Ubuntu 22.04 LTS</td>
<td>Linux after 2019</td>
<td>Windows 10 / 11</td>
<td>macOS 11+</td>
</tr>
<tr>
<td colspan="3">CPU</td>
<td>x86_64(unsupported ARM Linux)</td>
<td>x86_64 / arm64</td>
<td>x86_64(unsupported ARM Windows)</td>
<td>x86_64 / arm64</td>
</tr>
<tr>
<td colspan="3">Memory</td>
<td colspan="3">Memory Requirements</td>
<td colspan="3">16GB or more, recommended 32GB+</td>
</tr>
<tr>
<td colspan="3">Storage Requirements</td>
<td colspan="3">20GB or more, with a preference for SSD</td>
</tr>
<tr>
<td colspan="3">Python Version</td>
<td colspan="3">3.10(Please make sure to create a Python 3.10 virtual environment using conda)</td>
<td colspan="3">3.10~3.12</td>
</tr>
<tr>
<td colspan="3">Nvidia Driver Version</td>
......@@ -67,22 +71,22 @@ Also you can try `online demo <https://www.modelscope.cn/studios/OpenDataLab/Min
</tr>
<tr>
<td colspan="3">CUDA Environment</td>
<td>Automatic installation [12.1 (pytorch) + 11.8 (paddle)]</td>
<td>11.8 (manual installation) + cuDNN v8.7.0 (manual installation)</td>
<td>11.8/12.4/12.6</td>
<td>11.8/12.4/12.6</td>
<td>None</td>
</tr>
<tr>
<td rowspan="2">GPU Hardware Support List</td>
<td colspan="2">Minimum Requirement 8G+ VRAM</td>
<td colspan="2">3060ti/3070/4060<br>
8G VRAM enables layout, formula recognition acceleration and OCR acceleration</td>
<td rowspan="2">None</td>
<td colspan="3">CANN Environment(NPU support)</td>
<td>8.0+(Ascend 910b)</td>
<td>None</td>
<td>None</td>
</tr>
<tr>
<td colspan="2">Recommended Configuration 10G+ VRAM</td>
<td colspan="2">3080/3080ti/3090/3090ti/4070/4070ti/4070tisuper/4080/4090<br>
10G VRAM or more can enable layout, formula recognition, OCR acceleration and table recognition acceleration simultaneously
</td>
<td rowspan="2">GPU/MPS Hardware Support List</td>
<td colspan="2">GPU VRAM 6GB or more</td>
<td colspan="2">All GPUs with Tensor Cores produced from Volta(2017) onwards.<br>
More than 6GB VRAM </td>
<td rowspan="2">apple slicon</td>
</tr>
</table>
......@@ -93,9 +97,9 @@ Create an environment
.. code-block:: shell
conda create -n MinerU python=3.10
conda activate MinerU
pip install -U magic-pdf[full] --extra-index-url https://wheels.myhloli.com
conda create -n mineru 'python<3.13' -y
conda activate mineru
pip install -U "magic-pdf[full]"
Download model weight files
......
next_docs/zh_cn/user_guide/install/boost_with_cuda.rst
View file @ 01054426
......@@ -10,7 +10,7 @@
.. admonition:: Important
:class: tip
Docker 需要至少 16GB 显存的 GPU，并且所有加速功能默认启用。
Docker 需要至少 6GB 显存的 GPU，并且所有加速功能默认启用。
在运行此 Docker 容器之前，您可以使用以下命令检查您的设备是否支持 Docker 上的 CUDA 加速。
......@@ -20,9 +20,9 @@
.. code:: sh
wget https://github.com/opendatalab/MinerU/raw/master/Dockerfile
wget https://gcore.jsdelivr.net/gh/opendatalab/MinerU@master/docker/china/Dockerfile -O Dockerfile
docker build -t mineru:latest .
docker run --rm -it --gpus=all mineru:latest /bin/bash
docker run -it --name mineru --gpus=all mineru:latest /bin/bash -c "echo 'source /opt/mineru_venv/bin/activate' >> ~/.bashrc && exec bash"
magic-pdf --help
......@@ -42,12 +42,12 @@ Ubuntu 22.04 LTS
.. admonition:: Important
:class: tip
``CUDA Version`` 显示的版本号应 >=12.1，如显示的版本号小于12.1，请升级驱动
``CUDA Version`` 显示的版本号应 >= 12.4，如显示的版本号小于12.4，请升级驱动
.. code:: text
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 537.34 Driver Version: 537.34 CUDA Version: 12.2 |
| NVIDIA-SMI 570.133.07 Driver Version: 572.83 CUDA Version: 12.8 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
......@@ -66,7 +66,7 @@ Ubuntu 22.04 LTS
.. code:: bash
sudo apt-get update
sudo apt-get install nvidia-driver-545
sudo apt-get install nvidia-driver-570-server
安装专有驱动，安装完成后，重启电脑
......@@ -89,19 +89,17 @@ Ubuntu 22.04 LTS
4. 使用 conda 创建环境
---------------------
需指定 python 版本为3.10
.. code:: bash
conda create -n MinerU python=3.10
conda activate MinerU
conda create -n mineru 'python<3.13' -y
conda activate mineru
5. 安装应用
-----------
.. code:: bash
pip install -U magic-pdf[full] --extra-index-url https://wheels.myhloli.com -i https://mirrors.aliyun.com/pypi/simple
pip install -U magic-pdf[full] -i https://mirrors.aliyun.com/pypi/simple
.. admonition:: Important
:class: tip
......@@ -112,7 +110,7 @@ Ubuntu 22.04 LTS
magic-pdf --version
如果版本号小于0.7.0，请到issue中向我们反馈
如果版本号小于1.3.0，请到issue中向我们反馈
6. 下载模型
-----------
......@@ -136,7 +134,7 @@ Ubuntu 22.04 LTS
.. code:: bash
wget https://gcore.jsdelivr.net/gh/opendatalab/MinerU@master/demo/small_ocr.pdf
wget https://gcore.jsdelivr.net/gh/opendatalab/MinerU@master/demo/pdfs/small_ocr.pdf
magic-pdf -p small_ocr.pdf -o ./output
9. 测试CUDA加速
......@@ -163,27 +161,8 @@ Ubuntu 22.04 LTS
.. admonition:: Tip
:class: tip
CUDA 加速是否生效可以根据 log 中输出的各个阶段 cost 耗时来简单判断，通常情况下， ``layout detection cost`` 和 ``mfr time`` 应提速10倍以上。
10. 为 ocr 开启 cuda 加速
---------------------
**1.下载paddlepaddle-gpu, 安装完成后会自动开启ocr加速**
.. code:: bash
python -m pip install paddlepaddle-gpu==3.0.0b1 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/
CUDA 加速是否生效可以根据 log 中输出的各个阶段的耗时来简单判断，通常情况下，cuda应比cpu更快。
**2.运行以下命令测试ocr加速效果**
.. code:: bash
magic-pdf -p small_ocr.pdf -o ./output
.. admonition:: Tip
:class: tip
CUDA 加速是否生效可以根据 log 中输出的各个阶段 cost 耗时来简单判断，通常情况下， ``ocr cost`` 应提速10倍以上。
.. _windows_10_or_11_section:
......@@ -194,10 +173,12 @@ Windows 10/11
1. 安装 cuda 和 cuDNN
------------------
需要安装的版本 CUDA 11.8 + cuDNN 8.7.0
需要安装符合torch要求的cuda版本，torch目前支持11.8/12.4/12.6
- CUDA 11.8 https://developer.nvidia.com/cuda-11-8-0-download-archive
- cuDNN v8.7.0 (November 28th, 2022), for CUDA 11.x https://developer.nvidia.com/rdp/cudnn-archive
- CUDA 12.4 https://developer.nvidia.com/cuda-12-4-0-download-archive
- CUDA 12.6 https://developer.nvidia.com/cuda-12-6-0-download-archive
2. 安装 anaconda
---------------
......@@ -209,19 +190,17 @@ Windows 10/11
3. 使用 conda 创建环境
---------------------
需指定python版本为3.10
.. code:: bash
conda create -n MinerU python=3.10
conda activate MinerU
conda create -n mineru 'python<3.13' -y
conda activate mineru
4. 安装应用
-----------
.. code:: bash
pip install -U magic-pdf[full] --extra-index-url https://wheels.myhloli.com -i https://mirrors.aliyun.com/pypi/simple
pip install -U magic-pdf[full] -i https://mirrors.aliyun.com/pypi/simple
.. admonition:: Important
:class: tip
......@@ -232,7 +211,7 @@ Windows 10/11
magic-pdf --version
如果版本号小于0.7.0，请到issue中向我们反馈
如果版本号小于1.3.0，请到issue中向我们反馈
5. 下载模型
-----------
......@@ -256,7 +235,7 @@ Windows 10/11
.. code:: powershell
wget https://github.com/opendatalab/MinerU/raw/master/demo/small_ocr.pdf -O small_ocr.pdf
wget https://github.com/opendatalab/MinerU/raw/master/demo/pdfs/small_ocr.pdf -O small_ocr.pdf
magic-pdf -p small_ocr.pdf -o ./output
8. 测试 CUDA 加速
......@@ -264,22 +243,13 @@ Windows 10/11
如果您的显卡显存大于等于 **8GB**，可以进行以下流程，测试 CUDA 解析加速效果
**1.覆盖安装支持cuda的torch和torchvision**
.. code:: bash
pip install --force-reinstall torch==2.3.1 torchvision==0.18.1 --index-url https://download.pytorch.org/whl/cu118
**1.覆盖安装支持cuda的torch和torchvision**(请根据cuda版本选择合适的index-url，具体可参考[torch官网](https://pytorch.org/get-started/locally/))
.. admonition:: Important
:class: tip
务必在命令中指定以下版本
.. code:: bash
.. code:: bash
torch==2.3.1 torchvision==0.18.1
pip install --force-reinstall torch==2.6.0 torchvision==0.21.1 "numpy<2.0.0" --index-url https://download.pytorch.org/whl/cu124
这是我们支持的最高版本，如果不指定版本会自动安装更高版本导致程序无法运行
**2.修改【用户目录】中配置文件magic-pdf.json中”device-mode”的值**
......@@ -298,24 +268,5 @@ Windows 10/11
.. admonition:: Tip
:class: tip
CUDA 加速是否生效可以根据 log 中输出的各个阶段的耗时来简单判断，通常情况下， ``layout detection time`` 和 ``mfr time`` 应提速10倍以上。
9. 为 ocr 开启 cuda 加速
--------------------
**1.下载paddlepaddle-gpu, 安装完成后会自动开启ocr加速**
.. code:: bash
pip install paddlepaddle-gpu==2.6.1
**2.运行以下命令测试ocr加速效果**
.. code:: bash
magic-pdf -p small_ocr.pdf -o ./output
.. admonition:: Tip
:class: tip
CUDA 加速是否生效可以根据 log 中输出的各个阶段的耗时来简单判断，通常情况下， cuda会比cpu更快。
CUDA 加速是否生效可以根据 log 中输出的各个阶段 cost 耗时来简单判断，通常情况下， ``ocr time`` 应提速10倍以上。
next_docs/zh_cn/user_guide/install/install.rst
View file @ 01054426
......@@ -28,13 +28,13 @@
<td colspan="3" rowspan="2">操作系统</td>
</tr>
<tr>
<td>Ubuntu 22.04 LTS</td>
<td>Linux after 2019</td>
<td>Windows 10 / 11</td>
<td>macOS 11+</td>
</tr>
<tr>
<td colspan="3">CPU</td>
<td>x86_64(暂不支持ARM Linux)</td>
<td>x86_64 / arm64</td>
<td>x86_64(暂不支持ARM Windows)</td>
<td>x86_64 / arm64</td>
</tr>
......@@ -42,9 +42,13 @@
<td colspan="3">内存</td>
<td colspan="3">大于等于16GB，推荐32G以上</td>
</tr>
<tr>
<td colspan="3">存储空间</td>
<td colspan="3">大于等于20GB，推荐使用SSD以获得最佳性能</td>
</tr>
<tr>
<td colspan="3">python版本</td>
<td colspan="3">3.10 (请务必通过conda创建3.10虚拟环境)</td>
<td colspan="3">>=3.9,<=3.12</td>
</tr>
<tr>
<td colspan="3">Nvidia Driver 版本</td>
......@@ -54,22 +58,23 @@
</tr>
<tr>
<td colspan="3">CUDA环境</td>
<td>自动安装[12.1(pytorch)+11.8(paddle)]</td>
<td>11.8(手动安装)+cuDNN v8.7.0(手动安装)</td>
<td>11.8/12.4/12.6</td>
<td>11.8/12.4/12.6</td>
<td>None</td>
</tr>
<tr>
<td rowspan="2">GPU硬件支持列表</td>
<td colspan="2">最低要求 8G+显存</td>
<td colspan="2">3060ti/3070/4060<br>
8G显存可开启layout、公式识别和ocr加速</td>
<td rowspan="2">None</td>
<td colspan="3">CANN环境(NPU支持)</td>
<td>8.0+(Ascend 910b)</td>
<td>None</td>
<td>None</td>
</tr>
<tr>
<td colspan="2">推荐配置 10G+显存</td>
<td colspan="2">3080/3080ti/3090/3090ti/4070/4070ti/4070tisuper/4080/4090<br>
10G显存及以上可以同时开启layout、公式识别和ocr加速和表格识别加速<br>
</td>
<td rowspan="2">GPU/MPS 硬件支持列表</td>
<td colspan="2">显存6G以上</td>
<td colspan="2">
Volta(2017)及之后生产的全部带Tensor Core的GPU <br>
6G显存及以上</td>
<td rowspan="2">apple slicon</td>
</tr>
</table>
......@@ -79,9 +84,9 @@
.. code-block:: shell
conda create -n MinerU python=3.10
conda activate MinerU
pip install -U magic-pdf[full] --extra-index-url https://wheels.myhloli.com -i https://mirrors.aliyun.com/pypi/simple
conda create -n mineru 'python<3.13' -y
conda activate mineru
pip install -U "magic-pdf[full]" -i https://mirrors.aliyun.com/pypi/simple
下载模型权重文件
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment