README.md 16.9 KB
Newer Older
xuchao's avatar
xuchao committed
1
2
<div align="center" xmlns="http://www.w3.org/1999/html">
<!-- logo -->
徐超's avatar
徐超 committed
3
<p align="center">
drunkpig's avatar
drunkpig committed
4
  <img src="docs/images/MinerU-logo.png" width="300px" style="vertical-align:middle;">
徐超's avatar
徐超 committed
5
6
</p>

xuchao's avatar
xuchao committed
7
<!-- icon -->
8

赵小蒙's avatar
赵小蒙 committed
9
10
11
[![stars](https://img.shields.io/github/stars/opendatalab/MinerU.svg)](https://github.com/opendatalab/MinerU)
[![forks](https://img.shields.io/github/forks/opendatalab/MinerU.svg)](https://github.com/opendatalab/MinerU)
[![open issues](https://img.shields.io/github/issues-raw/opendatalab/MinerU)](https://github.com/opendatalab/MinerU/issues)
myhloli's avatar
myhloli committed
12
13
14
15
[![issue resolution](https://img.shields.io/github/issues-closed-raw/opendatalab/MinerU)](https://github.com/opendatalab/MinerU/issues)
[![PyPI version](https://badge.fury.io/py/magic-pdf.svg)](https://badge.fury.io/py/magic-pdf)
[![Downloads](https://static.pepy.tech/badge/magic-pdf)](https://pepy.tech/project/magic-pdf)
[![Downloads](https://static.pepy.tech/badge/magic-pdf/month)](https://pepy.tech/project/magic-pdf)
xuchao's avatar
xuchao committed
16
<a href="https://trendshift.io/repositories/11174" target="_blank"><img src="https://trendshift.io/api/badge/repositories/11174" alt="opendatalab%2FMinerU | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
drunkpig's avatar
drunkpig committed
17

xuchao's avatar
xuchao committed
18
<!-- language -->
19

xuchao's avatar
xuchao committed
20
[English](README.md) | [简体中文](README_zh-CN.md)
赵小蒙's avatar
赵小蒙 committed
21

xuchao's avatar
xuchao committed
22
<!-- hot link -->
23

徐超's avatar
徐超 committed
24
<p align="center">
xuchao's avatar
xuchao committed
25
<a href="https://github.com/opendatalab/PDF-Extract-Kit">PDF-Extract-Kit: High-Quality PDF Extraction Toolkit</a>🔥🔥🔥
徐超's avatar
徐超 committed
26
27
</p>

xuchao's avatar
xuchao committed
28
<!-- join us -->
29

徐超's avatar
徐超 committed
30
<p align="center">
xuchao's avatar
xuchao committed
31
    👋 join us on <a href="https://discord.gg/Tdedn9GTXq" target="_blank">Discord</a> and <a href="https://cdn.vansin.top/internlm/mineru.jpg" target="_blank">WeChat</a>
徐超's avatar
徐超 committed
32
</p>
赵小蒙's avatar
赵小蒙 committed
33

xuchao's avatar
xuchao committed
34
</div>
赵小蒙's avatar
赵小蒙 committed
35

xuchao's avatar
xuchao committed
36
# Changelog
37
- 2024/08/30: Version 0.7.1 released, add paddle tablemaster table recognition option
xuchao's avatar
xuchao committed
38
39
40
41
42
- 2024/08/09: Version 0.7.0b1 released, simplified installation process, added table recognition functionality
- 2024/08/01: Version 0.6.2b1 released, optimized dependency conflict issues and installation documentation
- 2024/07/05: Initial open-source release

<!-- TABLE OF CONTENT -->
43

xuchao's avatar
xuchao committed
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
<details open="open">
  <summary><h2 style="display: inline-block">Table of Contents</h2></summary>
  <ol>
    <li>
      <a href="#mineru">MinerU</a>
      <ul>
        <li><a href="#project-introduction">Project Introduction</a></li>
        <li><a href="#key-features">Key Features</a></li>
        <li><a href="#quick-start">Quick Start</a>
            <ul>
            <li><a href="#online-demo">Online Demo</a></li>
            <li><a href="#quick-cpu-demo">Quick CPU Demo</a></li>
            <li><a href="#using-gpu">Using GPU</a></li>
            </ul>
        </li>
        <li><a href="#usage">Usage</a>
            <ul>
            <li><a href="#command-line">Command Line</a></li>
            <li><a href="#api">API</a></li>
            <li><a href="#development-guide">Development Guide</a></li>
            </ul>
        </li>
      </ul>
    </li>
    <li><a href="#todo">TODO</a></li>
    <li><a href="#known-issues">Known Issues</a></li>
    <li><a href="#faq">FAQ</a></li>
    <li><a href="#all-thanks-to-our-contributors">All Thanks To Our Contributors</a></li>
    <li><a href="#license-information">License Information</a></li>
    <li><a href="#acknowledgments">Acknowledgments</a></li>
    <li><a href="#citation">Citation</a></li>
    <li><a href="#star-history">Star History</a></li>
    <li><a href="#magic-doc">Magic-doc</a></li>
    <li><a href="#magic-html">Magic-html</a></li>
    <li><a href="#links">Links</a></li>
  </ol>
</details>

# MinerU
83

xuchao's avatar
xuchao committed
84
## Project Introduction
85

xuchao's avatar
xuchao committed
86
87
88
MinerU is a tool that converts PDFs into machine-readable formats (e.g., markdown, JSON), allowing for easy extraction into any format.
MinerU was born during the pre-training process of [InternLM](https://github.com/InternLM/InternLM). We focus on solving symbol conversion issues in scientific literature and hope to contribute to technological development in the era of large models.
Compared to well-known commercial products, MinerU is still young. If you encounter any issues or if the results are not as expected, please submit an issue on [issue](https://github.com/opendatalab/MinerU/issues) and **attach the relevant PDF**.
myhloli's avatar
myhloli committed
89

Xiaomeng Zhao's avatar
Xiaomeng Zhao committed
90
https://github.com/user-attachments/assets/4bea02c9-6d54-4cd6-97ed-dff14340982c
myhloli's avatar
myhloli committed
91

xuchao's avatar
xuchao committed
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
## Key Features

- Removes elements such as headers, footers, footnotes, and page numbers while maintaining semantic continuity
- Outputs text in a human-readable order from multi-column documents
- Retains the original structure of the document, including titles, paragraphs, and lists
- Extracts images, image captions, tables, and table captions
- Automatically recognizes formulas in the document and converts them to LaTeX
- Automatically recognizes tables in the document and converts them to LaTeX
- Automatically detects and enables OCR for corrupted PDFs
- Supports both CPU and GPU environments
- Supports Windows, Linux, and Mac platforms

## Quick Start

If you encounter any installation issues, please first consult the <a href="#faq">FAQ</a>. </br>
If the parsing results are not as expected, refer to the <a href="#known-issues">Known Issues</a>. </br>
There are three different ways to experience MinerU:
109

xuchao's avatar
xuchao committed
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
- [Online Demo (No Installation Required)](#online-demo)
- [Quick CPU Demo (Windows, Linux, Mac)](#quick-cpu-demo)
- [Linux/Windows + CUDA](#Using-GPU)

**⚠️ Pre-installation Notice—Hardware and Software Environment Support**

To ensure the stability and reliability of the project, we only optimize and test for specific hardware and software environments during development. This ensures that users deploying and running the project on recommended system configurations will get the best performance with the fewest compatibility issues.

By focusing resources on the mainline environment, our team can more efficiently resolve potential bugs and develop new features.

In non-mainline environments, due to the diversity of hardware and software configurations, as well as third-party dependency compatibility issues, we cannot guarantee 100% project availability. Therefore, for users who wish to use this project in non-recommended environments, we suggest carefully reading the documentation and FAQ first. Most issues already have corresponding solutions in the FAQ. We also encourage community feedback to help us gradually expand support.

<table>
    <tr>
        <td colspan="3" rowspan="2">Operating System</td>
    </tr>
    <tr>
        <td>Ubuntu 22.04 LTS</td>
        <td>Windows 10 / 11</td>
        <td>macOS 11+</td>
    </tr>
    <tr>
        <td colspan="3">CPU</td>
        <td>x86_64</td>
        <td>x86_64</td>
        <td>x86_64 / arm64</td>
    </tr>
    <tr>
        <td colspan="3">Memory</td>
        <td colspan="3">16GB or more, recommended 32GB+</td>
    </tr>
    <tr>
        <td colspan="3">Python Version</td>
        <td colspan="3">3.10</td>
    </tr>
    <tr>
        <td colspan="3">Nvidia Driver Version</td>
        <td>latest (Proprietary Driver)</td>
        <td>latest</td>
        <td>None</td>
    </tr>
    <tr>
        <td colspan="3">CUDA Environment</td>
        <td>Automatic installation [12.1 (pytorch) + 11.8 (paddle)]</td>
        <td>11.8 (manual installation) + cuDNN v8.7.0 (manual installation)</td>
        <td>None</td>
    </tr>
    <tr>
        <td rowspan="2">GPU Hardware Support List</td>
        <td colspan="2">Minimum Requirement 8G+ VRAM</td>
        <td colspan="2">3060ti/3070/3080/3080ti/4060/4070/4070ti<br>
        8G VRAM only enables layout and formula recognition acceleration</td>
        <td rowspan="2">None</td>
    </tr>
    <tr>
        <td colspan="2">Recommended Configuration 16G+ VRAM</td>
        <td colspan="2">3090/3090ti/4070ti super/4080/4090<br>
sfk's avatar
sfk committed
167
168
169
        16G or more can enable layout, formula recognition, and OCR acceleration simultaneously<br>
        24G or more can enable layout, formula recognition, OCR acceleration and table recognition simultaneously
        </td>
xuchao's avatar
xuchao committed
170
171
172
173
174
175
176
177
178
179
    </tr>
</table>

### Online Demo

[Click here for the online demo](https://opendatalab.com/OpenSourceTools/Extractor/PDF)

### Quick CPU Demo

#### 1. Install magic-pdf
180

181
182
183
```bash
conda create -n MinerU python=3.10
conda activate MinerU
184
pip install -U magic-pdf[full] --extra-index-url https://wheels.myhloli.com
185
```
186

xuchao's avatar
xuchao committed
187
188
189
#### 2. Download model weight files

Refer to [How to Download Model Files](docs/how_to_download_models_en.md) for detailed instructions.
190

xuchao's avatar
xuchao committed
191
> ❗️After downloading the models, please make sure to verify the completeness of the model files.
192
>
xuchao's avatar
xuchao committed
193
194
195
> Check if the model file sizes match the description on the webpage. If possible, use sha256 to verify the integrity of the files.

#### 3. Copy and configure the template file
196

xuchao's avatar
xuchao committed
197
You can find the `magic-pdf.template.json` template configuration file in the root directory of the repository.
198

xuchao's avatar
xuchao committed
199
> ❗️Make sure to execute the following command to copy the configuration file to your **user directory**; otherwise, the program will not run.
200
>
xuchao's avatar
xuchao committed
201
> The user directory for Windows is `C:\Users\YourUsername`, for Linux it is `/home/YourUsername`, and for macOS it is `/Users/YourUsername`.
202

赵小蒙's avatar
赵小蒙 committed
203
```bash
赵小蒙's avatar
赵小蒙 committed
204
cp magic-pdf.template.json ~/magic-pdf.json
205
```
206

xuchao's avatar
xuchao committed
207
Find the `magic-pdf.json` file in your user directory and configure the "models-dir" path to point to the directory where the model weight files were downloaded in [Step 2](#2-download-model-weight-files).
208

xuchao's avatar
xuchao committed
209
210
211
> ❗️Make sure to correctly configure the **absolute path** to the model weight files directory, otherwise the program will not run because it can't find the model files.
>
> On Windows, this path should include the drive letter and all backslashes (`\`) in the path should be replaced with forward slashes (`/`) to avoid syntax errors in the JSON file due to escape sequences.
212
>
xuchao's avatar
xuchao committed
213
> For example: If the models are stored in the "models" directory at the root of the D drive, the "model-dir" value should be `D:/models`.
214

215
216
```json
{
xuchao's avatar
xuchao committed
217
218
219
  // other config
  "models-dir": "D:/models",
  "table-config": {
220
        "model": "TableMaster", // Another option of this value is 'struct_eqtable'
xuchao's avatar
xuchao committed
221
222
223
        "is_table_recog_enable": false, // Table recognition is disabled by default, modify this value to enable it
        "max_time": 400
    }
224
225
226
}
```

xuchao's avatar
xuchao committed
227
### Using GPU
228

xuchao's avatar
xuchao committed
229
If your device supports CUDA and meets the GPU requirements of the mainline environment, you can use GPU acceleration. Please select the appropriate guide based on your system:
230

xuchao's avatar
xuchao committed
231
232
- [Ubuntu 22.04 LTS + GPU](docs/README_Ubuntu_CUDA_Acceleration_en_US.md)
- [Windows 10/11 + GPU](docs/README_Windows_CUDA_Acceleration_en_US.md)
233
234
235
236
237
238
239
240
- Quick Deployment with Docker
    > Docker requires a GPU with at least 16GB of VRAM, and all acceleration features are enabled by default.
  ```bash
  wget https://github.com/opendatalab/MinerU/raw/master/Dockerfile
  docker build -t mineru:0.7.0b1 .
  docker run --rm -it --gpus=all mineru:0.7.0b1 /bin/bash
  magic-pdf --help
  ```
241

xuchao's avatar
xuchao committed
242
## Usage
243

xuchao's avatar
xuchao committed
244
### Command Line
245
246

```bash
xuchao's avatar
xuchao committed
247
248
249
250
251
252
253
magic-pdf --help
Usage: magic-pdf [OPTIONS]

Options:
  -v, --version                display the version and exit
  -p, --path PATH              local pdf filepath or directory  [required]
  -o, --output-dir TEXT        output local directory
254
  -m, --method [ocr|txt|auto]  the method for parsing pdf.
xuchao's avatar
xuchao committed
255
256
257
258
                               ocr: using ocr technique to extract information from pdf,
                               txt: suitable for the text-based pdf only and outperform ocr,
                               auto: automatically choose the best method for parsing pdf
                                  from ocr and txt.
259
                               without method specified, auto will be used by default.
xuchao's avatar
xuchao committed
260
261
262
263
264
265
266
267
  --help                       Show this message and exit.


## show version
magic-pdf -v

## command line example
magic-pdf -p {some_pdf} -o {some_output_dir} -m auto
268
269
```

xuchao's avatar
xuchao committed
270
271
272
273
`{some_pdf}` can be a single PDF file or a directory containing multiple PDFs.
The results will be saved in the `{some_output_dir}` directory. The output file list is as follows:

```text
274
275
276
277
278
279
280
├── some_pdf.md                          # markdown file
├── images                               # directory for storing images
├── some_pdf_layout.pdf                  # layout diagram
├── some_pdf_middle.json                 # MinerU intermediate processing result
├── some_pdf_model.json                  # model inference result
├── some_pdf_origin.pdf                  # original PDF file
└── some_pdf_spans.pdf                   # smallest granularity bbox position information diagram
281
282
```

xuchao's avatar
xuchao committed
283
For more information about the output files, please refer to the [Output File Description](docs/output_file_en_us.md).
284

xuchao's avatar
xuchao committed
285
### API
赵小蒙's avatar
赵小蒙 committed
286

xuchao's avatar
xuchao committed
287
Processing files from local disk
288

赵小蒙's avatar
赵小蒙 committed
289
290
291
```python
image_writer = DiskReaderWriter(local_image_dir)
image_dir = str(os.path.basename(local_image_dir))
292
jso_useful_key = {"_pdf_type": "", "model_list": []}
赵小蒙's avatar
赵小蒙 committed
293
294
pipe = UNIPipe(pdf_bytes, jso_useful_key, image_writer)
pipe.pipe_classify()
xuchao's avatar
xuchao committed
295
pipe.pipe_analyze()
赵小蒙's avatar
赵小蒙 committed
296
297
298
299
pipe.pipe_parse()
md_content = pipe.pipe_mk_markdown(image_dir, drop_mode="none")
```

xuchao's avatar
xuchao committed
300
Processing files from object storage
301

赵小蒙's avatar
赵小蒙 committed
302
303
304
305
306
```python
s3pdf_cli = S3ReaderWriter(pdf_ak, pdf_sk, pdf_endpoint)
image_dir = "s3://img_bucket/"
s3image_cli = S3ReaderWriter(img_ak, img_sk, img_endpoint, parent_path=image_dir)
pdf_bytes = s3pdf_cli.read(s3_pdf_path, mode=s3pdf_cli.MODE_BIN)
307
jso_useful_key = {"_pdf_type": "", "model_list": []}
赵小蒙's avatar
赵小蒙 committed
308
309
pipe = UNIPipe(pdf_bytes, jso_useful_key, s3image_cli)
pipe.pipe_classify()
xuchao's avatar
xuchao committed
310
pipe.pipe_analyze()
赵小蒙's avatar
赵小蒙 committed
311
312
313
314
pipe.pipe_parse()
md_content = pipe.pipe_mk_markdown(image_dir, drop_mode="none")
```

xuchao's avatar
xuchao committed
315
For detailed implementation, refer to:
316

xuchao's avatar
xuchao committed
317
318
- [demo.py Simplest Processing Method](demo/demo.py)
- [magic_pdf_parse_main.py More Detailed Processing Workflow](demo/magic_pdf_parse_main.py)
赵小蒙's avatar
赵小蒙 committed
319

xuchao's avatar
xuchao committed
320
### Development Guide
赵小蒙's avatar
赵小蒙 committed
321

xuchao's avatar
xuchao committed
322
TODO
赵小蒙's avatar
赵小蒙 committed
323

xuchao's avatar
xuchao committed
324
# TODO
赵小蒙's avatar
赵小蒙 committed
325

xuchao's avatar
xuchao committed
326
327
328
329
330
- [ ] Semantic-based reading order
- [ ] List recognition within the text
- [ ] Code block recognition within the text
- [ ] Table of contents recognition
- [x] Table recognition
331
- [ ] [Chemical formula recognition](docs/chemical_knowledge_introduction/introduction.pdf)
xuchao's avatar
xuchao committed
332
- [ ] Geometric shape recognition
赵小蒙's avatar
赵小蒙 committed
333

xuchao's avatar
xuchao committed
334
# Known Issues
335

xuchao's avatar
xuchao committed
336
337
338
339
340
341
- Reading order is segmented based on rules, which can cause disordered sequences in some cases
- Vertical text is not supported
- Lists, code blocks, and table of contents are not yet supported in the layout model
- Comic books, art books, elementary school textbooks, and exercise books are not well-parsed yet
- Enabling OCR may produce better results in PDFs with a high density of formulas
- If you are processing PDFs with a large number of formulas, it is strongly recommended to enable the OCR function. When using PyMuPDF to extract text, overlapping text lines can occur, leading to inaccurate formula insertion positions.
赵小蒙's avatar
赵小蒙 committed
342

343

赵小蒙's avatar
赵小蒙 committed
344

xuchao's avatar
xuchao committed
345
# FAQ
346

xuchao's avatar
xuchao committed
347
[FAQ in Chinese](docs/FAQ_zh_cn.md)
赵小蒙's avatar
赵小蒙 committed
348

xuchao's avatar
xuchao committed
349
[FAQ in English](docs/FAQ_en_us.md)
赵小蒙's avatar
赵小蒙 committed
350

赵小蒙's avatar
赵小蒙 committed
351
352
# All Thanks To Our Contributors

353
<a href="https://github.com/opendatalab/MinerU/graphs/contributors">
赵小蒙's avatar
赵小蒙 committed
354
355
356
357
358
359
360
  <img src="https://contrib.rocks/image?repo=opendatalab/MinerU" />
</a>

# License Information

[LICENSE.md](LICENSE.md)

xuchao's avatar
xuchao committed
361
This project currently uses PyMuPDF to achieve advanced functionality. However, since it adheres to the AGPL license, it may impose restrictions on certain usage scenarios. In future iterations, we plan to explore and replace it with a more permissive PDF processing library to enhance user-friendliness and flexibility.
赵小蒙's avatar
赵小蒙 committed
362
363

# Acknowledgments
364

xuchao's avatar
xuchao committed
365
366
- [PDF-Extract-Kit](https://github.com/opendatalab/PDF-Extract-Kit)
- [StructEqTable](https://github.com/UniModal4Reasoning/StructEqTable-Deploy)
赵小蒙's avatar
赵小蒙 committed
367
368
- [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)
- [PyMuPDF](https://github.com/pymupdf/PyMuPDF)
赵小蒙's avatar
赵小蒙 committed
369
370
- [fast-langdetect](https://github.com/LlmKira/fast-langdetect)
- [pdfminer.six](https://github.com/pdfminer/pdfminer.six)
赵小蒙's avatar
赵小蒙 committed
371

赵小蒙's avatar
赵小蒙 committed
372
373
374
# Citation

```bibtex
Conghui He's avatar
Conghui He committed
375
376
377
378
379
380
381
@article{he2024opendatalab,
  title={Opendatalab: Empowering general artificial intelligence with open datasets},
  author={He, Conghui and Li, Wei and Jin, Zhenjiang and Xu, Chao and Wang, Bin and Lin, Dahua},
  journal={arXiv preprint arXiv:2407.13773},
  year={2024}
}

赵小蒙's avatar
赵小蒙 committed
382
383
384
385
386
387
388
389
390
@misc{2024mineru,
    title={MinerU: A One-stop, Open-source, High-quality Data Extraction Tool},
    author={MinerU Contributors},
    howpublished = {\url{https://github.com/opendatalab/MinerU}},
    year={2024}
}
```

# Star History
赵小蒙's avatar
赵小蒙 committed
391

赵小蒙's avatar
赵小蒙 committed
392
393
394
395
396
397
<a>
 <picture>
   <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=opendatalab/MinerU&type=Date&theme=dark" />
   <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=opendatalab/MinerU&type=Date" />
   <img alt="Star History Chart" src="https://api.star-history.com/svg?repos=opendatalab/MinerU&type=Date" />
 </picture>
myhloli's avatar
myhloli committed
398
</a>
qiangqiang199's avatar
qiangqiang199 committed
399

xuchao's avatar
xuchao committed
400
# Magic-doc
401

xuchao's avatar
xuchao committed
402
403
404
[Magic-Doc](https://github.com/InternLM/magic-doc) Fast speed ppt/pptx/doc/docx/pdf extraction tool

# Magic-html
405

xuchao's avatar
xuchao committed
406
407
[Magic-HTML](https://github.com/opendatalab/magic-html) Mixed web page extraction tool

qiangqiang199's avatar
qiangqiang199 committed
408
# Links
xuchao's avatar
xuchao committed
409

qiangqiang199's avatar
qiangqiang199 committed
410
411
- [LabelU (A Lightweight Multi-modal Data Annotation Tool)](https://github.com/opendatalab/labelU)
- [LabelLLM (An Open-source LLM Dialogue Annotation Platform)](https://github.com/opendatalab/LabelLLM)
qiangqiang199's avatar
qiangqiang199 committed
412
- [PDF-Extract-Kit (A Comprehensive Toolkit for High-Quality PDF Content Extraction)](https://github.com/opendatalab/PDF-Extract-Kit)