readme_en.md 9.61 KB
Newer Older
1
# Server-side C++ Inference
littletomatodonkey's avatar
littletomatodonkey committed
2

fanruinet's avatar
fanruinet committed
3
4
5
This chapter introduces the C++ deployment steps of the PaddleOCR model. The corresponding Python predictive deployment method refers to [document](../../doc/doc_ch/inference.md).
C++ is better than python in terms of performance. Therefore, in CPU and GPU deployment scenarios, C++ deployment is mostly used.
This section will introduce how to configure the C++ environment and deploy PaddleOCR in Linux (CPU\GPU) environment. For Windows deployment please refer to [Windows](./docs/windows_vs2019_build.md) compilation guidelines.
littletomatodonkey's avatar
littletomatodonkey committed
6
7


8
## 1. Prepare the Environment
littletomatodonkey's avatar
littletomatodonkey committed
9
10
11
12
13
14

### Environment

- Linux, docker is recommended.


15
### 1.1 Compile OpenCV
littletomatodonkey's avatar
littletomatodonkey committed
16

fanruinet's avatar
fanruinet committed
17
* First of all, you need to download the source code compiled package in the Linux environment from the OpenCV official website. Taking OpenCV 3.4.7 as an example, the download command is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
18

littletomatodonkey's avatar
littletomatodonkey committed
19
```bash
WenmuZhou's avatar
WenmuZhou committed
20
cd deploy/cpp_infer
littletomatodonkey's avatar
littletomatodonkey committed
21
22
wget https://paddleocr.bj.bcebos.com/libs/opencv/opencv-3.4.7.tar.gz
tar -xf opencv-3.4.7.tar.gz
littletomatodonkey's avatar
littletomatodonkey committed
23
24
```

fanruinet's avatar
fanruinet committed
25
Finally, you will see the folder of `opencv-3.4.7/` in the current directory.
littletomatodonkey's avatar
littletomatodonkey committed
26

fanruinet's avatar
fanruinet committed
27
* Compile OpenCV, the OpenCV source path (`root_path`) and installation path (`install_path`) should be set by yourself. Enter the OpenCV source code path and compile it in the following way.
littletomatodonkey's avatar
littletomatodonkey committed
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59


```shell
root_path=your_opencv_root_path
install_path=${root_path}/opencv3

rm -rf build
mkdir build
cd build

cmake .. \
    -DCMAKE_INSTALL_PREFIX=${install_path} \
    -DCMAKE_BUILD_TYPE=Release \
    -DBUILD_SHARED_LIBS=OFF \
    -DWITH_IPP=OFF \
    -DBUILD_IPP_IW=OFF \
    -DWITH_LAPACK=OFF \
    -DWITH_EIGEN=OFF \
    -DCMAKE_INSTALL_LIBDIR=lib64 \
    -DWITH_ZLIB=ON \
    -DBUILD_ZLIB=ON \
    -DWITH_JPEG=ON \
    -DBUILD_JPEG=ON \
    -DWITH_PNG=ON \
    -DBUILD_PNG=ON \
    -DWITH_TIFF=ON \
    -DBUILD_TIFF=ON

make -j
make install
```

fanruinet's avatar
fanruinet committed
60
In the above commands, `root_path` is the downloaded OpenCV source code path, and `install_path` is the installation path of OpenCV. After `make install` is completed, the OpenCV header file and library file will be generated in this folder for later OCR source code compilation.
littletomatodonkey's avatar
littletomatodonkey committed
61
62
63



fanruinet's avatar
fanruinet committed
64
The final file structure under the OpenCV installation path is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
65
66
67
68
69
70
71
72
73
74

```
opencv3/
|-- bin
|-- include
|-- lib
|-- lib64
|-- share
```

75
### 1.2 Compile or Download or the Paddle Inference Library
littletomatodonkey's avatar
littletomatodonkey committed
76
77
78

* There are 2 ways to obtain the Paddle inference library, described in detail below.

littletomatodonkey's avatar
littletomatodonkey committed
79
#### 1.2.1 Direct download and installation
littletomatodonkey's avatar
littletomatodonkey committed
80

WenmuZhou's avatar
WenmuZhou committed
81
[Paddle inference library official website](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html#linux). You can review and select the appropriate version of the inference library on the official website.
littletomatodonkey's avatar
littletomatodonkey committed
82
83


fanruinet's avatar
fanruinet committed
84
* After downloading, use the following command to extract files.
littletomatodonkey's avatar
littletomatodonkey committed
85
86
87
88
89

```
tar -xf paddle_inference.tgz
```

fanruinet's avatar
fanruinet committed
90
Finally you will see the the folder of `paddle_inference/` in the current path.
littletomatodonkey's avatar
littletomatodonkey committed
91

fanruinet's avatar
fanruinet committed
92
93
94
#### 1.2.2 Compile the inference source code
* If you want to get the latest Paddle inference library features, you can download the latest code from Paddle GitHub repository and compile the inference library from the source code. It is recommended to download the inference library with paddle version greater than or equal to 2.0.1.
* You can refer to [Paddle inference library] (https://www.paddlepaddle.org.cn/documentation/docs/en/advanced_guide/inference_deployment/inference/build_and_install_lib_en.html) to get the Paddle source code from GitHub, and then compile To generate the latest inference library. The method of using git to access the code is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
95
96
97
98


```shell
git clone https://github.com/PaddlePaddle/Paddle.git
LDOUBLEV's avatar
LDOUBLEV committed
99
git checkout develop
littletomatodonkey's avatar
littletomatodonkey committed
100
101
```

fanruinet's avatar
fanruinet committed
102
* Enter the Paddle directory and run the following commands to compile the paddle inference library.
littletomatodonkey's avatar
littletomatodonkey committed
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121

```shell
rm -rf build
mkdir build
cd build

cmake  .. \
    -DWITH_CONTRIB=OFF \
    -DWITH_MKL=ON \
    -DWITH_MKLDNN=ON  \
    -DWITH_TESTING=OFF \
    -DCMAKE_BUILD_TYPE=Release \
    -DWITH_INFERENCE_API_TEST=OFF \
    -DON_INFER=ON \
    -DWITH_PYTHON=ON
make -j
make inference_lib_dist
```

LDOUBLEV's avatar
LDOUBLEV committed
122
For more compilation parameter options, please refer to the [document](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0/guides/05_inference_deployment/inference/build_and_install_lib_cn.html#congyuanmabianyi).
littletomatodonkey's avatar
littletomatodonkey committed
123
124


LDOUBLEV's avatar
LDOUBLEV committed
125
* After the compilation process, you can see the following files in the folder of `build/paddle_inference_install_dir/`.
littletomatodonkey's avatar
littletomatodonkey committed
126
127

```
LDOUBLEV's avatar
LDOUBLEV committed
128
build/paddle_inference_install_dir/
littletomatodonkey's avatar
littletomatodonkey committed
129
130
131
132
133
134
|-- CMakeCache.txt
|-- paddle
|-- third_party
|-- version.txt
```

fanruinet's avatar
fanruinet committed
135
`paddle` is the Paddle library required for C++ prediction later, and `version.txt` contains the version information of the current inference library.
littletomatodonkey's avatar
littletomatodonkey committed
136
137


138
## 2. Compile and Run the Demo
littletomatodonkey's avatar
littletomatodonkey committed
139
140
141

### 2.1 Export the inference model

fanruinet's avatar
fanruinet committed
142
* You can refer to [Model inference](../../doc/doc_ch/inference.md) and export the inference model. After the model is exported, assuming it is placed in the `inference` directory, the directory structure is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
143
144
145
146

```
inference/
|-- det_db
MissPenguin's avatar
MissPenguin committed
147
148
|   |--inference.pdiparams
|   |--inference.pdmodel
littletomatodonkey's avatar
littletomatodonkey committed
149
|-- rec_rcnn
MissPenguin's avatar
MissPenguin committed
150
151
|   |--inference.pdiparams
|   |--inference.pdmodel
littletomatodonkey's avatar
littletomatodonkey committed
152
153
154
155
156
157
158
159
160
```


### 2.2 Compile PaddleOCR C++ inference demo


* The compilation commands are as follows. The addresses of Paddle C++ inference library, opencv and other Dependencies need to be replaced with the actual addresses on your own machines.

```shell
MissPenguin's avatar
MissPenguin committed
161
sh tools/build.sh
littletomatodonkey's avatar
littletomatodonkey committed
162
163
```

MissPenguin's avatar
MissPenguin committed
164
Specifically, you should modify the paths in `tools/build.sh`. The related content is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
165
166
167
168
169
170
171
172

```shell
OPENCV_DIR=your_opencv_dir
LIB_DIR=your_paddle_inference_dir
CUDA_LIB_DIR=your_cuda_lib_dir
CUDNN_LIB_DIR=your_cudnn_lib_dir
```

fanruinet's avatar
fanruinet committed
173
`OPENCV_DIR` is the OpenCV installation path; `LIB_DIR` is the download (`paddle_inference` folder)
LDOUBLEV's avatar
LDOUBLEV committed
174
or the generated Paddle inference library path (`build/paddle_inference_install_dir` folder);
fanruinet's avatar
fanruinet committed
175
`CUDA_LIB_DIR` is the CUDA library file path, in docker; it is `/usr/local/cuda/lib64`; `CUDNN_LIB_DIR` is the cuDNN library file path, in docker it is `/usr/lib/x86_64-linux-gnu/`.
littletomatodonkey's avatar
littletomatodonkey committed
176
177


MissPenguin's avatar
MissPenguin committed
178
* After the compilation is completed, an executable file named `ppocr` will be generated in the `build` folder.
littletomatodonkey's avatar
littletomatodonkey committed
179
180
181


### Run the demo
fanruinet's avatar
fanruinet committed
182
Execute the built executable file:
MissPenguin's avatar
MissPenguin committed
183
184
```shell
./build/ppocr <mode> [--param1] [--param2] [...]
185
```
fanruinet's avatar
fanruinet committed
186
187
188
189
190
191
192
193
194
`mode` is a required parameter,and the valid values are

mode value | Model used
-----|------
det  | Detection only
rec  | Recognition only
system | End-to-end system

Specifically,
MissPenguin's avatar
MissPenguin committed
195
196

##### 1. run det demo:
littletomatodonkey's avatar
littletomatodonkey committed
197
```shell
MissPenguin's avatar
MissPenguin committed
198
./build/ppocr det \
MissPenguin's avatar
MissPenguin committed
199
    --det_model_dir=inference/ch_ppocr_mobile_v2.0_det_infer \
MissPenguin's avatar
MissPenguin committed
200
    --image_dir=../../doc/imgs/12.jpg
littletomatodonkey's avatar
littletomatodonkey committed
201
```
MissPenguin's avatar
MissPenguin committed
202
##### 2. run rec demo:
MissPenguin's avatar
MissPenguin committed
203
```shell
MissPenguin's avatar
MissPenguin committed
204
./build/ppocr rec \
MissPenguin's avatar
MissPenguin committed
205
    --rec_model_dir=inference/ch_ppocr_mobile_v2.0_rec_infer \
MissPenguin's avatar
MissPenguin committed
206
    --image_dir=../../doc/imgs_words/ch/
zhoujun's avatar
zhoujun committed
207
```
MissPenguin's avatar
MissPenguin committed
208
##### 3. run system demo:
MissPenguin's avatar
MissPenguin committed
209
210
```shell
# without text direction classifier
MissPenguin's avatar
MissPenguin committed
211
./build/ppocr system \
MissPenguin's avatar
MissPenguin committed
212
213
    --det_model_dir=inference/ch_ppocr_mobile_v2.0_det_infer \
    --rec_model_dir=inference/ch_ppocr_mobile_v2.0_rec_infer \
MissPenguin's avatar
MissPenguin committed
214
215
    --image_dir=../../doc/imgs/12.jpg
# with text direction classifier
MissPenguin's avatar
MissPenguin committed
216
./build/ppocr system \
MissPenguin's avatar
MissPenguin committed
217
218
219
220
    --det_model_dir=inference/ch_ppocr_mobile_v2.0_det_infer \
    --use_angle_cls=true \
    --cls_model_dir=inference/ch_ppocr_mobile_v2.0_cls_infer \
    --rec_model_dir=inference/ch_ppocr_mobile_v2.0_rec_infer \
MissPenguin's avatar
MissPenguin committed
221
222
223
    --image_dir=../../doc/imgs/12.jpg
```

fanruinet's avatar
fanruinet committed
224
More parameters are as follows,
MissPenguin's avatar
MissPenguin committed
225

fanruinet's avatar
fanruinet committed
226
- Common parameters
MissPenguin's avatar
MissPenguin committed
227

MissPenguin's avatar
MissPenguin committed
228
229
230
231
232
233
|parameter|data type|default|meaning|
| --- | --- | --- | --- |
|use_gpu|bool|false|Whether to use GPU|
|gpu_id|int|0|GPU id when use_gpu is true|
|gpu_mem|int|4000|GPU memory requested|
|cpu_math_library_num_threads|int|10|Number of threads when using CPU inference. When machine cores is enough, the large the value, the faster the inference speed|
WenmuZhou's avatar
WenmuZhou committed
234
|enable_mkldnn|bool|true|Whether to use mkdlnn library|
MissPenguin's avatar
MissPenguin committed
235

fanruinet's avatar
fanruinet committed
236
- Detection related parameters
MissPenguin's avatar
MissPenguin committed
237
238
239

|parameter|data type|default|meaning|
| --- | --- | --- | --- |
MissPenguin's avatar
MissPenguin committed
240
241
242
243
244
245
246
|det_model_dir|string|-|Address of detection inference model|
|max_side_len|int|960|Limit the maximum image height and width to 960|
|det_db_thresh|float|0.3|Used to filter the binarized image of DB prediction, setting 0.-0.3 has no obvious effect on the result|
|det_db_box_thresh|float|0.5|DB post-processing filter box threshold, if there is a missing box detected, it can be reduced as appropriate|
|det_db_unclip_ratio|float|1.6|Indicates the compactness of the text box, the smaller the value, the closer the text box to the text|
|use_polygon_score|bool|false|Whether to use polygon box to calculate bbox score, false means to use rectangle box to calculate. Use rectangular box to calculate faster, and polygonal box more accurate for curved text area.|
|visualize|bool|true|Whether to visualize the results,when it is set as true, The prediction result will be save in the image file `./ocr_vis.png`.|
MissPenguin's avatar
MissPenguin committed
247

fanruinet's avatar
fanruinet committed
248
- Classifier related parameters
MissPenguin's avatar
MissPenguin committed
249
250
251

|parameter|data type|default|meaning|
| --- | --- | --- | --- |
MissPenguin's avatar
MissPenguin committed
252
253
254
|use_angle_cls|bool|false|Whether to use the direction classifier|
|cls_model_dir|string|-|Address of direction classifier inference model|
|cls_thresh|float|0.9|Score threshold of the  direction classifier|
MissPenguin's avatar
MissPenguin committed
255

fanruinet's avatar
fanruinet committed
256
- Recognition related parameters
MissPenguin's avatar
MissPenguin committed
257
258
259

|parameter|data type|default|meaning|
| --- | --- | --- | --- |
MissPenguin's avatar
MissPenguin committed
260
261
262
263
|rec_model_dir|string|-|Address of recognition inference model|
|char_list_file|string|../../ppocr/utils/ppocr_keys_v1.txt|dictionary file|

* Multi-language inference is also supported in PaddleOCR, you can refer to [recognition tutorial](../../doc/doc_en/recognition_en.md) for more supported languages and models in PaddleOCR. Specifically, if you want to infer using multi-language models, you just need to modify values of `char_list_file` and `rec_model_dir`.
zhoujun's avatar
zhoujun committed
264
265


littletomatodonkey's avatar
littletomatodonkey committed
266
267
268
The detection results will be shown on the screen, which is as follows.

<div align="center">
littletomatodonkey's avatar
littletomatodonkey committed
269
    <img src="./imgs/cpp_infer_pred_12.png" width="600">
littletomatodonkey's avatar
littletomatodonkey committed
270
271
272
</div>


zhoujun's avatar
zhoujun committed
273
### 2.3 Notes
littletomatodonkey's avatar
littletomatodonkey committed
274

fanruinet's avatar
fanruinet committed
275
* Paddle 2.0.0 inference model library is recommended for this tutorial.