readme_en.md 10.8 KB
Newer Older
WenmuZhou's avatar
WenmuZhou committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
- [Server-side C++ Inference](#server-side-c-inference)
  - [1. Prepare the Environment](#1-prepare-the-environment)
    - [Environment](#environment)
    - [1.1 Compile OpenCV](#11-compile-opencv)
    - [1.2 Compile or Download or the Paddle Inference Library](#12-compile-or-download-or-the-paddle-inference-library)
      - [1.2.1 Direct download and installation](#121-direct-download-and-installation)
      - [1.2.2 Compile the inference source code](#122-compile-the-inference-source-code)
  - [2. Compile and Run the Demo](#2-compile-and-run-the-demo)
    - [2.1 Export the inference model](#21-export-the-inference-model)
    - [2.2 Compile PaddleOCR C++ inference demo](#22-compile-paddleocr-c-inference-demo)
    - [Run the demo](#run-the-demo)
        - [1. run det demo:](#1-run-det-demo)
        - [2. run rec demo:](#2-run-rec-demo)
        - [3. run system demo:](#3-run-system-demo)
  - [3. FAQ](#3-faq)

17
# Server-side C++ Inference
littletomatodonkey's avatar
littletomatodonkey committed
18

fanruinet's avatar
fanruinet committed
19
20
21
This chapter introduces the C++ deployment steps of the PaddleOCR model. The corresponding Python predictive deployment method refers to [document](../../doc/doc_ch/inference.md).
C++ is better than python in terms of performance. Therefore, in CPU and GPU deployment scenarios, C++ deployment is mostly used.
This section will introduce how to configure the C++ environment and deploy PaddleOCR in Linux (CPU\GPU) environment. For Windows deployment please refer to [Windows](./docs/windows_vs2019_build.md) compilation guidelines.
littletomatodonkey's avatar
littletomatodonkey committed
22
23


24
## 1. Prepare the Environment
littletomatodonkey's avatar
littletomatodonkey committed
25
26
27
28
29
30

### Environment

- Linux, docker is recommended.


31
### 1.1 Compile OpenCV
littletomatodonkey's avatar
littletomatodonkey committed
32

fanruinet's avatar
fanruinet committed
33
* First of all, you need to download the source code compiled package in the Linux environment from the OpenCV official website. Taking OpenCV 3.4.7 as an example, the download command is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
34

littletomatodonkey's avatar
littletomatodonkey committed
35
```bash
WenmuZhou's avatar
WenmuZhou committed
36
cd deploy/cpp_infer
littletomatodonkey's avatar
littletomatodonkey committed
37
38
wget https://paddleocr.bj.bcebos.com/libs/opencv/opencv-3.4.7.tar.gz
tar -xf opencv-3.4.7.tar.gz
littletomatodonkey's avatar
littletomatodonkey committed
39
40
```

fanruinet's avatar
fanruinet committed
41
Finally, you will see the folder of `opencv-3.4.7/` in the current directory.
littletomatodonkey's avatar
littletomatodonkey committed
42

fanruinet's avatar
fanruinet committed
43
* Compile OpenCV, the OpenCV source path (`root_path`) and installation path (`install_path`) should be set by yourself. Enter the OpenCV source code path and compile it in the following way.
littletomatodonkey's avatar
littletomatodonkey committed
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75


```shell
root_path=your_opencv_root_path
install_path=${root_path}/opencv3

rm -rf build
mkdir build
cd build

cmake .. \
    -DCMAKE_INSTALL_PREFIX=${install_path} \
    -DCMAKE_BUILD_TYPE=Release \
    -DBUILD_SHARED_LIBS=OFF \
    -DWITH_IPP=OFF \
    -DBUILD_IPP_IW=OFF \
    -DWITH_LAPACK=OFF \
    -DWITH_EIGEN=OFF \
    -DCMAKE_INSTALL_LIBDIR=lib64 \
    -DWITH_ZLIB=ON \
    -DBUILD_ZLIB=ON \
    -DWITH_JPEG=ON \
    -DBUILD_JPEG=ON \
    -DWITH_PNG=ON \
    -DBUILD_PNG=ON \
    -DWITH_TIFF=ON \
    -DBUILD_TIFF=ON

make -j
make install
```

fanruinet's avatar
fanruinet committed
76
In the above commands, `root_path` is the downloaded OpenCV source code path, and `install_path` is the installation path of OpenCV. After `make install` is completed, the OpenCV header file and library file will be generated in this folder for later OCR source code compilation.
littletomatodonkey's avatar
littletomatodonkey committed
77
78
79



fanruinet's avatar
fanruinet committed
80
The final file structure under the OpenCV installation path is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
81
82
83
84
85
86
87
88
89
90

```
opencv3/
|-- bin
|-- include
|-- lib
|-- lib64
|-- share
```

91
### 1.2 Compile or Download or the Paddle Inference Library
littletomatodonkey's avatar
littletomatodonkey committed
92
93
94

* There are 2 ways to obtain the Paddle inference library, described in detail below.

littletomatodonkey's avatar
littletomatodonkey committed
95
#### 1.2.1 Direct download and installation
littletomatodonkey's avatar
littletomatodonkey committed
96

WenmuZhou's avatar
WenmuZhou committed
97
[Paddle inference library official website](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html#linux). You can review and select the appropriate version of the inference library on the official website.
littletomatodonkey's avatar
littletomatodonkey committed
98
99


fanruinet's avatar
fanruinet committed
100
* After downloading, use the following command to extract files.
littletomatodonkey's avatar
littletomatodonkey committed
101
102
103
104
105

```
tar -xf paddle_inference.tgz
```

fanruinet's avatar
fanruinet committed
106
Finally you will see the the folder of `paddle_inference/` in the current path.
littletomatodonkey's avatar
littletomatodonkey committed
107

fanruinet's avatar
fanruinet committed
108
109
110
#### 1.2.2 Compile the inference source code
* If you want to get the latest Paddle inference library features, you can download the latest code from Paddle GitHub repository and compile the inference library from the source code. It is recommended to download the inference library with paddle version greater than or equal to 2.0.1.
* You can refer to [Paddle inference library] (https://www.paddlepaddle.org.cn/documentation/docs/en/advanced_guide/inference_deployment/inference/build_and_install_lib_en.html) to get the Paddle source code from GitHub, and then compile To generate the latest inference library. The method of using git to access the code is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
111
112
113
114


```shell
git clone https://github.com/PaddlePaddle/Paddle.git
LDOUBLEV's avatar
LDOUBLEV committed
115
git checkout develop
littletomatodonkey's avatar
littletomatodonkey committed
116
117
```

fanruinet's avatar
fanruinet committed
118
* Enter the Paddle directory and run the following commands to compile the paddle inference library.
littletomatodonkey's avatar
littletomatodonkey committed
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137

```shell
rm -rf build
mkdir build
cd build

cmake  .. \
    -DWITH_CONTRIB=OFF \
    -DWITH_MKL=ON \
    -DWITH_MKLDNN=ON  \
    -DWITH_TESTING=OFF \
    -DCMAKE_BUILD_TYPE=Release \
    -DWITH_INFERENCE_API_TEST=OFF \
    -DON_INFER=ON \
    -DWITH_PYTHON=ON
make -j
make inference_lib_dist
```

LDOUBLEV's avatar
LDOUBLEV committed
138
For more compilation parameter options, please refer to the [document](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0/guides/05_inference_deployment/inference/build_and_install_lib_cn.html#congyuanmabianyi).
littletomatodonkey's avatar
littletomatodonkey committed
139
140


LDOUBLEV's avatar
LDOUBLEV committed
141
* After the compilation process, you can see the following files in the folder of `build/paddle_inference_install_dir/`.
littletomatodonkey's avatar
littletomatodonkey committed
142
143

```
LDOUBLEV's avatar
LDOUBLEV committed
144
build/paddle_inference_install_dir/
littletomatodonkey's avatar
littletomatodonkey committed
145
146
147
148
149
150
|-- CMakeCache.txt
|-- paddle
|-- third_party
|-- version.txt
```

fanruinet's avatar
fanruinet committed
151
`paddle` is the Paddle library required for C++ prediction later, and `version.txt` contains the version information of the current inference library.
littletomatodonkey's avatar
littletomatodonkey committed
152
153


154
## 2. Compile and Run the Demo
littletomatodonkey's avatar
littletomatodonkey committed
155
156
157

### 2.1 Export the inference model

fanruinet's avatar
fanruinet committed
158
* You can refer to [Model inference](../../doc/doc_ch/inference.md) and export the inference model. After the model is exported, assuming it is placed in the `inference` directory, the directory structure is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
159
160
161
162

```
inference/
|-- det_db
MissPenguin's avatar
MissPenguin committed
163
164
|   |--inference.pdiparams
|   |--inference.pdmodel
littletomatodonkey's avatar
littletomatodonkey committed
165
|-- rec_rcnn
MissPenguin's avatar
MissPenguin committed
166
167
|   |--inference.pdiparams
|   |--inference.pdmodel
littletomatodonkey's avatar
littletomatodonkey committed
168
169
170
171
172
173
174
175
176
```


### 2.2 Compile PaddleOCR C++ inference demo


* The compilation commands are as follows. The addresses of Paddle C++ inference library, opencv and other Dependencies need to be replaced with the actual addresses on your own machines.

```shell
MissPenguin's avatar
MissPenguin committed
177
sh tools/build.sh
littletomatodonkey's avatar
littletomatodonkey committed
178
179
```

MissPenguin's avatar
MissPenguin committed
180
Specifically, you should modify the paths in `tools/build.sh`. The related content is as follows.
littletomatodonkey's avatar
littletomatodonkey committed
181
182
183
184
185
186
187
188

```shell
OPENCV_DIR=your_opencv_dir
LIB_DIR=your_paddle_inference_dir
CUDA_LIB_DIR=your_cuda_lib_dir
CUDNN_LIB_DIR=your_cudnn_lib_dir
```

fanruinet's avatar
fanruinet committed
189
`OPENCV_DIR` is the OpenCV installation path; `LIB_DIR` is the download (`paddle_inference` folder)
LDOUBLEV's avatar
LDOUBLEV committed
190
or the generated Paddle inference library path (`build/paddle_inference_install_dir` folder);
fanruinet's avatar
fanruinet committed
191
`CUDA_LIB_DIR` is the CUDA library file path, in docker; it is `/usr/local/cuda/lib64`; `CUDNN_LIB_DIR` is the cuDNN library file path, in docker it is `/usr/lib/x86_64-linux-gnu/`.
littletomatodonkey's avatar
littletomatodonkey committed
192
193


MissPenguin's avatar
MissPenguin committed
194
* After the compilation is completed, an executable file named `ppocr` will be generated in the `build` folder.
littletomatodonkey's avatar
littletomatodonkey committed
195
196
197


### Run the demo
fanruinet's avatar
fanruinet committed
198
Execute the built executable file:
MissPenguin's avatar
MissPenguin committed
199
200
```shell
./build/ppocr <mode> [--param1] [--param2] [...]
201
```
fanruinet's avatar
fanruinet committed
202
203
204
205
206
207
208
209
210
`mode` is a required parameter,and the valid values are

mode value | Model used
-----|------
det  | Detection only
rec  | Recognition only
system | End-to-end system

Specifically,
MissPenguin's avatar
MissPenguin committed
211
212

##### 1. run det demo:
littletomatodonkey's avatar
littletomatodonkey committed
213
```shell
MissPenguin's avatar
MissPenguin committed
214
./build/ppocr det \
MissPenguin's avatar
MissPenguin committed
215
    --det_model_dir=inference/ch_ppocr_mobile_v2.0_det_infer \
MissPenguin's avatar
MissPenguin committed
216
    --image_dir=../../doc/imgs/12.jpg
littletomatodonkey's avatar
littletomatodonkey committed
217
```
MissPenguin's avatar
MissPenguin committed
218
##### 2. run rec demo:
MissPenguin's avatar
MissPenguin committed
219
```shell
MissPenguin's avatar
MissPenguin committed
220
./build/ppocr rec \
MissPenguin's avatar
MissPenguin committed
221
    --rec_model_dir=inference/ch_ppocr_mobile_v2.0_rec_infer \
MissPenguin's avatar
MissPenguin committed
222
    --image_dir=../../doc/imgs_words/ch/
zhoujun's avatar
zhoujun committed
223
```
MissPenguin's avatar
MissPenguin committed
224
##### 3. run system demo:
MissPenguin's avatar
MissPenguin committed
225
226
```shell
# without text direction classifier
MissPenguin's avatar
MissPenguin committed
227
./build/ppocr system \
MissPenguin's avatar
MissPenguin committed
228
229
    --det_model_dir=inference/ch_ppocr_mobile_v2.0_det_infer \
    --rec_model_dir=inference/ch_ppocr_mobile_v2.0_rec_infer \
MissPenguin's avatar
MissPenguin committed
230
231
    --image_dir=../../doc/imgs/12.jpg
# with text direction classifier
MissPenguin's avatar
MissPenguin committed
232
./build/ppocr system \
MissPenguin's avatar
MissPenguin committed
233
234
235
236
    --det_model_dir=inference/ch_ppocr_mobile_v2.0_det_infer \
    --use_angle_cls=true \
    --cls_model_dir=inference/ch_ppocr_mobile_v2.0_cls_infer \
    --rec_model_dir=inference/ch_ppocr_mobile_v2.0_rec_infer \
MissPenguin's avatar
MissPenguin committed
237
238
239
    --image_dir=../../doc/imgs/12.jpg
```

fanruinet's avatar
fanruinet committed
240
More parameters are as follows,
MissPenguin's avatar
MissPenguin committed
241

fanruinet's avatar
fanruinet committed
242
- Common parameters
MissPenguin's avatar
MissPenguin committed
243

MissPenguin's avatar
MissPenguin committed
244
245
246
247
248
249
|parameter|data type|default|meaning|
| --- | --- | --- | --- |
|use_gpu|bool|false|Whether to use GPU|
|gpu_id|int|0|GPU id when use_gpu is true|
|gpu_mem|int|4000|GPU memory requested|
|cpu_math_library_num_threads|int|10|Number of threads when using CPU inference. When machine cores is enough, the large the value, the faster the inference speed|
WenmuZhou's avatar
WenmuZhou committed
250
|enable_mkldnn|bool|true|Whether to use mkdlnn library|
MissPenguin's avatar
MissPenguin committed
251

fanruinet's avatar
fanruinet committed
252
- Detection related parameters
MissPenguin's avatar
MissPenguin committed
253
254
255

|parameter|data type|default|meaning|
| --- | --- | --- | --- |
MissPenguin's avatar
MissPenguin committed
256
257
258
259
260
261
262
|det_model_dir|string|-|Address of detection inference model|
|max_side_len|int|960|Limit the maximum image height and width to 960|
|det_db_thresh|float|0.3|Used to filter the binarized image of DB prediction, setting 0.-0.3 has no obvious effect on the result|
|det_db_box_thresh|float|0.5|DB post-processing filter box threshold, if there is a missing box detected, it can be reduced as appropriate|
|det_db_unclip_ratio|float|1.6|Indicates the compactness of the text box, the smaller the value, the closer the text box to the text|
|use_polygon_score|bool|false|Whether to use polygon box to calculate bbox score, false means to use rectangle box to calculate. Use rectangular box to calculate faster, and polygonal box more accurate for curved text area.|
|visualize|bool|true|Whether to visualize the results,when it is set as true, The prediction result will be save in the image file `./ocr_vis.png`.|
MissPenguin's avatar
MissPenguin committed
263

fanruinet's avatar
fanruinet committed
264
- Classifier related parameters
MissPenguin's avatar
MissPenguin committed
265
266
267

|parameter|data type|default|meaning|
| --- | --- | --- | --- |
MissPenguin's avatar
MissPenguin committed
268
269
270
|use_angle_cls|bool|false|Whether to use the direction classifier|
|cls_model_dir|string|-|Address of direction classifier inference model|
|cls_thresh|float|0.9|Score threshold of the  direction classifier|
MissPenguin's avatar
MissPenguin committed
271

fanruinet's avatar
fanruinet committed
272
- Recognition related parameters
MissPenguin's avatar
MissPenguin committed
273
274
275

|parameter|data type|default|meaning|
| --- | --- | --- | --- |
MissPenguin's avatar
MissPenguin committed
276
|rec_model_dir|string|-|Address of recognition inference model|
WenmuZhou's avatar
WenmuZhou committed
277
|rec_char_dict_path|string|../../ppocr/utils/ppocr_keys_v1.txt|dictionary file|
MissPenguin's avatar
MissPenguin committed
278

WenmuZhou's avatar
WenmuZhou committed
279
* Multi-language inference is also supported in PaddleOCR, you can refer to [recognition tutorial](../../doc/doc_en/recognition_en.md) for more supported languages and models in PaddleOCR. Specifically, if you want to infer using multi-language models, you just need to modify values of `rec_char_dict_path` and `rec_model_dir`.
zhoujun's avatar
zhoujun committed
280
281


littletomatodonkey's avatar
littletomatodonkey committed
282
283
284
The detection results will be shown on the screen, which is as follows.

<div align="center">
littletomatodonkey's avatar
littletomatodonkey committed
285
    <img src="./imgs/cpp_infer_pred_12.png" width="600">
littletomatodonkey's avatar
littletomatodonkey committed
286
287
288
</div>


WenmuZhou's avatar
WenmuZhou committed
289
## 3. FAQ
littletomatodonkey's avatar
littletomatodonkey committed
290

WenmuZhou's avatar
WenmuZhou committed
291
 1.  Encountered the error `unable to access 'https://github.com/LDOUBLEV/AutoLog.git/': gnutls_handshake() failed: The TLS connection was non-properly terminated.` First import `https://github. com/LDOUBLEV/AutoLog` project on gitee, and then change the github address in `deploy/cpp_infer/external-cmake/auto-log.cmake` to the gitee address.