Fixed merge conflict in README.md

README.md

@@ -3,10 +3,6 @@ English | [简体中文](README_ch.md)
 <p align="center">
  <img src="./doc/PaddleOCR_log.png" align="middle" width = "600"/>
 <p align="center">
-
-
-------------------------------------------------------------------------------------------
-
 <p align="left">
     <a href="./LICENSE"><img src="https://img.shields.io/badge/license-Apache%202-dfd.svg"></a>
     <a href="https://github.com/PaddlePaddle/PaddleOCR/releases"><img src="https://img.shields.io/github/v/release/PaddlePaddle/PaddleOCR?color=ffa"></a>
@@ -32,61 +28,44 @@ PaddleOCR aims to create multilingual, awesome, leading, and practical OCR tools
 
 - [more](./doc/doc_en/update_en.md)
 
+
 ## Features
-- PP-OCR - A series of high-quality pre-trained models, comparable to commercial products
-    - Ultra lightweight PP-OCRv2 series models: detection (3.1M) + direction classifier (1.4M) + recognition 8.5M) = 13.0M
-    - Ultra lightweight PP-OCR mobile series models: detection (3.0M) + direction classifier (1.4M) + recognition (5.0M) = 9.4M
-    - General PP-OCR server series models: detection (47.1M) + direction classifier (1.4M) + recognition (94.9M) = 143.4M
-    - Support Chinese, English, and digit recognition, vertical text recognition, and long text recognition
-    - Support multi-lingual recognition: about 80 languages like Korean, Japanese, German, French, etc
-- PP-Structure: a document structurize system
-    - Support layout analysis and table recognition (support export to Excel)
-    - Support key information extraction
-    - Support DocVQA
-- Rich OCR toolkit
-    - Semi-automatic data annotation tool, i.e., PPOCRLabel: support fast and efficient data annotation
-    - Data synthesis tool, i.e., Style-Text: easy to synthesize a large number of images which are similar to the target scene image
-- Support user-defined training, provides rich predictive inference deployment solutions
-- Support PIP installation, easy to use
-- Support Linux, Windows, MacOS and other systems
-- Supports metric logging to [VisualDL](https://www.paddlepaddle.org.cn/documentation/docs/en/guides/03_VisualDL/visualdl_usage_en.html) and [Weights & Biases](docs.wandb.ai)
-
-## Visualization
 
-<div align="center">
-    <img src="doc/imgs_results/ch_ppocr_mobile_v2.0/test_add_91.jpg" width="800">
-    <img src="doc/imgs_results/multi_lang/img_01.jpg" width="800">
-    <img src="doc/imgs_results/multi_lang/img_02.jpg" width="800">
-</div>
+PaddleOCR support a variety of cutting-edge algorithms related to OCR, and developed industrial featured models/solution [PP-OCR](./doc/doc_en/ppocr_introduction_en.md) and [PP-Structure](./ppstructure/README.md) on this basis, and get through the whole process of data production, model training, compression, inference and deployment.
 
-The above pictures are the visualizations of the general ppocr_server model. For more effect pictures, please see [More visualizations](./doc/doc_en/visualization_en.md).
+PaddleOCR also supports metric and model logging during training to [VisualDL](https://www.paddlepaddle.org.cn/documentation/docs/en/guides/03_VisualDL/visualdl_usage_en.html) and [Weights & Biases](https://docs.wandb.ai/).
 
-<a name="Community"></a>
-## Community
-- Scan the QR code below with your Wechat, you can join the official technical discussion group. Looking forward to your participation.
+![](./doc/features_en.png)
 
-<div align="center">
-<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleOCR/dygraph/doc/joinus.PNG"  width = "200" height = "200" />
-</div>
+> It is recommended to start with the “quick experience” in the document tutorial
 
 
 ## Quick Experience
 
-You can also quickly experience the ultra-lightweight OCR : [Online Experience](https://www.paddlepaddle.org.cn/hub/scene/ocr)
+- Web online experience for the ultra-lightweight OCR: [Online Experience](https://www.paddlepaddle.org.cn/hub/scene/ocr)
+- Mobile DEMO experience (based on EasyEdge and Paddle-Lite, supports iOS and Android systems): [Sign in to the website to obtain the QR code for  installing the App](https://ai.baidu.com/easyedge/app/openSource?from=paddlelite)
+- One line of code quick use: [Quick Start](./doc/doc_en/quickstart_en.md)
+
 
-Mobile DEMO experience (based on EasyEdge and Paddle-Lite, supports iOS and Android systems): [Sign in to the website to obtain the QR code for  installing the App](https://ai.baidu.com/easyedge/app/openSource?from=paddlelite)
+<a name="book"></a>
+## E-book: *Dive Into OCR*
+- [Dive Into OCR 📚](./doc/doc_en/ocr_book_en.md)
+
+
+<a name="Community"></a>
+## Community
+
+- **Join us**👬: Scan the QR code below with your Wechat, you can join the official technical discussion group. Looking forward to your participation.
+- **Contribution**🏅️: [Contribution page](./doc/doc_en/thirdparty.md) contains various tools and applications developed by community developers using PaddleOCR, as well as the functions, optimized documents and codes contributed to PaddleOCR. It is an official honor wall for community developers and a broadcasting station to help publicize high-quality projects.
+- **Regular Season**🎁: The community regular season is a point competition for OCR developers, covering four types: documents, codes, models and applications. Awards are selected and awarded on a quarterly basis. Please refer to the [link](https://github.com/PaddlePaddle/PaddleOCR/issues/4982) for more details.
 
- Also, you can scan the QR code below to install the App (**Android support only**)
 
 <div align="center">
-<img src="./doc/ocr-android-easyedge.png"  width = "200" height = "200" />
+<img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleOCR/dygraph/doc/joinus.PNG"  width = "200" height = "200" />
 </div>
 
-- [**OCR Quick Start**](./doc/doc_en/quickstart_en.md)
 
 <a name="Supported-Chinese-model-list"></a>
-
-
 ## PP-OCR Series Model List（Update on September 8th）
 
 | Model introduction                                           | Model name                   | Recommended scene | Detection model                                              | Direction classifier                                         | Recognition model                                            |
@@ -96,41 +75,48 @@ Mobile DEMO experience (based on EasyEdge and Paddle-Lite, supports iOS and Andr
 | Chinese and English general PP-OCR model (143.4M)               | ch_ppocr_server_v2.0_xx      | Server            |[inference model](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_infer.tar) / [trained model](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_train.tar)    |[inference model](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar) / [trained model](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar)    |[inference model](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_infer.tar) / [trained model](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_train.tar)  |
 
 
-For more model downloads (including multiple languages), please refer to [PP-OCR series model downloads](./doc/doc_en/models_list_en.md).
-
-For a new language request, please refer to [Guideline for new language_requests](#language_requests).
+- For more model downloads (including multiple languages), please refer to [PP-OCR series model downloads](./doc/doc_en/models_list_en.md).
+- For a new language request, please refer to [Guideline for new language_requests](#language_requests).
+- For structural document analysis models, please refer to [PP-Structure models](./ppstructure/docs/models_list_en.md).
 
 ## Tutorials
 - [Environment Preparation](./doc/doc_en/environment_en.md)
 - [Quick Start](./doc/doc_en/quickstart_en.md)
-- [PaddleOCR Overview and Project Clone](./doc/doc_en/paddleOCR_overview_en.md)
-- PP-OCR Industry Landing: from Training to Deployment
-    - [PP-OCR Model Zoo](./doc/doc_en/models_en.md)
-        - [PP-OCR Model Download](./doc/doc_en/models_list_en.md)
-        - [Python Inference for PP-OCR Model Zoo](./doc/doc_en/inference_ppocr_en.md)
-    - [PP-OCR Training](./doc/doc_en/training_en.md)
+- [PP-OCR 🔥](./doc/doc_en/ppocr_introduction_en.md)
+    - [Quick Start](./doc/doc_en/quickstart_en.md)
+    - [Model Zoo](./doc/doc_en/models_en.md)
+    - [Model training](./doc/doc_en/training_en.md)
         - [Text Detection](./doc/doc_en/detection_en.md)
         - [Text Recognition](./doc/doc_en/recognition_en.md)
         - [Text Direction Classification](./doc/doc_en/angle_class_en.md)
-        - [Yml Configuration](./doc/doc_en/config_en.md)
-    - PP-OCR Models Compression
-        - [Knowledge Distillation](./doc/doc_en/knowledge_distillation_en.md)
+    - Model Compression
         - [Model Quantization](./deploy/slim/quantization/README_en.md)
         - [Model Pruning](./deploy/slim/prune/README_en.md)
-    - Inference and Deployment
-        - [C++ Inference](./deploy/cpp_infer/readme_en.md)
+        - [Knowledge Distillation](./doc/doc_en/knowledge_distillation_en.md)
+    - [Inference and Deployment](./deploy/README.md)
+        - [Python Inference](./doc/doc_en/inference_ppocr_en.md)
+        - [C++ Inference](./deploy/cpp_infer/readme.md)
         - [Serving](./deploy/pdserving/README.md)
-        - [Mobile](./deploy/lite/readme_en.md)
+        - [Mobile](./deploy/lite/readme.md)
+        - [Paddle2ONNX](./deploy/paddle2onnx/readme.md)
         - [Benchmark](./doc/doc_en/benchmark_en.md)  
-- [PP-Structure: Information Extraction](./ppstructure/README.md)
+- [PP-Structure 🔥](./ppstructure/README.md)
+    - [Quick Start](./ppstructure/docs/quickstart_en.md)
+    - [Model Zoo](./ppstructure/docs/models_list_en.md)
+    - [Model training](./doc/doc_en/training_en.md)    
         - [Layout Parser](./ppstructure/layout/README.md)
         - [Table Recognition](./ppstructure/table/README.md)
         - [DocVQA](./ppstructure/vqa/README.md)
-    - [Key Information Extraction](./ppstructure/docs/kie.md)
-- Academic Circles
-    - [Two-stage Algorithm](./doc/doc_en/algorithm_overview_en.md)
-    - [PGNet Algorithm](./doc/doc_en/pgnet_en.md)
-    - [Python Inference](./doc/doc_en/inference_en.md)
+        - [Key Information Extraction](./ppstructure/docs/kie_en.md)
+    - [Inference and Deployment](./deploy/README.md)
+        - [Python Inference](./ppstructure/docs/inference_en.md)
+        - [C++ Inference]()
+        - [Serving](./deploy/pdserving/README.md)
+- [Academic algorithms](./doc/doc_en/algorithms_en.md)
+    - [Text detection](./doc/doc_en/algorithm_overview_en.md)
+    - [Text recognition](./doc/doc_en/algorithm_overview_en.md)
+    - [End-to-end](./doc/doc_en/algorithm_overview_en.md)
+    - [Add New Algorithms to PaddleOCR](./doc/doc_en/add_new_algorithm_en.md)
 - Data Annotation and Synthesis
     - [Semi-automatic Annotation Tool: PPOCRLabel](./PPOCRLabel/README.md)
     - [Data Synthesis Tool: Style-Text](./StyleText/README.md)
@@ -140,28 +126,16 @@ For a new language request, please refer to [Guideline for new language_requests
     - [General OCR Datasets(Chinese/English)](./doc/doc_en/datasets_en.md)
     - [HandWritten_OCR_Datasets(Chinese)](./doc/doc_en/handwritten_datasets_en.md)
     - [Various OCR Datasets(multilingual)](./doc/doc_en/vertical_and_multilingual_datasets_en.md)
+- [Code Structure](./doc/doc_en/tree_en.md)
 - [Visualization](#Visualization)
+- [Community](#Community)
 - [New language requests](#language_requests)
 - [FAQ](./doc/doc_en/FAQ_en.md)
-- [Community](#Community)
 - [References](./doc/doc_en/reference_en.md)
 - [License](#LICENSE)
-- [Contribution](#CONTRIBUTION)
-
-<a name="PP-OCRv2"></a>
-
-## PP-OCRv2 Pipeline
-<div align="center">
-    <img src="./doc/ppocrv2_framework.jpg" width="800">
-</div>
-
-[1] PP-OCR is a practical ultra-lightweight OCR system. It is mainly composed of three parts: DB text detection, detection frame correction and CRNN text recognition. The system adopts 19 effective strategies from 8 aspects including backbone network selection and adjustment, prediction head design, data augmentation, learning rate transformation strategy, regularization parameter selection, pre-training model use, and automatic model tailoring and quantization to optimize and slim down the models of each module (as shown in the green box above). The final results are an ultra-lightweight Chinese and English OCR model with an overall size of 3.5M and a 2.8M English digital OCR model. For more details, please refer to the PP-OCR technical article (https://arxiv.org/abs/2009.09941).
-
-[2] On the basis of PP-OCR, PP-OCRv2 is further optimized in five aspects. The detection model adopts CML(Collaborative Mutual Learning) knowledge distillation strategy and CopyPaste data expansion strategy. The recognition model adopts LCNet lightweight backbone network, U-DML knowledge distillation strategy and enhanced CTC loss function improvement (as shown in the red box above), which further improves the inference speed and prediction effect. For more details, please refer to the technical report of PP-OCRv2 (https://arxiv.org/abs/2109.03144).
-
-
 
 
+<a name="Visualization"></a>
 ## Visualization [more](./doc/doc_en/visualization_en.md)
 - Chinese OCR model
 <div align="center">
@@ -199,19 +173,3 @@ More details, please refer to [Multilingual OCR Development Plan](https://github
 <a name="LICENSE"></a>
 ## License
 This project is released under <a href="https://github.com/PaddlePaddle/PaddleOCR/blob/master/LICENSE">Apache 2.0 license</a>
\ No newline at end of file
-
-<a name="CONTRIBUTION"></a>
-## Contribution
-We welcome all the contributions to PaddleOCR and appreciate for your feedback very much.
-
-- Many thanks to [Khanh Tran](https://github.com/xxxpsyduck) and [Karl Horky](https://github.com/karlhorky) for contributing and revising the English documentation.
-- Many thanks to [zhangxin](https://github.com/ZhangXinNan) for contributing the new visualize function、add .gitignore and discard set PYTHONPATH manually.
-- Many thanks to [lyl120117](https://github.com/lyl120117) for contributing the code for printing the network structure.
-- Thanks [xiangyubo](https://github.com/xiangyubo) for contributing the handwritten Chinese OCR datasets.
-- Thanks [authorfu](https://github.com/authorfu) for contributing Android demo  and [xiadeye](https://github.com/xiadeye) contributing iOS demo, respectively.
-- Thanks [BeyondYourself](https://github.com/BeyondYourself) for contributing many great suggestions and simplifying part of the code style.
-- Thanks [tangmq](https://gitee.com/tangmq) for contributing Dockerized deployment services to PaddleOCR and supporting the rapid release of callable Restful API services.
-- Thanks [lijinhan](https://github.com/lijinhan) for contributing a new way, i.e., java SpringBoot, to achieve the request for the Hubserving deployment.
-- Thanks [Mejans](https://github.com/Mejans) for contributing the Occitan corpus and character set.
-- Thanks [LKKlein](https://github.com/LKKlein) for contributing a new deploying package with the Golang program language.
-- Thanks [Evezerest](https://github.com/Evezerest), [ninetailskim](https://github.com/ninetailskim), [edencfc](https://github.com/edencfc), [BeyondYourself](https://github.com/BeyondYourself) and [1084667371](https://github.com/1084667371) for contributing a new data annotation tool, i.e., PPOCRLabel。

README_ch.md

@@ -17,11 +17,18 @@
 
 PaddleOCR旨在打造一套丰富、领先、且实用的OCR工具库，助力开发者训练出更好的模型，并应用落地。
 
+<div align="center">
+    <img src="./doc/imgs_results/ch_ppocr_mobile_v2.0/test_add_91.jpg" width="800">
+</div>
+
+<div align="center">
+    <img src="./doc/imgs_results/ch_ppocr_mobile_v2.0/00018069.jpg" width="800">
+</div>
+
 ## 近期更新
 
 - 2021.12.21《动手学OCR · 十讲》课程开讲，12月21日起每晚八点半线上授课！[免费报名地址](https://aistudio.baidu.com/aistudio/course/introduce/25207)。
 - 2021.12.21 发布PaddleOCR v2.4。OCR算法新增1种文本检测算法（PSENet），3种文本识别算法（NRTR、SEED、SAR）；文档结构化算法新增1种关键信息提取算法（SDMGR，[文档](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/ppstructure/docs/kie.md)），3种DocVQA算法（LayoutLM、LayoutLMv2，LayoutXLM，[文档](https://github.com/PaddlePaddle/PaddleOCR/tree/release/2.4/ppstructure/vqa)）。
-- PaddleOCR研发团队对最新发版内容技术深入解读，9月8日晚上20:15，[课程回放](https://aistudio.baidu.com/aistudio/education/group/info/6758)。
 - 2021.9.7 发布PaddleOCR v2.3与[PP-OCRv2](#PP-OCRv2)，CPU推理速度相比于PP-OCR server提升220%；效果相比于PP-OCR mobile 提升7%。
 - 2021.8.3 发布PaddleOCR v2.2，新增文档结构分析[PP-Structure](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.2/ppstructure/README_ch.md)工具包，支持版面分析与表格识别（含Excel导出）。
 
@@ -29,41 +36,35 @@ PaddleOCR旨在打造一套丰富、领先、且实用的OCR工具库，助力
 
 ## 特性
 
-- PP-OCR系列高质量预训练模型，准确的识别效果
-    - 超轻量PP-OCRv2系列：检测（3.1M）+ 方向分类器（1.4M）+ 识别（8.5M）= 13.0M
-    - 超轻量PP-OCR mobile移动端系列：检测（3.0M）+方向分类器（1.4M）+ 识别（5.0M）= 9.4M
-    - 通用PP-OCR server系列：检测（47.1M）+方向分类器（1.4M）+ 识别（94.9M）= 143.4M
-    - 支持中英文数字组合识别、竖排文本识别、长文本识别
-    - 支持多语言识别：韩语、日语、德语、法语等约80种语言
-- PP-Structure文档结构化系统
-    - 支持版面分析与表格识别（含Excel导出）
-    - 支持关键信息提取任务
-    - 支持DocVQA任务
-- 丰富易用的OCR相关工具组件
-    - 半自动数据标注工具PPOCRLabel：支持快速高效的数据标注
-    - 数据合成工具Style-Text：批量合成大量与目标场景类似的图像
-- 支持用户自定义训练，提供丰富的预测推理部署方案
-- 支持PIP快速安装使用
-- 可运行于Linux、Windows、MacOS等多种系统
+支持多种OCR相关前沿算法，在此基础上打造产业级特色模型[PP-OCR](./doc/doc_ch/ppocr_introduction.md)和[PP-Structure](./ppstructure/README_ch.md)，并打通数据生产、模型训练、压缩、预测部署全流程。
+
+![](./doc/features.png)
 
 > 上述内容的使用方法建议从文档教程中的快速开始体验
 
-<a name="贡献代码"></a>
 
-## 社区、社区贡献与社区常规赛
+## 快速开始
+
+- 在线网站体验：超轻量PP-OCR mobile模型体验地址：https://www.paddlepaddle.org.cn/hub/scene/ocr
+- 移动端demo体验：[安装包DEMO下载地址](https://ai.baidu.com/easyedge/app/openSource?from=paddlelite)(基于EasyEdge和Paddle-Lite, 支持iOS和Android系统)
+- 一行命令快速使用：[快速开始（中英文/多语言/文档分析）](./doc/doc_ch/quickstart.md)
+
+<a name="电子书"></a>
+## 《动手学OCR》电子书
+- [《动手学OCR》电子书📚](./doc/doc_ch/ocr_book.md)
+
 
-- 加入社区：微信扫描下方二维码加入官方交流群，与各行各业开发者充分交流，期待您的加入。
-- 社区贡献：[社区贡献](./doc/doc_ch/thirdparty.md)文档中包含了社区用户**使用PaddleOCR开发的各种工具、应用**以及**为PaddleOCR贡献的功能、优化的文档与代码**等，是官方为社区开发者打造的荣誉墙、也是帮助优质项目宣传的广播站。如果您的OCR项目未被收集在文档中，可根据文档说明与我们联系。
-- 社区常规赛：社区常规赛是面向OCR开发者的积分赛事，覆盖文档、代码、模型和应用四大类型，以季度为单位评选并发放奖励，赛题详情与报名方法可参考[链接](https://github.com/PaddlePaddle/PaddleOCR/issues/4982)。
+<a name="开源社区"></a>
+## 开源社区
+
+- **加入社区**👬：微信扫描下方二维码加入官方交流群，与各行各业开发者充分交流，期待您的加入。
+- **社区贡献**🏅️：[社区贡献](./doc/doc_ch/thirdparty.md)文档中包含了社区用户**使用PaddleOCR开发的各种工具、应用**以及**为PaddleOCR贡献的功能、优化的文档与代码**等，是官方为社区开发者打造的荣誉墙，也是帮助优质项目宣传的广播站。
+- **社区常规赛**🎁：社区常规赛是面向OCR开发者的积分赛事，覆盖文档、代码、模型和应用四大类型，以季度为单位评选并发放奖励，赛题详情与报名方法可参考[链接](https://github.com/PaddlePaddle/PaddleOCR/issues/4982)。
 
 <div align="center">
 <img src="https://raw.githubusercontent.com/PaddlePaddle/PaddleOCR/dygraph/doc/joinus.PNG"  width = "200" height = "200" />
 </div>
 
-## 零代码体验
-
-- 在线网站体验：超轻量PP-OCR mobile模型体验地址：https://www.paddlepaddle.org.cn/hub/scene/ocr
-- 移动端：[安装包DEMO下载地址](https://ai.baidu.com/easyedge/app/openSource?from=paddlelite)(基于EasyEdge和Paddle-Lite, 支持iOS和Android系统)
 
 <a name="模型下载"></a>
 ## PP-OCR系列模型列表（更新中）
@@ -74,42 +75,53 @@ PaddleOCR旨在打造一套丰富、领先、且实用的OCR工具库，助力
 | 中英文超轻量PP-OCR mobile模型（9.4M） | ch_ppocr_mobile_v2.0_xx | 移动端&服务器端 | [推理模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_pre.tar) |
 | 中英文通用PP-OCR server模型（143.4M） | ch_ppocr_server_v2.0_xx | 服务器端        | [推理模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_det_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar) | [推理模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_infer.tar) / [预训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_server_v2.0_rec_pre.tar) |
 
-更多模型下载（包括多语言），可以参考[PP-OCR 系列模型下载](./doc/doc_ch/models_list.md)
+更多模型下载（包括多语言），可以参考[PP-OCR 系列模型下载](./doc/doc_ch/models_list.md)，文档分析相关模型参考[PP-Structure 系列模型下载](./ppstructure/docs/models_list.md)
+
 
 ## 文档教程
 
 - [运行环境准备](./doc/doc_ch/environment.md)
-- [快速开始（中英文/多语言/版面分析）](./doc/doc_ch/quickstart.md)
-- [PaddleOCR全景图与项目克隆](./doc/doc_ch/paddleOCR_overview.md)
-- PP-OCR产业落地：从训练到部署
-    - [PP-OCR模型库](./doc/doc_ch/models.md)
-        - [PP-OCR模型下载](./doc/doc_ch/models_list.md)
-        - [Python引擎的PP-OCR模型库推理](./doc/doc_ch/inference_ppocr.md)
-    - [PP-OCR模型训练](./doc/doc_ch/training.md)
+- [快速开始（中英文/多语言/文档分析）](./doc/doc_ch/quickstart.md)
+- [PP-OCR文本检测识别🔥](./doc/doc_ch/ppocr_introduction.md)
+    - [快速开始](./doc/doc_ch/quickstart.md)
+    - [模型库](./doc/doc_ch/models_list.md)
+    - [模型训练](./doc/doc_ch/training.md)
         - [文本检测](./doc/doc_ch/detection.md)
         - [文本识别](./doc/doc_ch/recognition.md)
         - [文本方向分类器](./doc/doc_ch/angle_class.md)
-        - [配置文件内容与生成](./doc/doc_ch/config.md)
-    - PP-OCR模型压缩
-        - [知识蒸馏](./doc/doc_ch/knowledge_distillation.md)
+    - 模型压缩
         - [模型量化](./deploy/slim/quantization/README.md)
         - [模型裁剪](./deploy/slim/prune/README.md)
-    - PP-OCR模型推理部署
+        - [知识蒸馏](./doc/doc_ch/knowledge_distillation.md)
+    - [推理部署](./deploy/README_ch.md)
+        - [基于Python预测引擎推理](./doc/doc_ch/inference_ppocr.md)
         - [基于C++预测引擎推理](./deploy/cpp_infer/readme.md)
         - [服务化部署](./deploy/pdserving/README_CN.md)
         - [端侧部署](./deploy/lite/readme.md)
         - [Paddle2ONNX模型转化与预测](./deploy/paddle2onnx/readme.md)
         - [Benchmark](./doc/doc_ch/benchmark.md)
-- [PP-Structure信息提取](./ppstructure/README_ch.md)
+- [PP-Structure文档分析🔥](./ppstructure/README_ch.md)
+    - [快速开始](./ppstructure/docs/quickstart.md)
+    - [模型库](./ppstructure/docs/models_list.md)
+    - [模型训练](./doc/doc_ch/training.md)
         - [版面分析](./ppstructure/layout/README_ch.md)
         - [表格识别](./ppstructure/table/README_ch.md)
-    - [DocVQA](./ppstructure/vqa/README.md)
         - [关键信息提取](./ppstructure/docs/kie.md)
-- OCR学术圈
-    - [两阶段模型介绍与下载](./doc/doc_ch/algorithm_overview.md)
-    - [端到端PGNet算法](./doc/doc_ch/pgnet.md)
-    - [基于Python脚本预测引擎推理](./doc/doc_ch/inference.md)
+        - [DocVQA](./ppstructure/vqa/README_ch.md)
+    - [推理部署](./deploy/README_ch.md)
+        - [基于Python预测引擎推理](./ppstructure/docs/inference.md)
+        - [基于C++预测引擎推理]()
+        - [服务化部署](./deploy/pdserving/README_CN.md)
+- [前沿算法与模型🚀](./doc/doc_ch/algorithm.md)
+    - [文本检测算法](./doc/doc_ch/algorithm_overview.md#11-%E6%96%87%E6%9C%AC%E6%A3%80%E6%B5%8B%E7%AE%97%E6%B3%95)
+    - [文本识别算法](./doc/doc_ch/algorithm_overview.md#12-%E6%96%87%E6%9C%AC%E8%AF%86%E5%88%AB%E7%AE%97%E6%B3%95)
+    - [端到端算法](./doc/doc_ch/algorithm_overview.md#2-%E6%96%87%E6%9C%AC%E8%AF%86%E5%88%AB%E7%AE%97%E6%B3%95)
     - [使用PaddleOCR架构添加新算法](./doc/doc_ch/add_new_algorithm.md)
+- [场景应用](./doc/doc_ch/application.md)
+    - [金融场景（表单/票据等）]()
+    - [工业场景（电表度数/车牌等）]()
+    - [教育场景（手写体/公式等）]()
+    - [医疗场景（化验单等）]()
 - 数据标注与合成
     - [半自动标注工具PPOCRLabel](./PPOCRLabel/README_ch.md)
     - [数据合成工具Style-Text](./StyleText/README_ch.md)
@@ -119,28 +131,26 @@ PaddleOCR旨在打造一套丰富、领先、且实用的OCR工具库，助力
     - [通用中英文OCR数据集](./doc/doc_ch/datasets.md)
     - [手写中文OCR数据集](./doc/doc_ch/handwritten_datasets.md)
     - [垂类多语言OCR数据集](./doc/doc_ch/vertical_and_multilingual_datasets.md)
+    - [版面分析数据集](./doc/doc_ch/layout_datasets.md)
+    - [表格识别数据集](./doc/doc_ch/table_datasets.md)
+    - [DocVQA数据集](./doc/doc_ch/docvqa_datasets.md)
+- [代码组织结构](./doc/doc_ch/tree.md)
 - [效果展示](#效果展示)
+- [《动手学OCR》电子书📚](./doc/doc_ch/ocr_book.md)
+- [开源社区](#开源社区)
 - FAQ
     - [通用问题](./doc/doc_ch/FAQ.md)
     - [PaddleOCR实战问题](./doc/doc_ch/FAQ.md)
 - [参考文献](./doc/doc_ch/reference.md)
 - [许可证书](#许可证书)
-- [代码组织结构](./doc/doc_ch/tree.md)
-
-<a name="PP-OCRv2"></a>
-
-## PP-OCRv2 Pipeline
-<div align="center">
-    <img src="./doc/ppocrv2_framework.jpg" width="800">
-</div>
-[1] PP-OCR是一个实用的超轻量OCR系统。主要由DB文本检测、检测框矫正和CRNN文本识别三部分组成。该系统从骨干网络选择和调整、预测头部的设计、数据增强、学习率变换策略、正则化参数选择、预训练模型使用以及模型自动裁剪量化8个方面，采用19个有效策略，对各个模块的模型进行效果调优和瘦身(如绿框所示)，最终得到整体大小为3.5M的超轻量中英文OCR和2.8M的英文数字OCR。更多细节请参考PP-OCR技术方案 https://arxiv.org/abs/2009.09941
 
-[2] PP-OCRv2在PP-OCR的基础上，进一步在5个方面重点优化，检测模型采用CML协同互学习知识蒸馏策略和CopyPaste数据增广策略；识别模型采用LCNet轻量级骨干网络、UDML 改进知识蒸馏策略和[Enhanced CTC loss](./doc/doc_ch/enhanced_ctc_loss.md)损失函数改进（如上图红框所示），进一步在推理速度和预测效果上取得明显提升。更多细节请参考PP-OCRv2[技术报告](https://arxiv.org/abs/2109.03144)。
 
 <a name="效果展示"></a>
 
 ## 效果展示 [more](./doc/doc_ch/visualization.md)
-- 中文模型
+
+<details open>
+<summary>PP-OCRv2 中文模型</summary>
 
 <div align="center">
       <img src="doc/imgs_results/ch_ppocr_mobile_v2.0/test_add_91.jpg" width="800">
@@ -151,16 +161,49 @@ PaddleOCR旨在打造一套丰富、领先、且实用的OCR工具库，助力
     <img src="./doc/imgs_results/ch_ppocr_mobile_v2.0/rotate_00052204.jpg" width="800">
 </div>
     
-- 英文模型
+</details>
+
+
+<details open>
+<summary>PP-OCRv2 英文模型</summary>
+    
 <div align="center">
     <img src="./doc/imgs_results/ch_ppocr_mobile_v2.0/img_12.jpg" width="800">
 </div>
 
-- 其他语言模型
+</details>
+
+
+<details open>
+<summary>PP-OCRv2 其他语言模型</summary>
+    
 <div align="center">
     <img src="./doc/imgs_results/french_0.jpg" width="800">
     <img src="./doc/imgs_results/korean.jpg" width="800">
 </div>
+    
+</details>
+
+<details open>
+<summary>PP-Structure 文档分析</summary>
+
+- 版面分析+表格识别  
+<div align="center">
+    <img src="./ppstructure/docs/table/ppstructure.GIF" width="800">
+</div>
+
+- SER（语义实体识别）  
+<div align="center">
+    <img src="./ppstructure/docs/vqa/result_ser/zh_val_0_ser.jpg" width="800">
+</div>
+    
+- RE（关系提取） 
+<div align="center">
+    <img src="./ppstructure/docs/vqa/result_re/zh_val_21_re.jpg" width="800">
+</div>
+
+</details>
+
 <a name="许可证书"></a>
 
 ## 许可证书

deploy/README.md

+English | [简体中文](README_ch.md)
+
+# PP-OCR Deployment
+
+- [Paddle Deployment Introduction](#1)
+- [PP-OCR Deployment](#2)
+
+<a name="1"></a>
+## Paddle Deployment Introduction
+
+Paddle provides a variety of deployment schemes to meet the deployment requirements of different scenarios. Please choose according to the actual situation: 
+
+<div align="center">
+    <img src="../doc/deployment_en.png" width="800">
+</div>
+
+
+<a name="2"></a>
+## PP-OCR Deployment
+
+PP-OCR has supported muti deployment schemes. Click the link to get the specific tutorial.
+
+- [Python Inference](../doc/doc_en/inference_ppocr_en.md)
+- [C++ Inference](./cpp_infer/readme.md)
+- [Serving](./pdserving/README.md)
+- [Paddle-Lite](./lite/readme.md)
+- [Paddle.js](./paddlejs/README.md)
+- [Paddle2ONNX](./paddle2onnx/readme.md)
+
+If you need the deployment tutorial of academic algorithm models other than PP-OCR, please directly enter the main page of corresponding algorithms, [entrance](../doc/doc_en/algorithm_overview_en.md)。
\ No newline at end of file

deploy/README_ch.md

+[English](README.md) | 简体中文
+
+# PP-OCR 模型推理部署
+
+- [Paddle 推理部署方式简介](#1)
+- [PP-OCR 推理部署](#2)
+
+<a name="1"></a>
+## Paddle 推理部署方式简介
+
+飞桨提供多种部署方案，以满足不同场景的部署需求，请根据实际情况进行选择：
+
+<div align="center">
+    <img src="../doc/deployment.png" width="800">
+</div>
+
+
+<a name="2"></a>
+## PP-OCR 推理部署
+
+PP-OCR模型已打通多种场景部署方案，点击链接获取具体的使用教程。
+
+- [Python 推理](../doc/doc_ch/inference_ppocr.md)
+- [C++ 推理](./cpp_infer/readme_ch.md)
+- [Serving 服务化部署](./pdserving/README_CN.md)
+- [Paddle-Lite 端侧部署](./lite/readme_ch.md)
+- [Paddle.js 服务化部署](./paddlejs/README_ch.md)
+- [Paddle2ONNX 推理](./paddle2onnx/readme_ch.md)
+
+需要PP-OCR以外的学术算法模型的推理部署，请直接进入相应算法主页面，[入口](../doc/doc_ch/algorithm_overview.md)。
\ No newline at end of file

deploy/cpp_infer/include/args.h

+// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#pragma once
+
+#include <gflags/gflags.h>
+
+// common args
+DECLARE_bool(use_gpu);
+DECLARE_bool(use_tensorrt);
+DECLARE_int32(gpu_id);
+DECLARE_int32(gpu_mem);
+DECLARE_int32(cpu_threads);
+DECLARE_bool(enable_mkldnn);
+DECLARE_string(precision);
+DECLARE_bool(benchmark);
+DECLARE_string(output);
+DECLARE_string(image_dir);
+DECLARE_string(type);
+// detection related
+DECLARE_string(det_model_dir);
+DECLARE_int32(max_side_len);
+DECLARE_double(det_db_thresh);
+DECLARE_double(det_db_box_thresh);
+DECLARE_double(det_db_unclip_ratio);
+DECLARE_bool(use_dilation);
+DECLARE_string(det_db_score_mode);
+DECLARE_bool(visualize);
+// classification related
+DECLARE_bool(use_angle_cls);
+DECLARE_string(cls_model_dir);
+DECLARE_double(cls_thresh);
+DECLARE_int32(cls_batch_num);
+// recognition related
+DECLARE_string(rec_model_dir);
+DECLARE_int32(rec_batch_num);
+DECLARE_string(rec_char_dict_path);
+// forward related
+DECLARE_bool(det);
+DECLARE_bool(rec);
+DECLARE_bool(cls);

deploy/cpp_infer/include/ocr_cls.h

@@ -42,7 +42,8 @@ public:
                       const int &gpu_id, const int &gpu_mem,
                       const int &cpu_math_library_num_threads,
                       const bool &use_mkldnn, const double &cls_thresh,
-                      const bool &use_tensorrt, const std::string &precision) {
+                      const bool &use_tensorrt, const std::string &precision,
+                      const int &cls_batch_num) {
     this->use_gpu_ = use_gpu;
     this->gpu_id_ = gpu_id;
     this->gpu_mem_ = gpu_mem;
@@ -52,14 +53,17 @@ public:
     this->cls_thresh = cls_thresh;
     this->use_tensorrt_ = use_tensorrt;
     this->precision_ = precision;
+    this->cls_batch_num_ = cls_batch_num;
 
     LoadModel(model_dir);
   }
+  double cls_thresh = 0.9;
 
   // Load Paddle inference model
   void LoadModel(const std::string &model_dir);
 
-  cv::Mat Run(cv::Mat &img);
+  void Run(std::vector<cv::Mat> img_list, std::vector<int> &cls_labels,
+           std::vector<float> &cls_scores, std::vector<double> &times);
 
 private:
   std::shared_ptr<Predictor> predictor_;
@@ -69,17 +73,17 @@ private:
   int gpu_mem_ = 4000;
   int cpu_math_library_num_threads_ = 4;
   bool use_mkldnn_ = false;
-  double cls_thresh = 0.5;
 
   std::vector<float> mean_ = {0.5f, 0.5f, 0.5f};
   std::vector<float> scale_ = {1 / 0.5f, 1 / 0.5f, 1 / 0.5f};
   bool is_scale_ = true;
   bool use_tensorrt_ = false;
   std::string precision_ = "fp32";
+  int cls_batch_num_ = 1;
   // pre-process
   ClsResizeImg resize_op_;
   Normalize normalize_op_;
-  Permute permute_op_;
+  PermuteBatch permute_op_;
 
 }; // class Classifier
 

deploy/cpp_infer/include/ocr_det.h

@@ -73,7 +73,7 @@ public:
 
   // Run predictor
   void Run(cv::Mat &img, std::vector<std::vector<std::vector<int>>> &boxes,
-           std::vector<double> *times);
+           std::vector<double> &times);
 
 private:
   std::shared_ptr<Predictor> predictor_;

deploy/cpp_infer/include/ocr_rec.h

@@ -30,7 +30,6 @@
 #include <numeric>
 
 #include <include/ocr_cls.h>
-#include <include/postprocess_op.h>
 #include <include/preprocess_op.h>
 #include <include/utility.h>
 
@@ -68,7 +67,7 @@ public:
   void LoadModel(const std::string &model_dir);
 
   void Run(std::vector<cv::Mat> img_list, std::vector<std::string> &rec_texts,
-           std::vector<float> &rec_text_scores, std::vector<double> *times);
+           std::vector<float> &rec_text_scores, std::vector<double> &times);
 
 private:
   std::shared_ptr<Predictor> predictor_;
@@ -93,9 +92,6 @@ private:
   Normalize normalize_op_;
   PermuteBatch permute_op_;
 
-  // post-process
-  PostProcessor post_processor_;
-
 }; // class CrnnRecognizer
 
 } // namespace PaddleOCR

deploy/cpp_infer/include/paddleocr.h

+// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#pragma once
+
+#include "opencv2/core.hpp"
+#include "opencv2/imgcodecs.hpp"
+#include "opencv2/imgproc.hpp"
+#include "paddle_api.h"
+#include "paddle_inference_api.h"
+#include <chrono>
+#include <iomanip>
+#include <iostream>
+#include <ostream>
+#include <vector>
+
+#include <cstring>
+#include <fstream>
+#include <numeric>
+
+#include <include/ocr_cls.h>
+#include <include/ocr_det.h>
+#include <include/ocr_rec.h>
+#include <include/preprocess_op.h>
+#include <include/utility.h>
+
+using namespace paddle_infer;
+
+namespace PaddleOCR {
+
+class PPOCR {
+public:
+  explicit PPOCR();
+  ~PPOCR();
+  std::vector<std::vector<OCRPredictResult>>
+  ocr(std::vector<cv::String> cv_all_img_names, bool det = true,
+      bool rec = true, bool cls = true);
+
+private:
+  DBDetector *detector_ = nullptr;
+  Classifier *classifier_ = nullptr;
+  CRNNRecognizer *recognizer_ = nullptr;
+
+  void det(cv::Mat img, std::vector<OCRPredictResult> &ocr_results,
+           std::vector<double> &times);
+  void rec(std::vector<cv::Mat> img_list,
+           std::vector<OCRPredictResult> &ocr_results,
+           std::vector<double> &times);
+  void cls(std::vector<cv::Mat> img_list,
+           std::vector<OCRPredictResult> &ocr_results,
+           std::vector<double> &times);
+  void log(std::vector<double> &det_times, std::vector<double> &rec_times,
+           std::vector<double> &cls_times, int img_num);
+};
+
+} // namespace PaddleOCR

deploy/cpp_infer/include/utility.h

@@ -32,13 +32,20 @@
 
 namespace PaddleOCR {
 
+struct OCRPredictResult {
+  std::vector<std::vector<int>> box;
+  std::string text;
+  float score = -1.0;
+  float cls_score;
+  int cls_label = -1;
+};
+
 class Utility {
 public:
   static std::vector<std::string> ReadDict(const std::string &path);
 
-  static void
-  VisualizeBboxes(const cv::Mat &srcimg,
-                  const std::vector<std::vector<std::vector<int>>> &boxes,
+  static void VisualizeBboxes(const cv::Mat &srcimg,
+                              const std::vector<OCRPredictResult> &ocr_result,
                               const std::string &save_path);
 
   template <class ForwardIterator>
@@ -55,6 +62,12 @@ public:
   static std::vector<int> argsort(const std::vector<float> &array);
 
   static std::string basename(const std::string &filename);
+
+  static bool PathExists(const std::string &path);
+
+  static void CreateDir(const std::string &path);
+
+  static void print_result(const std::vector<OCRPredictResult> &ocr_result);
 };
 
 } // namespace PaddleOCR
\ No newline at end of file

deploy/cpp_infer/readme_ch.md

+- [服务器端C++预测](#服务器端c预测)
+  - [1. 准备环境](#1-准备环境)
+    - [1.0 运行准备](#10-运行准备)
+    - [1.1 编译opencv库](#11-编译opencv库)
+    - [1.2 下载或者编译Paddle预测库](#12-下载或者编译paddle预测库)
+      - [1.2.1 直接下载安装](#121-直接下载安装)
+      - [1.2.2 预测库源码编译](#122-预测库源码编译)
+  - [2 开始运行](#2-开始运行)
+    - [2.1 将模型导出为inference model](#21-将模型导出为inference-model)
+    - [2.2 编译PaddleOCR C++预测demo](#22-编译paddleocr-c预测demo)
+    - [2.3 运行demo](#23-运行demo)
+        - [1. 检测+分类+识别：](#1-检测分类识别)
+        - [2. 检测+识别：](#2-检测识别)
+        - [3. 检测：](#3-检测)
+        - [4. 分类+识别：](#4-分类识别)
+        - [5. 识别：](#5-识别)
+        - [6. 分类：](#6-分类)
+  - [3. FAQ](#3-faq)
+
+# 服务器端C++预测
+
+本章节介绍PaddleOCR 模型的的C++部署方法，与之对应的python预测部署方式参考[文档](../../doc/doc_ch/inference.md)。
+C++在性能计算上优于python，因此，在大多数CPU、GPU部署场景，多采用C++的部署方式，本节将介绍如何在Linux\Windows (CPU\GPU)环境下配置C++环境并完成
+PaddleOCR模型部署。
+
+
+<a name="1"></a>
+
+## 1. 准备环境
+
+<a name="10"></a>
+
+### 1.0 运行准备
+
+- Linux环境，推荐使用docker。
+- Windows环境。
+
+* 该文档主要介绍基于Linux环境的PaddleOCR C++预测流程，如果需要在Windows下基于预测库进行C++预测，具体编译方法请参考[Windows下编译教程](./docs/windows_vs2019_build.md)
+
+<a name="11"></a>
+
+### 1.1 编译opencv库
+
+* 首先需要从opencv官网上下载在Linux环境下源码编译的包，以opencv3.4.7为例，下载命令如下。
+
+```bash
+cd deploy/cpp_infer
+wget https://paddleocr.bj.bcebos.com/libs/opencv/opencv-3.4.7.tar.gz
+tar -xf opencv-3.4.7.tar.gz
+```
+
+最终可以在当前目录下看到`opencv-3.4.7/`的文件夹。
+
+* 编译opencv，设置opencv源码路径(`root_path`)以及安装路径(`install_path`)。进入opencv源码路径下，按照下面的方式进行编译。
+
+```shell
+root_path="your_opencv_root_path"
+install_path=${root_path}/opencv3
+build_dir=${root_path}/build
+
+rm -rf ${build_dir}
+mkdir ${build_dir}
+cd ${build_dir}
+
+cmake .. \
+    -DCMAKE_INSTALL_PREFIX=${install_path} \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DBUILD_SHARED_LIBS=OFF \
+    -DWITH_IPP=OFF \
+    -DBUILD_IPP_IW=OFF \
+    -DWITH_LAPACK=OFF \
+    -DWITH_EIGEN=OFF \
+    -DCMAKE_INSTALL_LIBDIR=lib64 \
+    -DWITH_ZLIB=ON \
+    -DBUILD_ZLIB=ON \
+    -DWITH_JPEG=ON \
+    -DBUILD_JPEG=ON \
+    -DWITH_PNG=ON \
+    -DBUILD_PNG=ON \
+    -DWITH_TIFF=ON \
+    -DBUILD_TIFF=ON
+
+make -j
+make install
+```
+
+也可以直接修改`tools/build_opencv.sh`的内容，然后直接运行下面的命令进行编译。
+
+```shell
+sh tools/build_opencv.sh
+```
+
+其中`root_path`为下载的opencv源码路径，`install_path`为opencv的安装路径，`make install`完成之后，会在该文件夹下生成opencv头文件和库文件，用于后面的OCR代码编译。
+
+最终在安装路径下的文件结构如下所示。
+
+```
+opencv3/
+|-- bin
+|-- include
+|-- lib
+|-- lib64
+|-- share
+```
+
+<a name="12"></a>
+
+### 1.2 下载或者编译Paddle预测库
+
+* 有2种方式获取Paddle预测库，下面进行详细介绍。
+
+
+#### 1.2.1 直接下载安装
+
+* [Paddle预测库官网](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html#linux) 上提供了不同cuda版本的Linux预测库，可以在官网查看并选择合适的预测库版本（*建议选择paddle版本>=2.0.1版本的预测库* ）。
+
+* 下载之后使用下面的方法解压。
+
+```
+tar -xf paddle_inference.tgz
+```
+
+最终会在当前的文件夹中生成`paddle_inference/`的子文件夹。
+
+#### 1.2.2 预测库源码编译
+* 如果希望获取最新预测库特性，可以从Paddle github上克隆最新代码，源码编译预测库。
+* 可以参考[Paddle预测库安装编译说明](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0/guides/05_inference_deployment/inference/build_and_install_lib_cn.html#congyuanmabianyi) 的说明，从github上获取Paddle代码，然后进行编译，生成最新的预测库。使用git获取代码方法如下。
+
+```shell
+git clone https://github.com/PaddlePaddle/Paddle.git
+git checkout develop
+```
+
+* 进入Paddle目录后，编译方法如下。
+
+```shell
+rm -rf build
+mkdir build
+cd build
+
+cmake  .. \
+    -DWITH_CONTRIB=OFF \
+    -DWITH_MKL=ON \
+    -DWITH_MKLDNN=ON  \
+    -DWITH_TESTING=OFF \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DWITH_INFERENCE_API_TEST=OFF \
+    -DON_INFER=ON \
+    -DWITH_PYTHON=ON
+make -j
+make inference_lib_dist
+```
+
+更多编译参数选项介绍可以参考[文档说明](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0/guides/05_inference_deployment/inference/build_and_install_lib_cn.html#congyuanmabianyi)。
+
+
+* 编译完成之后，可以在`build/paddle_inference_install_dir/`文件下看到生成了以下文件及文件夹。
+
+```
+build/paddle_inference_install_dir/
+|-- CMakeCache.txt
+|-- paddle
+|-- third_party
+|-- version.txt
+```
+
+其中`paddle`就是C++预测所需的Paddle库，`version.txt`中包含当前预测库的版本信息。
+
+<a name="2"></a>
+
+## 2 开始运行
+
+<a name="21"></a>
+
+### 2.1 将模型导出为inference model
+
+* 可以参考[模型预测章节](../../doc/doc_ch/inference.md)，导出inference model，用于模型预测。模型导出之后，假设放在`inference`目录下，则目录结构如下。
+
+```
+inference/
+|-- det_db
+|   |--inference.pdiparams
+|   |--inference.pdmodel
+|-- rec_rcnn
+|   |--inference.pdiparams
+|   |--inference.pdmodel
+|-- cls
+|   |--inference.pdiparams
+|   |--inference.pdmodel
+```
+
+<a name="22"></a>
+
+### 2.2 编译PaddleOCR C++预测demo
+
+* 编译命令如下，其中Paddle C++预测库、opencv等其他依赖库的地址需要换成自己机器上的实际地址。
+
+```shell
+sh tools/build.sh
+```
+
+* 具体的，需要修改`tools/build.sh`中环境路径，相关内容如下：
+
+```shell
+OPENCV_DIR=your_opencv_dir
+LIB_DIR=your_paddle_inference_dir
+CUDA_LIB_DIR=your_cuda_lib_dir
+CUDNN_LIB_DIR=/your_cudnn_lib_dir
+```
+
+其中，`OPENCV_DIR`为opencv编译安装的地址；`LIB_DIR`为下载(`paddle_inference`文件夹)或者编译生成的Paddle预测库地址(`build/paddle_inference_install_dir`文件夹)；`CUDA_LIB_DIR`为cuda库文件地址，在docker中为`/usr/local/cuda/lib64`；`CUDNN_LIB_DIR`为cudnn库文件地址，在docker中为`/usr/lib/x86_64-linux-gnu/`。**注意：以上路径都写绝对路径，不要写相对路径。**
+
+
+* 编译完成之后，会在`build`文件夹下生成一个名为`ppocr`的可执行文件。
+
+<a name="23"></a>
+
+### 2.3 运行demo
+
+运行方式：  
+```shell
+./build/ppocr [--param1] [--param2] [...]
+```
+具体命令如下：
+
+##### 1. 检测+分类+识别：
+```shell
+./build/ppocr --det_model_dir=inference/det_db \
+    --rec_model_dir=inference/rec_rcnn \
+    --cls_model_dir=inference/cls \
+    --image_dir=../../doc/imgs/12.jpg \
+    --use_angle_cls=true \
+    --det=true \
+    --rec=true \
+    --cls=true \
+```
+
+##### 2. 检测+识别：
+```shell
+./build/ppocr --det_model_dir=inference/det_db \
+    --rec_model_dir=inference/rec_rcnn \
+    --image_dir=../../doc/imgs/12.jpg \
+    --use_angle_cls=false \
+    --det=true \
+    --rec=true \
+    --cls=false \
+```
+
+##### 3. 检测：
+```shell
+./build/ppocr --det_model_dir=inference/det_db \
+    --image_dir=../../doc/imgs/12.jpg \
+    --det=true \
+    --rec=false
+```
+
+##### 4. 分类+识别：
+```shell
+./build/ppocr --rec_model_dir=inference/rec_rcnn \
+    --cls_model_dir=inference/cls \
+    --image_dir=../../doc/imgs_words/ch/word_1.jpg \
+    --use_angle_cls=true \
+    --det=false \
+    --rec=true \
+    --cls=true \
+```
+
+##### 5. 识别：
+```shell
+./build/ppocr --rec_model_dir=inference/rec_rcnn \
+    --image_dir=../../doc/imgs_words/ch/word_1.jpg \
+    --use_angle_cls=false \
+    --det=false \
+    --rec=true \
+    --cls=false \
+```
+
+##### 6. 分类：
+```shell
+./build/ppocr --cls_model_dir=inference/cls \
+    --cls_model_dir=inference/cls \
+    --image_dir=../../doc/imgs_words/ch/word_1.jpg \
+    --use_angle_cls=true \
+    --det=false \
+    --rec=false \
+    --cls=true \
+```
+
+更多支持的可调节参数解释如下：
+
+- 通用参数
+
+|参数名称|类型|默认参数|意义|
+| :---: | :---: | :---: | :---: |
+|use_gpu|bool|false|是否使用GPU|
+|gpu_id|int|0|GPU id，使用GPU时有效|
+|gpu_mem|int|4000|申请的GPU内存|
+|cpu_math_library_num_threads|int|10|CPU预测时的线程数，在机器核数充足的情况下，该值越大，预测速度越快|
+|enable_mkldnn|bool|true|是否使用mkldnn库|
+|output|str|./output|可视化结果保存的路径|
+
+- 前向相关
+
+|参数名称|类型|默认参数|意义|
+| :---: | :---: | :---: | :---: |
+|det|bool|true|前向是否执行文字检测|
+|rec|bool|true|前向是否执行文字识别|
+|cls|bool|false|前向是否执行文字方向分类|
+
+
+- 检测模型相关
+
+|参数名称|类型|默认参数|意义|
+| :---: | :---: | :---: | :---: |
+|det_model_dir|string|-|检测模型inference model地址|
+|max_side_len|int|960|输入图像长宽大于960时，等比例缩放图像，使得图像最长边为960|
+|det_db_thresh|float|0.3|用于过滤DB预测的二值化图像，设置为0.-0.3对结果影响不明显|
+|det_db_box_thresh|float|0.5|DB后处理过滤box的阈值，如果检测存在漏框情况，可酌情减小|
+|det_db_unclip_ratio|float|1.6|表示文本框的紧致程度，越小则文本框更靠近文本|
+|det_db_score_mode|string|slow|slow:使用多边形框计算bbox score，fast:使用矩形框计算。矩形框计算速度更快，多边形框对弯曲文本区域计算更准确。|
+|visualize|bool|true|是否对结果进行可视化，为1时，预测结果会保存在`output`字段指定的文件夹下和输入图像同名的图像上。|
+
+- 方向分类器相关
+
+|参数名称|类型|默认参数|意义|
+| :---: | :---: | :---: | :---: |
+|use_angle_cls|bool|false|是否使用方向分类器|
+|cls_model_dir|string|-|方向分类器inference model地址|
+|cls_thresh|float|0.9|方向分类器的得分阈值|
+|cls_batch_num|int|1|方向分类器batchsize|
+
+- 识别模型相关
+
+|参数名称|类型|默认参数|意义|
+| :---: | :---: | :---: | :---: |
+|rec_model_dir|string|-|识别模型inference model地址|
+|rec_char_dict_path|string|../../ppocr/utils/ppocr_keys_v1.txt|字典文件|
+|rec_batch_num|int|6|识别模型batchsize|
+
+
+* PaddleOCR也支持多语言的预测，更多支持的语言和模型可以参考[识别文档](../../doc/doc_ch/recognition.md)中的多语言字典与模型部分，如果希望进行多语言预测，只需将修改`rec_char_dict_path`（字典文件路径）以及`rec_model_dir`（inference模型路径）字段即可。
+
+最终屏幕上会输出检测结果如下。
+
+```bash
+predict img: ../../doc/imgs/12.jpg
+../../doc/imgs/12.jpg
+0       det boxes: [[79,553],[399,541],[400,573],[80,585]] rec text: 打浦路252935号 rec score: 0.933757
+1       det boxes: [[31,509],[510,488],[511,529],[33,549]] rec text: 绿洲仕格维花园公寓 rec score: 0.951745
+2       det boxes: [[181,456],[395,448],[396,480],[182,488]] rec text: 打浦路15号 rec score: 0.91956
+3       det boxes: [[43,413],[480,391],[481,428],[45,450]] rec text: 上海斯格威铂尔多大酒店 rec score: 0.915914
+The detection visualized image saved in ./output//12.jpg
+```
+
+## 3. FAQ
+
+ 1.  遇到报错 `unable to access 'https://github.com/LDOUBLEV/AutoLog.git/': gnutls_handshake() failed: The TLS connection was non-properly terminated.`， 将 `deploy/cpp_infer/external-cmake/auto-log.cmake` 中的github地址改为 https://gitee.com/Double_V/AutoLog 地址即可。

deploy/cpp_infer/readme_en.md

-- [Server-side C++ Inference](#server-side-c-inference)
-  - [1. Prepare the Environment](#1-prepare-the-environment)
-    - [Environment](#environment)
-    - [1.1 Compile OpenCV](#11-compile-opencv)
-    - [1.2 Compile or Download or the Paddle Inference Library](#12-compile-or-download-or-the-paddle-inference-library)
-      - [1.2.1 Direct download and installation](#121-direct-download-and-installation)
-      - [1.2.2 Compile the inference source code](#122-compile-the-inference-source-code)
-  - [2. Compile and Run the Demo](#2-compile-and-run-the-demo)
-    - [2.1 Export the inference model](#21-export-the-inference-model)
-    - [2.2 Compile PaddleOCR C++ inference demo](#22-compile-paddleocr-c-inference-demo)
-    - [Run the demo](#run-the-demo)
-        - [1. run det demo:](#1-run-det-demo)
-        - [2. run rec demo:](#2-run-rec-demo)
-        - [3. run system demo:](#3-run-system-demo)
-  - [3. FAQ](#3-faq)
-
-# Server-side C++ Inference
-
-This chapter introduces the C++ deployment steps of the PaddleOCR model. The corresponding Python predictive deployment method refers to [document](../../doc/doc_ch/inference.md).
-C++ is better than python in terms of performance. Therefore, in CPU and GPU deployment scenarios, C++ deployment is mostly used.
-This section will introduce how to configure the C++ environment and deploy PaddleOCR in Linux (CPU\GPU) environment. For Windows deployment please refer to [Windows](./docs/windows_vs2019_build.md) compilation guidelines.
-
-
-## 1. Prepare the Environment
-
-### Environment
-
-- Linux, docker is recommended.
-- Windows.
-
-
-### 1.1 Compile OpenCV
-
-* First of all, you need to download the source code compiled package in the Linux environment from the OpenCV official website. Taking OpenCV 3.4.7 as an example, the download command is as follows.
-
-```bash
-cd deploy/cpp_infer
-wget https://paddleocr.bj.bcebos.com/libs/opencv/opencv-3.4.7.tar.gz
-tar -xf opencv-3.4.7.tar.gz
-```
-
-Finally, you will see the folder of `opencv-3.4.7/` in the current directory.
-
-* Compile OpenCV, the OpenCV source path (`root_path`) and installation path (`install_path`) should be set by yourself. Enter the OpenCV source code path and compile it in the following way.
-
-
-```shell
-root_path=your_opencv_root_path
-install_path=${root_path}/opencv3
-
-rm -rf build
-mkdir build
-cd build
-
-cmake .. \
-    -DCMAKE_INSTALL_PREFIX=${install_path} \
-    -DCMAKE_BUILD_TYPE=Release \
-    -DBUILD_SHARED_LIBS=OFF \
-    -DWITH_IPP=OFF \
-    -DBUILD_IPP_IW=OFF \
-    -DWITH_LAPACK=OFF \
-    -DWITH_EIGEN=OFF \
-    -DCMAKE_INSTALL_LIBDIR=lib64 \
-    -DWITH_ZLIB=ON \
-    -DBUILD_ZLIB=ON \
-    -DWITH_JPEG=ON \
-    -DBUILD_JPEG=ON \
-    -DWITH_PNG=ON \
-    -DBUILD_PNG=ON \
-    -DWITH_TIFF=ON \
-    -DBUILD_TIFF=ON
-
-make -j
-make install
-```
-
-In the above commands, `root_path` is the downloaded OpenCV source code path, and `install_path` is the installation path of OpenCV. After `make install` is completed, the OpenCV header file and library file will be generated in this folder for later OCR source code compilation.
-
-
-
-The final file structure under the OpenCV installation path is as follows.
-
-```
-opencv3/
-|-- bin
-|-- include
-|-- lib
-|-- lib64
-|-- share
-```
-
-### 1.2 Compile or Download or the Paddle Inference Library
-
-* There are 2 ways to obtain the Paddle inference library, described in detail below.
-
-#### 1.2.1 Direct download and installation
-
-[Paddle inference library official website](https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html#linux). You can review and select the appropriate version of the inference library on the official website.
-
-
-* After downloading, use the following command to extract files.
-
-```
-tar -xf paddle_inference.tgz
-```
-
-Finally you will see the the folder of `paddle_inference/` in the current path.
-
-#### 1.2.2 Compile the inference source code
-* If you want to get the latest Paddle inference library features, you can download the latest code from Paddle GitHub repository and compile the inference library from the source code. It is recommended to download the inference library with paddle version greater than or equal to 2.0.1.
-* You can refer to [Paddle inference library] (https://www.paddlepaddle.org.cn/documentation/docs/en/advanced_guide/inference_deployment/inference/build_and_install_lib_en.html) to get the Paddle source code from GitHub, and then compile To generate the latest inference library. The method of using git to access the code is as follows.
-
-
-```shell
-git clone https://github.com/PaddlePaddle/Paddle.git
-git checkout develop
-```
-
-* Enter the Paddle directory and run the following commands to compile the paddle inference library.
-
-```shell
-rm -rf build
-mkdir build
-cd build
-
-cmake  .. \
-    -DWITH_CONTRIB=OFF \
-    -DWITH_MKL=ON \
-    -DWITH_MKLDNN=ON  \
-    -DWITH_TESTING=OFF \
-    -DCMAKE_BUILD_TYPE=Release \
-    -DWITH_INFERENCE_API_TEST=OFF \
-    -DON_INFER=ON \
-    -DWITH_PYTHON=ON
-make -j
-make inference_lib_dist
-```
-
-For more compilation parameter options, please refer to the [document](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0/guides/05_inference_deployment/inference/build_and_install_lib_cn.html#congyuanmabianyi).
-
-
-* After the compilation process, you can see the following files in the folder of `build/paddle_inference_install_dir/`.
-
-```
-build/paddle_inference_install_dir/
-|-- CMakeCache.txt
-|-- paddle
-|-- third_party
-|-- version.txt
-```
-
-`paddle` is the Paddle library required for C++ prediction later, and `version.txt` contains the version information of the current inference library.
-
-
-## 2. Compile and Run the Demo
-
-### 2.1 Export the inference model
-
-* You can refer to [Model inference](../../doc/doc_ch/inference.md) and export the inference model. After the model is exported, assuming it is placed in the `inference` directory, the directory structure is as follows.
-
-```
-inference/
-|-- det_db
-|   |--inference.pdiparams
-|   |--inference.pdmodel
-|-- rec_rcnn
-|   |--inference.pdiparams
-|   |--inference.pdmodel
-```
-
-
-### 2.2 Compile PaddleOCR C++ inference demo
-
-
-* The compilation commands are as follows. The addresses of Paddle C++ inference library, opencv and other Dependencies need to be replaced with the actual addresses on your own machines.
-
-```shell
-sh tools/build.sh
-```
-
-Specifically, you should modify the paths in `tools/build.sh`. The related content is as follows.
-
-```shell
-OPENCV_DIR=your_opencv_dir
-LIB_DIR=your_paddle_inference_dir
-CUDA_LIB_DIR=your_cuda_lib_dir
-CUDNN_LIB_DIR=your_cudnn_lib_dir
-```
-
-`OPENCV_DIR` is the OpenCV installation path; `LIB_DIR` is the download (`paddle_inference` folder)
-or the generated Paddle inference library path (`build/paddle_inference_install_dir` folder);
-`CUDA_LIB_DIR` is the CUDA library file path, in docker; it is `/usr/local/cuda/lib64`; `CUDNN_LIB_DIR` is the cuDNN library file path, in docker it is `/usr/lib/x86_64-linux-gnu/`.
-
-
-* After the compilation is completed, an executable file named `ppocr` will be generated in the `build` folder.
-
-
-### Run the demo
-Execute the built executable file:
-```shell
-./build/ppocr <mode> [--param1] [--param2] [...]
-```
-`mode` is a required parameter，and the valid values are
-
-mode value | Model used
------|------
-det  | Detection only
-rec  | Recognition only
-system | End-to-end system
-
-Specifically,
-
-##### 1. run det demo:
-```shell
-./build/ppocr det \
-    --det_model_dir=inference/ch_ppocr_mobile_v2.0_det_infer \
-    --image_dir=../../doc/imgs/12.jpg
-```
-##### 2. run rec demo:
-```shell
-./build/ppocr rec \
-    --rec_model_dir=inference/ch_ppocr_mobile_v2.0_rec_infer \
-    --image_dir=../../doc/imgs_words/ch/
-```
-##### 3. run system demo:
-```shell
-# without text direction classifier
-./build/ppocr system \
-    --det_model_dir=inference/ch_ppocr_mobile_v2.0_det_infer \
-    --rec_model_dir=inference/ch_ppocr_mobile_v2.0_rec_infer \
-    --image_dir=../../doc/imgs/12.jpg
-# with text direction classifier
-./build/ppocr system \
-    --det_model_dir=inference/ch_ppocr_mobile_v2.0_det_infer \
-    --use_angle_cls=true \
-    --cls_model_dir=inference/ch_ppocr_mobile_v2.0_cls_infer \
-    --rec_model_dir=inference/ch_ppocr_mobile_v2.0_rec_infer \
-    --image_dir=../../doc/imgs/12.jpg
-```
-
-More parameters are as follows,
-
-- Common parameters
-
-|parameter|data type|default|meaning|
-| --- | --- | --- | --- |
-|use_gpu|bool|false|Whether to use GPU|
-|gpu_id|int|0|GPU id when use_gpu is true|
-|gpu_mem|int|4000|GPU memory requested|
-|cpu_math_library_num_threads|int|10|Number of threads when using CPU inference. When machine cores is enough, the large the value, the faster the inference speed|
-|enable_mkldnn|bool|true|Whether to use mkdlnn library|
-|output|str|./output|Path where visualization results are saved|
-
-- Detection related parameters
-
-|parameter|data type|default|meaning|
-| --- | --- | --- | --- |
-|det_model_dir|string|-|Address of detection inference model|
-|max_side_len|int|960|Limit the maximum image height and width to 960|
-|det_db_thresh|float|0.3|Used to filter the binarized image of DB prediction, setting 0.-0.3 has no obvious effect on the result|
-|det_db_box_thresh|float|0.5|DB post-processing filter box threshold, if there is a missing box detected, it can be reduced as appropriate|
-|det_db_unclip_ratio|float|1.6|Indicates the compactness of the text box, the smaller the value, the closer the text box to the text|
-|det_db_score_mode|string|slow| slow: use polygon box to calculate bbox score, fast: use rectangle box to calculate. Use rectangular box to calculate faster, and polygonal box more accurate for curved text area.|
-|visualize|bool|true|Whether to visualize the results，when it is set as true, the prediction results will be saved in the folder specified by the `output` field on an image with the same name as the input image.|
-
-- Classifier related parameters
-
-|parameter|data type|default|meaning|
-| --- | --- | --- | --- |
-|use_angle_cls|bool|false|Whether to use the direction classifier|
-|cls_model_dir|string|-|Address of direction classifier inference model|
-|cls_thresh|float|0.9|Score threshold of the  direction classifier|
-
-- Recognition related parameters
-
-|parameter|data type|default|meaning|
-| --- | --- | --- | --- |
-|rec_model_dir|string|-|Address of recognition inference model|
-|rec_char_dict_path|string|../../ppocr/utils/ppocr_keys_v1.txt|dictionary file|
-
-* Multi-language inference is also supported in PaddleOCR, you can refer to [recognition tutorial](../../doc/doc_en/recognition_en.md) for more supported languages and models in PaddleOCR. Specifically, if you want to infer using multi-language models, you just need to modify values of `rec_char_dict_path` and `rec_model_dir`.
-
-
-The detection results will be shown on the screen, which is as follows.
-
-<div align="center">
-    <img src="./imgs/cpp_infer_pred_12.png" width="600">
-</div>
-
-
-## 3. FAQ
-
- 1.  Encountered the error `unable to access 'https://github.com/LDOUBLEV/AutoLog.git/': gnutls_handshake() failed: The TLS connection was non-properly terminated.`, change the github address in `deploy/cpp_infer/external-cmake/auto-log.cmake` to the https://gitee.com/Double_V/AutoLog address.

deploy/cpp_infer/src/args.cpp

+// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <gflags/gflags.h>
+
+// common args
+DEFINE_bool(use_gpu, false, "Infering with GPU or CPU.");
+DEFINE_bool(use_tensorrt, false, "Whether use tensorrt.");
+DEFINE_int32(gpu_id, 0, "Device id of GPU to execute.");
+DEFINE_int32(gpu_mem, 4000, "GPU id when infering with GPU.");
+DEFINE_int32(cpu_threads, 10, "Num of threads with CPU.");
+DEFINE_bool(enable_mkldnn, false, "Whether use mkldnn with CPU.");
+DEFINE_string(precision, "fp32", "Precision be one of fp32/fp16/int8");
+DEFINE_bool(benchmark, false, "Whether use benchmark.");
+DEFINE_string(output, "./output/", "Save benchmark log path.");
+DEFINE_string(image_dir, "", "Dir of input image.");
+DEFINE_string(
+    type, "ocr",
+    "Perform ocr or structure, the value is selected in ['ocr','structure'].");
+// detection related
+DEFINE_string(det_model_dir, "", "Path of det inference model.");
+DEFINE_int32(max_side_len, 960, "max_side_len of input image.");
+DEFINE_double(det_db_thresh, 0.3, "Threshold of det_db_thresh.");
+DEFINE_double(det_db_box_thresh, 0.6, "Threshold of det_db_box_thresh.");
+DEFINE_double(det_db_unclip_ratio, 1.5, "Threshold of det_db_unclip_ratio.");
+DEFINE_bool(use_dilation, false, "Whether use the dilation on output map.");
+DEFINE_string(det_db_score_mode, "slow", "Whether use polygon score.");
+DEFINE_bool(visualize, true, "Whether show the detection results.");
+// classification related
+DEFINE_bool(use_angle_cls, false, "Whether use use_angle_cls.");
+DEFINE_string(cls_model_dir, "", "Path of cls inference model.");
+DEFINE_double(cls_thresh, 0.9, "Threshold of cls_thresh.");
+DEFINE_int32(cls_batch_num, 1, "cls_batch_num.");
+// recognition related
+DEFINE_string(rec_model_dir, "", "Path of rec inference model.");
+DEFINE_int32(rec_batch_num, 6, "rec_batch_num.");
+DEFINE_string(rec_char_dict_path, "../../ppocr/utils/ppocr_keys_v1.txt",
+              "Path of dictionary.");
+
+// ocr forward related
+DEFINE_bool(det, true, "Whether use det in forward.");
+DEFINE_bool(rec, true, "Whether use rec in forward.");
+DEFINE_bool(cls, false, "Whether use cls in forward.");
\ No newline at end of file

deploy/cpp_infer/src/main.cpp

@@ -11,278 +11,19 @@
 // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 // See the License for the specific language governing permissions and
 // limitations under the License.
-
-#include "omp.h"
 #include "opencv2/core.hpp"
 #include "opencv2/imgcodecs.hpp"
 #include "opencv2/imgproc.hpp"
-#include <chrono>
-#include <iomanip>
 #include <iostream>
-#include <ostream>
-#include <sys/stat.h>
 #include <vector>
 
-#include <cstring>
-#include <fstream>
-#include <numeric>
-
-#include <include/ocr_cls.h>
-#include <include/ocr_det.h>
-#include <include/ocr_rec.h>
-#include <include/utility.h>
-#include <sys/stat.h>
-
-#include "auto_log/autolog.h"
-#include <gflags/gflags.h>
-
-// common args
-DEFINE_bool(use_gpu, false, "Infering with GPU or CPU.");
-DEFINE_bool(use_tensorrt, false, "Whether use tensorrt.");
-DEFINE_int32(gpu_id, 0, "Device id of GPU to execute.");
-DEFINE_int32(gpu_mem, 4000, "GPU id when infering with GPU.");
-DEFINE_int32(cpu_threads, 10, "Num of threads with CPU.");
-DEFINE_bool(enable_mkldnn, false, "Whether use mkldnn with CPU.");
-DEFINE_string(precision, "fp32", "Precision be one of fp32/fp16/int8");
-DEFINE_bool(benchmark, false, "Whether use benchmark.");
-DEFINE_string(output, "./output/", "Save benchmark log path.");
-DEFINE_string(image_dir, "", "Dir of input image.");
-DEFINE_bool(visualize, true, "Whether show the detection results.");
-// detection related
-DEFINE_string(det_model_dir, "", "Path of det inference model.");
-DEFINE_int32(max_side_len, 960, "max_side_len of input image.");
-DEFINE_double(det_db_thresh, 0.3, "Threshold of det_db_thresh.");
-DEFINE_double(det_db_box_thresh, 0.6, "Threshold of det_db_box_thresh.");
-DEFINE_double(det_db_unclip_ratio, 1.5, "Threshold of det_db_unclip_ratio.");
-DEFINE_bool(use_dilation, false, "Whether use the dilation on output map.");
-DEFINE_string(det_db_score_mode, "slow", "Whether use polygon score.");
-// classification related
-DEFINE_bool(use_angle_cls, false, "Whether use use_angle_cls.");
-DEFINE_string(cls_model_dir, "", "Path of cls inference model.");
-DEFINE_double(cls_thresh, 0.9, "Threshold of cls_thresh.");
-// recognition related
-DEFINE_string(rec_model_dir, "", "Path of rec inference model.");
-DEFINE_int32(rec_batch_num, 6, "rec_batch_num.");
-DEFINE_string(rec_char_dict_path, "../../ppocr/utils/ppocr_keys_v1.txt",
-              "Path of dictionary.");
+#include <include/args.h>
+#include <include/paddleocr.h>
 
-using namespace std;
-using namespace cv;
 using namespace PaddleOCR;
 
-static bool PathExists(const std::string &path) {
-#ifdef _WIN32
-  struct _stat buffer;
-  return (_stat(path.c_str(), &buffer) == 0);
-#else
-  struct stat buffer;
-  return (stat(path.c_str(), &buffer) == 0);
-#endif // !_WIN32
-}
-
-int main_det(std::vector<cv::String> cv_all_img_names) {
-  std::vector<double> time_info = {0, 0, 0};
-  DBDetector det(FLAGS_det_model_dir, FLAGS_use_gpu, FLAGS_gpu_id,
-                 FLAGS_gpu_mem, FLAGS_cpu_threads, FLAGS_enable_mkldnn,
-                 FLAGS_max_side_len, FLAGS_det_db_thresh,
-                 FLAGS_det_db_box_thresh, FLAGS_det_db_unclip_ratio,
-                 FLAGS_det_db_score_mode, FLAGS_use_dilation,
-                 FLAGS_use_tensorrt, FLAGS_precision);
-
-  if (!PathExists(FLAGS_output)) {
-    mkdir(FLAGS_output.c_str(), 0777);
-  }
-
-  for (int i = 0; i < cv_all_img_names.size(); ++i) {
-    if (!FLAGS_benchmark) {
-      cout << "The predict img: " << cv_all_img_names[i] << endl;
-    }
-
-    cv::Mat srcimg = cv::imread(cv_all_img_names[i], cv::IMREAD_COLOR);
-    if (!srcimg.data) {
-      std::cerr << "[ERROR] image read failed! image path: "
-                << cv_all_img_names[i] << endl;
-      exit(1);
-    }
-    std::vector<std::vector<std::vector<int>>> boxes;
-    std::vector<double> det_times;
-
-    det.Run(srcimg, boxes, &det_times);
-    // visualization
-    if (FLAGS_visualize) {
-      std::string file_name = Utility::basename(cv_all_img_names[i]);
-      Utility::VisualizeBboxes(srcimg, boxes, FLAGS_output + "/" + file_name);
-    }
-    time_info[0] += det_times[0];
-    time_info[1] += det_times[1];
-    time_info[2] += det_times[2];
-
-    if (FLAGS_benchmark) {
-      cout << cv_all_img_names[i] << "\t[";
-      for (int n = 0; n < boxes.size(); n++) {
-        cout << '[';
-        for (int m = 0; m < boxes[n].size(); m++) {
-          cout << '[' << boxes[n][m][0] << ',' << boxes[n][m][1] << "]";
-          if (m != boxes[n].size() - 1) {
-            cout << ',';
-          }
-        }
-        cout << ']';
-        if (n != boxes.size() - 1) {
-          cout << ',';
-        }
-      }
-      cout << ']' << endl;
-    }
-  }
-
-  if (FLAGS_benchmark) {
-    AutoLogger autolog("ocr_det", FLAGS_use_gpu, FLAGS_use_tensorrt,
-                       FLAGS_enable_mkldnn, FLAGS_cpu_threads, 1, "dynamic",
-                       FLAGS_precision, time_info, cv_all_img_names.size());
-    autolog.report();
-  }
-  return 0;
-}
-
-int main_rec(std::vector<cv::String> cv_all_img_names) {
-  std::vector<double> time_info = {0, 0, 0};
-
-  std::string rec_char_dict_path = FLAGS_rec_char_dict_path;
-  cout << "label file: " << rec_char_dict_path << endl;
-
-  CRNNRecognizer rec(FLAGS_rec_model_dir, FLAGS_use_gpu, FLAGS_gpu_id,
-                     FLAGS_gpu_mem, FLAGS_cpu_threads, FLAGS_enable_mkldnn,
-                     rec_char_dict_path, FLAGS_use_tensorrt, FLAGS_precision,
-                     FLAGS_rec_batch_num);
-
-  std::vector<cv::Mat> img_list;
-  for (int i = 0; i < cv_all_img_names.size(); ++i) {
-    cv::Mat srcimg = cv::imread(cv_all_img_names[i], cv::IMREAD_COLOR);
-    if (!srcimg.data) {
-      std::cerr << "[ERROR] image read failed! image path: "
-                << cv_all_img_names[i] << endl;
-      exit(1);
-    }
-    img_list.push_back(srcimg);
-  }
-  std::vector<std::string> rec_texts(img_list.size(), "");
-  std::vector<float> rec_text_scores(img_list.size(), 0);
-  std::vector<double> rec_times;
-  rec.Run(img_list, rec_texts, rec_text_scores, &rec_times);
-  // output rec results
-  for (int i = 0; i < rec_texts.size(); i++) {
-    cout << "The predict img: " << cv_all_img_names[i] << "\t" << rec_texts[i]
-         << "\t" << rec_text_scores[i] << endl;
-  }
-  time_info[0] += rec_times[0];
-  time_info[1] += rec_times[1];
-  time_info[2] += rec_times[2];
-
-  if (FLAGS_benchmark) {
-    AutoLogger autolog("ocr_rec", FLAGS_use_gpu, FLAGS_use_tensorrt,
-                       FLAGS_enable_mkldnn, FLAGS_cpu_threads,
-                       FLAGS_rec_batch_num, "dynamic", FLAGS_precision,
-                       time_info, cv_all_img_names.size());
-    autolog.report();
-  }
-  return 0;
-}
-
-int main_system(std::vector<cv::String> cv_all_img_names) {
-  std::vector<double> time_info_det = {0, 0, 0};
-  std::vector<double> time_info_rec = {0, 0, 0};
-
-  if (!PathExists(FLAGS_output)) {
-    mkdir(FLAGS_output.c_str(), 0777);
-  }
-
-  DBDetector det(FLAGS_det_model_dir, FLAGS_use_gpu, FLAGS_gpu_id,
-                 FLAGS_gpu_mem, FLAGS_cpu_threads, FLAGS_enable_mkldnn,
-                 FLAGS_max_side_len, FLAGS_det_db_thresh,
-                 FLAGS_det_db_box_thresh, FLAGS_det_db_unclip_ratio,
-                 FLAGS_det_db_score_mode, FLAGS_use_dilation,
-                 FLAGS_use_tensorrt, FLAGS_precision);
-
-  Classifier *cls = nullptr;
-  if (FLAGS_use_angle_cls) {
-    cls = new Classifier(FLAGS_cls_model_dir, FLAGS_use_gpu, FLAGS_gpu_id,
-                         FLAGS_gpu_mem, FLAGS_cpu_threads, FLAGS_enable_mkldnn,
-                         FLAGS_cls_thresh, FLAGS_use_tensorrt, FLAGS_precision);
-  }
-
-  std::string rec_char_dict_path = FLAGS_rec_char_dict_path;
-  cout << "label file: " << rec_char_dict_path << endl;
-
-  CRNNRecognizer rec(FLAGS_rec_model_dir, FLAGS_use_gpu, FLAGS_gpu_id,
-                     FLAGS_gpu_mem, FLAGS_cpu_threads, FLAGS_enable_mkldnn,
-                     rec_char_dict_path, FLAGS_use_tensorrt, FLAGS_precision,
-                     FLAGS_rec_batch_num);
-
-  for (int i = 0; i < cv_all_img_names.size(); ++i) {
-    cout << "The predict img: " << cv_all_img_names[i] << endl;
-
-    cv::Mat srcimg = cv::imread(cv_all_img_names[i], cv::IMREAD_COLOR);
-    if (!srcimg.data) {
-      std::cerr << "[ERROR] image read failed! image path: "
-                << cv_all_img_names[i] << endl;
-      exit(1);
-    }
-    // det
-    std::vector<std::vector<std::vector<int>>> boxes;
-    std::vector<double> det_times;
-    std::vector<double> rec_times;
-
-    det.Run(srcimg, boxes, &det_times);
-    if (FLAGS_visualize) {
-      std::string file_name = Utility::basename(cv_all_img_names[i]);
-      Utility::VisualizeBboxes(srcimg, boxes, FLAGS_output + "/" + file_name);
-    }
-    time_info_det[0] += det_times[0];
-    time_info_det[1] += det_times[1];
-    time_info_det[2] += det_times[2];
-
-    // rec
-    std::vector<cv::Mat> img_list;
-    for (int j = 0; j < boxes.size(); j++) {
-      cv::Mat crop_img;
-      crop_img = Utility::GetRotateCropImage(srcimg, boxes[j]);
-      if (cls != nullptr) {
-        crop_img = cls->Run(crop_img);
-      }
-      img_list.push_back(crop_img);
-    }
-    std::vector<std::string> rec_texts(img_list.size(), "");
-    std::vector<float> rec_text_scores(img_list.size(), 0);
-    rec.Run(img_list, rec_texts, rec_text_scores, &rec_times);
-    // output rec results
-    for (int i = 0; i < rec_texts.size(); i++) {
-      std::cout << i << "\t" << rec_texts[i] << "\t" << rec_text_scores[i]
-                << std::endl;
-    }
-    time_info_rec[0] += rec_times[0];
-    time_info_rec[1] += rec_times[1];
-    time_info_rec[2] += rec_times[2];
-  }
-
-  if (FLAGS_benchmark) {
-    AutoLogger autolog_det("ocr_det", FLAGS_use_gpu, FLAGS_use_tensorrt,
-                           FLAGS_enable_mkldnn, FLAGS_cpu_threads, 1, "dynamic",
-                           FLAGS_precision, time_info_det,
-                           cv_all_img_names.size());
-    AutoLogger autolog_rec("ocr_rec", FLAGS_use_gpu, FLAGS_use_tensorrt,
-                           FLAGS_enable_mkldnn, FLAGS_cpu_threads,
-                           FLAGS_rec_batch_num, "dynamic", FLAGS_precision,
-                           time_info_rec, cv_all_img_names.size());
-    autolog_det.report();
-    std::cout << endl;
-    autolog_rec.report();
-  }
-  return 0;
-}
-
-void check_params(char *mode) {
-  if (strcmp(mode, "det") == 0) {
+void check_params() {
+  if (FLAGS_det) {
     if (FLAGS_det_model_dir.empty() || FLAGS_image_dir.empty()) {
       std::cout << "Usage[det]: ./ppocr "
                    "--det_model_dir=/PATH/TO/DET_INFERENCE_MODEL/ "
@@ -290,7 +31,7 @@ void check_params(char *mode) {
       exit(1);
     }
   }
-  if (strcmp(mode, "rec") == 0) {
+  if (FLAGS_rec) {
     if (FLAGS_rec_model_dir.empty() || FLAGS_image_dir.empty()) {
       std::cout << "Usage[rec]: ./ppocr "
                    "--rec_model_dir=/PATH/TO/REC_INFERENCE_MODEL/ "
@@ -298,19 +39,10 @@ void check_params(char *mode) {
       exit(1);
     }
   }
-  if (strcmp(mode, "system") == 0) {
-    if ((FLAGS_det_model_dir.empty() || FLAGS_rec_model_dir.empty() ||
-         FLAGS_image_dir.empty()) ||
-        (FLAGS_use_angle_cls && FLAGS_cls_model_dir.empty())) {
-      std::cout << "Usage[system without angle cls]: ./ppocr "
-                   "--det_model_dir=/PATH/TO/DET_INFERENCE_MODEL/ "
-                << "--rec_model_dir=/PATH/TO/REC_INFERENCE_MODEL/ "
-                << "--image_dir=/PATH/TO/INPUT/IMAGE/" << std::endl;
-      std::cout << "Usage[system with angle cls]: ./ppocr "
-                   "--det_model_dir=/PATH/TO/DET_INFERENCE_MODEL/ "
-                << "--use_angle_cls=true "
-                << "--cls_model_dir=/PATH/TO/CLS_INFERENCE_MODEL/ "
-                << "--rec_model_dir=/PATH/TO/REC_INFERENCE_MODEL/ "
+  if (FLAGS_cls && FLAGS_use_angle_cls) {
+    if (FLAGS_cls_model_dir.empty() || FLAGS_image_dir.empty()) {
+      std::cout << "Usage[cls]: ./ppocr "
+                << "--cls_model_dir=/PATH/TO/REC_INFERENCE_MODEL/ "
                 << "--image_dir=/PATH/TO/INPUT/IMAGE/" << std::endl;
       exit(1);
     }
@@ -323,19 +55,11 @@ void check_params(char *mode) {
 }
 
 int main(int argc, char **argv) {
-  if (argc <= 1 ||
-      (strcmp(argv[1], "det") != 0 && strcmp(argv[1], "rec") != 0 &&
-       strcmp(argv[1], "system") != 0)) {
-    std::cout << "Please choose one mode of [det, rec, system] !" << std::endl;
-    return -1;
-  }
-  std::cout << "mode: " << argv[1] << endl;
-
   // Parsing command-line
   google::ParseCommandLineFlags(&argc, &argv, true);
-  check_params(argv[1]);
+  check_params();
 
-  if (!PathExists(FLAGS_image_dir)) {
+  if (!Utility::PathExists(FLAGS_image_dir)) {
     std::cerr << "[ERROR] image path not exist! image_dir: " << FLAGS_image_dir
               << endl;
     exit(1);
@@ -345,13 +69,37 @@ int main(int argc, char **argv) {
   cv::glob(FLAGS_image_dir, cv_all_img_names);
   std::cout << "total images num: " << cv_all_img_names.size() << endl;
 
-  if (strcmp(argv[1], "det") == 0) {
-    return main_det(cv_all_img_names);
+  PPOCR ocr = PPOCR();
+
+  std::vector<std::vector<OCRPredictResult>> ocr_results =
+      ocr.ocr(cv_all_img_names, FLAGS_det, FLAGS_rec, FLAGS_cls);
+
+  for (int i = 0; i < cv_all_img_names.size(); ++i) {
+    if (FLAGS_benchmark) {
+      cout << cv_all_img_names[i] << '\t';
+      for (int n = 0; n < ocr_results[i].size(); n++) {
+        for (int m = 0; m < ocr_results[i][n].box.size(); m++) {
+          cout << ocr_results[i][n].box[m][0] << ' '
+               << ocr_results[i][n].box[m][1] << ' ';
+        }
+      }
+      cout << endl;
+    } else {
+      cout << cv_all_img_names[i] << "\n";
+      Utility::print_result(ocr_results[i]);
+      if (FLAGS_visualize && FLAGS_det) {
+        cv::Mat srcimg = cv::imread(cv_all_img_names[i], cv::IMREAD_COLOR);
+        if (!srcimg.data) {
+          std::cerr << "[ERROR] image read failed! image path: "
+                    << cv_all_img_names[i] << endl;
+          exit(1);
+        }
+        std::string file_name = Utility::basename(cv_all_img_names[i]);
+
+        Utility::VisualizeBboxes(srcimg, ocr_results[i],
+                                 FLAGS_output + "/" + file_name);
       }
-  if (strcmp(argv[1], "rec") == 0) {
-    return main_rec(cv_all_img_names);
+      cout << "***************************" << endl;
     }
-  if (strcmp(argv[1], "system") == 0) {
-    return main_system(cv_all_img_names);
   }
 }

deploy/cpp_infer/src/ocr_cls.cpp

@@ -16,57 +16,84 @@
 
 namespace PaddleOCR {
 
-cv::Mat Classifier::Run(cv::Mat &img) {
-  cv::Mat src_img;
-  img.copyTo(src_img);
-  cv::Mat resize_img;
-
+void Classifier::Run(std::vector<cv::Mat> img_list,
+                     std::vector<int> &cls_labels,
+                     std::vector<float> &cls_scores,
+                     std::vector<double> &times) {
+  std::chrono::duration<float> preprocess_diff =
+      std::chrono::steady_clock::now() - std::chrono::steady_clock::now();
+  std::chrono::duration<float> inference_diff =
+      std::chrono::steady_clock::now() - std::chrono::steady_clock::now();
+  std::chrono::duration<float> postprocess_diff =
+      std::chrono::steady_clock::now() - std::chrono::steady_clock::now();
+
+  int img_num = img_list.size();
   std::vector<int> cls_image_shape = {3, 48, 192};
-  int index = 0;
-  float wh_ratio = float(img.cols) / float(img.rows);
-
-  this->resize_op_.Run(img, resize_img, this->use_tensorrt_, cls_image_shape);
+  for (int beg_img_no = 0; beg_img_no < img_num;
+       beg_img_no += this->cls_batch_num_) {
+    auto preprocess_start = std::chrono::steady_clock::now();
+    int end_img_no = min(img_num, beg_img_no + this->cls_batch_num_);
+    int batch_num = end_img_no - beg_img_no;
+    // preprocess
+    std::vector<cv::Mat> norm_img_batch;
+    for (int ino = beg_img_no; ino < end_img_no; ino++) {
+      cv::Mat srcimg;
+      img_list[ino].copyTo(srcimg);
+      cv::Mat resize_img;
+      this->resize_op_.Run(srcimg, resize_img, this->use_tensorrt_,
+                           cls_image_shape);
 
       this->normalize_op_.Run(&resize_img, this->mean_, this->scale_,
                               this->is_scale_);
-
-  std::vector<float> input(1 * 3 * resize_img.rows * resize_img.cols, 0.0f);
-
-  this->permute_op_.Run(&resize_img, input.data());
-
-  // Inference.
+      norm_img_batch.push_back(resize_img);
+    }
+    std::vector<float> input(batch_num * cls_image_shape[0] *
+                                 cls_image_shape[1] * cls_image_shape[2],
+                             0.0f);
+    this->permute_op_.Run(norm_img_batch, input.data());
+    auto preprocess_end = std::chrono::steady_clock::now();
+    preprocess_diff += preprocess_end - preprocess_start;
+
+    // inference.
     auto input_names = this->predictor_->GetInputNames();
     auto input_t = this->predictor_->GetInputHandle(input_names[0]);
-  input_t->Reshape({1, 3, resize_img.rows, resize_img.cols});
+    input_t->Reshape({batch_num, cls_image_shape[0], cls_image_shape[1],
+                      cls_image_shape[2]});
+    auto inference_start = std::chrono::steady_clock::now();
     input_t->CopyFromCpu(input.data());
     this->predictor_->Run();
 
-  std::vector<float> softmax_out;
-  std::vector<int64_t> label_out;
+    std::vector<float> predict_batch;
     auto output_names = this->predictor_->GetOutputNames();
-  auto softmax_out_t = this->predictor_->GetOutputHandle(output_names[0]);
-  auto softmax_shape_out = softmax_out_t->shape();
+    auto output_t = this->predictor_->GetOutputHandle(output_names[0]);
+    auto predict_shape = output_t->shape();
 
-  int softmax_out_num =
-      std::accumulate(softmax_shape_out.begin(), softmax_shape_out.end(), 1,
+    int out_num = std::accumulate(predict_shape.begin(), predict_shape.end(), 1,
                                   std::multiplies<int>());
-
-  softmax_out.resize(softmax_out_num);
-
-  softmax_out_t->CopyToCpu(softmax_out.data());
-
-  float score = 0;
-  int label = 0;
-  for (int i = 0; i < softmax_out_num; i++) {
-    if (softmax_out[i] > score) {
-      score = softmax_out[i];
-      label = i;
-    }
+    predict_batch.resize(out_num);
+
+    output_t->CopyToCpu(predict_batch.data());
+    auto inference_end = std::chrono::steady_clock::now();
+    inference_diff += inference_end - inference_start;
+
+    // postprocess
+    auto postprocess_start = std::chrono::steady_clock::now();
+    for (int batch_idx = 0; batch_idx < predict_shape[0]; batch_idx++) {
+      int label = int(
+          Utility::argmax(&predict_batch[batch_idx * predict_shape[1]],
+                          &predict_batch[(batch_idx + 1) * predict_shape[1]]));
+      float score = float(*std::max_element(
+          &predict_batch[batch_idx * predict_shape[1]],
+          &predict_batch[(batch_idx + 1) * predict_shape[1]]));
+      cls_labels[beg_img_no + batch_idx] = label;
+      cls_scores[beg_img_no + batch_idx] = score;
     }
-  if (label % 2 == 1 && score > this->cls_thresh) {
-    cv::rotate(src_img, src_img, 1);
+    auto postprocess_end = std::chrono::steady_clock::now();
+    postprocess_diff += postprocess_end - postprocess_start;
   }
-  return src_img;
+  times.push_back(double(preprocess_diff.count() * 1000));
+  times.push_back(double(inference_diff.count() * 1000));
+  times.push_back(double(postprocess_diff.count() * 1000));
 }
 
 void Classifier::LoadModel(const std::string &model_dir) {
@@ -84,10 +111,7 @@ void Classifier::LoadModel(const std::string &model_dir) {
       if (this->precision_ == "int8") {
         precision = paddle_infer::Config::Precision::kInt8;
       }
-      config.EnableTensorRtEngine(
-          1 << 20, 10, 3,
-          precision,
-          false, false);
+      config.EnableTensorRtEngine(1 << 20, 10, 3, precision, false, false);
     }
   } else {
     config.DisableGpu();

deploy/cpp_infer/src/ocr_det.cpp

@@ -94,7 +94,7 @@ void DBDetector::LoadModel(const std::string &model_dir) {
 
 void DBDetector::Run(cv::Mat &img,
                      std::vector<std::vector<std::vector<int>>> &boxes,
-                     std::vector<double> *times) {
+                     std::vector<double> &times) {
   float ratio_h{};
   float ratio_w{};
 
@@ -165,16 +165,15 @@ void DBDetector::Run(cv::Mat &img,
 
   boxes = post_processor_.FilterTagDetRes(boxes, ratio_h, ratio_w, srcimg);
   auto postprocess_end = std::chrono::steady_clock::now();
-  std::cout << "Detected boxes num: " << boxes.size() << endl;
 
   std::chrono::duration<float> preprocess_diff =
       preprocess_end - preprocess_start;
-  times->push_back(double(preprocess_diff.count() * 1000));
+  times.push_back(double(preprocess_diff.count() * 1000));
   std::chrono::duration<float> inference_diff = inference_end - inference_start;
-  times->push_back(double(inference_diff.count() * 1000));
+  times.push_back(double(inference_diff.count() * 1000));
   std::chrono::duration<float> postprocess_diff =
       postprocess_end - postprocess_start;
-  times->push_back(double(postprocess_diff.count() * 1000));
+  times.push_back(double(postprocess_diff.count() * 1000));
 }
 
 } // namespace PaddleOCR

deploy/cpp_infer/src/ocr_rec.cpp

@@ -19,7 +19,7 @@ namespace PaddleOCR {
 void CRNNRecognizer::Run(std::vector<cv::Mat> img_list,
                          std::vector<std::string> &rec_texts,
                          std::vector<float> &rec_text_scores,
-                         std::vector<double> *times) {
+                         std::vector<double> &times) {
   std::chrono::duration<float> preprocess_diff =
       std::chrono::steady_clock::now() - std::chrono::steady_clock::now();
   std::chrono::duration<float> inference_diff =
@@ -38,6 +38,7 @@ void CRNNRecognizer::Run(std::vector<cv::Mat> img_list,
        beg_img_no += this->rec_batch_num_) {
     auto preprocess_start = std::chrono::steady_clock::now();
     int end_img_no = min(img_num, beg_img_no + this->rec_batch_num_);
+    int batch_num = end_img_no - beg_img_no;
     float max_wh_ratio = 0;
     for (int ino = beg_img_no; ino < end_img_no; ino++) {
       int h = img_list[indices[ino]].rows;
@@ -45,6 +46,7 @@ void CRNNRecognizer::Run(std::vector<cv::Mat> img_list,
       float wh_ratio = w * 1.0 / h;
       max_wh_ratio = max(max_wh_ratio, wh_ratio);
     }
+
     int batch_width = 0;
     std::vector<cv::Mat> norm_img_batch;
     for (int ino = beg_img_no; ino < end_img_no; ino++) {
@@ -59,15 +61,14 @@ void CRNNRecognizer::Run(std::vector<cv::Mat> img_list,
       batch_width = max(resize_img.cols, batch_width);
     }
 
-    std::vector<float> input(this->rec_batch_num_ * 3 * 32 * batch_width, 0.0f);
+    std::vector<float> input(batch_num * 3 * 32 * batch_width, 0.0f);
     this->permute_op_.Run(norm_img_batch, input.data());
     auto preprocess_end = std::chrono::steady_clock::now();
     preprocess_diff += preprocess_end - preprocess_start;
-
     // Inference.
     auto input_names = this->predictor_->GetInputNames();
     auto input_t = this->predictor_->GetInputHandle(input_names[0]);
-    input_t->Reshape({this->rec_batch_num_, 3, 32, batch_width});
+    input_t->Reshape({batch_num, 3, 32, batch_width});
     auto inference_start = std::chrono::steady_clock::now();
     input_t->CopyFromCpu(input.data());
     this->predictor_->Run();
@@ -84,7 +85,6 @@ void CRNNRecognizer::Run(std::vector<cv::Mat> img_list,
     output_t->CopyToCpu(predict_batch.data());
     auto inference_end = std::chrono::steady_clock::now();
     inference_diff += inference_end - inference_start;
-
     // ctc decode
     auto postprocess_start = std::chrono::steady_clock::now();
     for (int m = 0; m < predict_shape[0]; m++) {
@@ -120,9 +120,9 @@ void CRNNRecognizer::Run(std::vector<cv::Mat> img_list,
     auto postprocess_end = std::chrono::steady_clock::now();
     postprocess_diff += postprocess_end - postprocess_start;
   }
-  times->push_back(double(preprocess_diff.count() * 1000));
-  times->push_back(double(inference_diff.count() * 1000));
-  times->push_back(double(postprocess_diff.count() * 1000));
+  times.push_back(double(preprocess_diff.count() * 1000));
+  times.push_back(double(inference_diff.count() * 1000));
+  times.push_back(double(postprocess_diff.count() * 1000));
 }
 
 void CRNNRecognizer::LoadModel(const std::string &model_dir) {

deploy/cpp_infer/src/paddleocr.cpp

+// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include <include/args.h>
+#include <include/paddleocr.h>
+
+#include "auto_log/autolog.h"
+#include <numeric>
+namespace PaddleOCR {
+
+PPOCR::PPOCR() {
+  if (FLAGS_det) {
+    this->detector_ = new DBDetector(
+        FLAGS_det_model_dir, FLAGS_use_gpu, FLAGS_gpu_id, FLAGS_gpu_mem,
+        FLAGS_cpu_threads, FLAGS_enable_mkldnn, FLAGS_max_side_len,
+        FLAGS_det_db_thresh, FLAGS_det_db_box_thresh, FLAGS_det_db_unclip_ratio,
+        FLAGS_det_db_score_mode, FLAGS_use_dilation, FLAGS_use_tensorrt,
+        FLAGS_precision);
+  }
+
+  if (FLAGS_cls && FLAGS_use_angle_cls) {
+    this->classifier_ = new Classifier(
+        FLAGS_cls_model_dir, FLAGS_use_gpu, FLAGS_gpu_id, FLAGS_gpu_mem,
+        FLAGS_cpu_threads, FLAGS_enable_mkldnn, FLAGS_cls_thresh,
+        FLAGS_use_tensorrt, FLAGS_precision, FLAGS_cls_batch_num);
+  }
+  if (FLAGS_rec) {
+    this->recognizer_ = new CRNNRecognizer(
+        FLAGS_rec_model_dir, FLAGS_use_gpu, FLAGS_gpu_id, FLAGS_gpu_mem,
+        FLAGS_cpu_threads, FLAGS_enable_mkldnn, FLAGS_rec_char_dict_path,
+        FLAGS_use_tensorrt, FLAGS_precision, FLAGS_rec_batch_num);
+  }
+};
+
+void PPOCR::det(cv::Mat img, std::vector<OCRPredictResult> &ocr_results,
+                std::vector<double> &times) {
+  std::vector<std::vector<std::vector<int>>> boxes;
+  std::vector<double> det_times;
+
+  this->detector_->Run(img, boxes, det_times);
+
+  for (int i = 0; i < boxes.size(); i++) {
+    OCRPredictResult res;
+    res.box = boxes[i];
+    ocr_results.push_back(res);
+  }
+
+  times[0] += det_times[0];
+  times[1] += det_times[1];
+  times[2] += det_times[2];
+}
+
+void PPOCR::rec(std::vector<cv::Mat> img_list,
+                std::vector<OCRPredictResult> &ocr_results,
+                std::vector<double> &times) {
+  std::vector<std::string> rec_texts(img_list.size(), "");
+  std::vector<float> rec_text_scores(img_list.size(), 0);
+  std::vector<double> rec_times;
+  this->recognizer_->Run(img_list, rec_texts, rec_text_scores, rec_times);
+  // output rec results
+  for (int i = 0; i < rec_texts.size(); i++) {
+    ocr_results[i].text = rec_texts[i];
+    ocr_results[i].score = rec_text_scores[i];
+  }
+  times[0] += rec_times[0];
+  times[1] += rec_times[1];
+  times[2] += rec_times[2];
+}
+
+void PPOCR::cls(std::vector<cv::Mat> img_list,
+                std::vector<OCRPredictResult> &ocr_results,
+                std::vector<double> &times) {
+  std::vector<int> cls_labels(img_list.size(), 0);
+  std::vector<float> cls_scores(img_list.size(), 0);
+  std::vector<double> cls_times;
+  this->classifier_->Run(img_list, cls_labels, cls_scores, cls_times);
+  // output cls results
+  for (int i = 0; i < cls_labels.size(); i++) {
+    ocr_results[i].cls_label = cls_labels[i];
+    ocr_results[i].cls_score = cls_scores[i];
+  }
+  times[0] += cls_times[0];
+  times[1] += cls_times[1];
+  times[2] += cls_times[2];
+}
+
+std::vector<std::vector<OCRPredictResult>>
+PPOCR::ocr(std::vector<cv::String> cv_all_img_names, bool det, bool rec,
+           bool cls) {
+  std::vector<double> time_info_det = {0, 0, 0};
+  std::vector<double> time_info_rec = {0, 0, 0};
+  std::vector<double> time_info_cls = {0, 0, 0};
+  std::vector<std::vector<OCRPredictResult>> ocr_results;
+
+  if (!det) {
+    std::vector<OCRPredictResult> ocr_result;
+    // read image
+    std::vector<cv::Mat> img_list;
+    for (int i = 0; i < cv_all_img_names.size(); ++i) {
+      cv::Mat srcimg = cv::imread(cv_all_img_names[i], cv::IMREAD_COLOR);
+      if (!srcimg.data) {
+        std::cerr << "[ERROR] image read failed! image path: "
+                  << cv_all_img_names[i] << endl;
+        exit(1);
+      }
+      img_list.push_back(srcimg);
+      OCRPredictResult res;
+      ocr_result.push_back(res);
+    }
+    if (cls && this->classifier_ != nullptr) {
+      this->cls(img_list, ocr_result, time_info_cls);
+      for (int i = 0; i < img_list.size(); i++) {
+        if (ocr_result[i].cls_label % 2 == 1 &&
+            ocr_result[i].cls_score > this->classifier_->cls_thresh) {
+          cv::rotate(img_list[i], img_list[i], 1);
+        }
+      }
+    }
+    if (rec) {
+      this->rec(img_list, ocr_result, time_info_rec);
+    }
+    for (int i = 0; i < cv_all_img_names.size(); ++i) {
+      std::vector<OCRPredictResult> ocr_result_tmp;
+      ocr_result_tmp.push_back(ocr_result[i]);
+      ocr_results.push_back(ocr_result_tmp);
+    }
+  } else {
+    if (!Utility::PathExists(FLAGS_output) && FLAGS_det) {
+      Utility::CreateDir(FLAGS_output);
+    }
+
+    for (int i = 0; i < cv_all_img_names.size(); ++i) {
+      std::vector<OCRPredictResult> ocr_result;
+      if (!FLAGS_benchmark) {
+        cout << "predict img: " << cv_all_img_names[i] << endl;
+      }
+
+      cv::Mat srcimg = cv::imread(cv_all_img_names[i], cv::IMREAD_COLOR);
+      if (!srcimg.data) {
+        std::cerr << "[ERROR] image read failed! image path: "
+                  << cv_all_img_names[i] << endl;
+        exit(1);
+      }
+      // det
+      this->det(srcimg, ocr_result, time_info_det);
+      // crop image
+      std::vector<cv::Mat> img_list;
+      for (int j = 0; j < ocr_result.size(); j++) {
+        cv::Mat crop_img;
+        crop_img = Utility::GetRotateCropImage(srcimg, ocr_result[j].box);
+        img_list.push_back(crop_img);
+      }
+
+      // cls
+      if (cls && this->classifier_ != nullptr) {
+        this->cls(img_list, ocr_result, time_info_cls);
+        for (int i = 0; i < img_list.size(); i++) {
+          if (ocr_result[i].cls_label % 2 == 1 &&
+              ocr_result[i].cls_score > this->classifier_->cls_thresh) {
+            cv::rotate(img_list[i], img_list[i], 1);
+          }
+        }
+      }
+      // rec
+      if (rec) {
+        this->rec(img_list, ocr_result, time_info_rec);
+      }
+      ocr_results.push_back(ocr_result);
+    }
+  }
+  if (FLAGS_benchmark) {
+    this->log(time_info_det, time_info_rec, time_info_cls,
+              cv_all_img_names.size());
+  }
+  return ocr_results;
+} // namespace PaddleOCR
+
+void PPOCR::log(std::vector<double> &det_times, std::vector<double> &rec_times,
+                std::vector<double> &cls_times, int img_num) {
+  if (det_times[0] + det_times[1] + det_times[2] > 0) {
+    AutoLogger autolog_det("ocr_det", FLAGS_use_gpu, FLAGS_use_tensorrt,
+                           FLAGS_enable_mkldnn, FLAGS_cpu_threads, 1, "dynamic",
+                           FLAGS_precision, det_times, img_num);
+    autolog_det.report();
+  }
+  if (rec_times[0] + rec_times[1] + rec_times[2] > 0) {
+    AutoLogger autolog_rec("ocr_rec", FLAGS_use_gpu, FLAGS_use_tensorrt,
+                           FLAGS_enable_mkldnn, FLAGS_cpu_threads,
+                           FLAGS_rec_batch_num, "dynamic", FLAGS_precision,
+                           rec_times, img_num);
+    autolog_rec.report();
+  }
+  if (cls_times[0] + cls_times[1] + cls_times[2] > 0) {
+    AutoLogger autolog_cls("ocr_cls", FLAGS_use_gpu, FLAGS_use_tensorrt,
+                           FLAGS_enable_mkldnn, FLAGS_cpu_threads,
+                           FLAGS_cls_batch_num, "dynamic", FLAGS_precision,
+                           cls_times, img_num);
+    autolog_cls.report();
+  }
+}
+PPOCR::~PPOCR() {
+  if (this->detector_ != nullptr) {
+    delete this->detector_;
+  }
+  if (this->classifier_ != nullptr) {
+    delete this->classifier_;
+  }
+  if (this->recognizer_ != nullptr) {
+    delete this->recognizer_;
+  }
+};
+
+} // namespace PaddleOCR