Fixed merge conflict in README.md

112ad00d · Manan Goel · b7aa10e5 · 95ac9c0e · 112ad00d · 112ad00d
Commit 112ad00d authored Apr 23, 2022 by Manan Goel
20 changed files
--- a/deploy/cpp_infer/src/utility.cpp
+++ b/deploy/cpp_infer/src/utility.cpp
@@ -16,10 +16,15 @@
 #include <include/utility.h>
 #include <iostream>
 #include <ostream>
-#include <sys/stat.h>
-#include <sys/types.h>
 #include <vector>
+#ifdef _WIN32
+#include <direct.h>
+#else
+#include <sys/stat.h>
+#endif
 namespace PaddleOCR {
 std::vector<std::string> Utility::ReadDict(const std::string &path) {
@@ -38,16 +43,16 @@ std::vector<std::string> Utility::ReadDict(const std::string &path) {
  return m_vec;
 }
-void Utility::VisualizeBboxes(
+void Utility::VisualizeBboxes(const cv::Mat &srcimg,
-    const cv::Mat &srcimg,
+                              const std::vector<OCRPredictResult> &ocr_result,
-    const std::vector<std::vector<std::vector<int>>> &boxes,
                              const std::string &save_path) {
  cv::Mat img_vis;
  srcimg.copyTo(img_vis);
-  for (int n = 0; n < boxes.size(); n++) {
+  for (int n = 0; n < ocr_result.size(); n++) {
    cv::Point rook_points[4];
-    for (int m = 0; m < boxes[n].size(); m++) {
+    for (int m = 0; m < ocr_result[n].box.size(); m++) {
-      rook_points[m] = cv::Point(int(boxes[n][m][0]), int(boxes[n][m][1]));
+      rook_points[m] =
+          cv::Point(int(ocr_result[n].box[m][0]), int(ocr_result[n].box[m][1]));
    }
    const cv::Point *ppt[1] = {rook_points};
@@ -196,4 +201,51 @@ std::string Utility::basename(const std::string &filename) {
  return filename.substr(index + 1, len - index);
 }
+bool Utility::PathExists(const std::string &path) {
+#ifdef _WIN32
+  struct _stat buffer;
+  return (_stat(path.c_str(), &buffer) == 0);
+#else
+  struct stat buffer;
+  return (stat(path.c_str(), &buffer) == 0);
+#endif // !_WIN32
+}
+void Utility::CreateDir(const std::string &path) {
+#ifdef _WIN32
+  _mkdir(path.c_str());
+#else
+  mkdir(path.c_str(), 0777);
+#endif // !_WIN32
+}
+void Utility::print_result(const std::vector<OCRPredictResult> &ocr_result) {
+  for (int i = 0; i < ocr_result.size(); i++) {
+    std::cout << i << "\t";
+    // det
+    std::vector<std::vector<int>> boxes = ocr_result[i].box;
+    if (boxes.size() > 0) {
+      std::cout << "det boxes: [";
+      for (int n = 0; n < boxes.size(); n++) {
+        std::cout << '[' << boxes[n][0] << ',' << boxes[n][1] << "]";
+        if (n != boxes.size() - 1) {
+          std::cout << ',';
+        }
+      }
+      std::cout << "] ";
+    }
+    // rec
+    if (ocr_result[i].score != -1.0) {
+      std::cout << "rec text: " << ocr_result[i].text
+                << " rec score: " << ocr_result[i].score << " ";
+    }
+    // cls
+    if (ocr_result[i].cls_label != -1) {
+      std::cout << "cls label: " << ocr_result[i].cls_label
+                << " cls score: " << ocr_result[i].cls_score;
+    }
+    std::cout << std::endl;
+  }
+}
 } // namespace PaddleOCR
\ No newline at end of file
--- a/deploy/lite/readme.md
+++ b/deploy/lite/readme.md
--- a/deploy/lite/readme_ch.md
+++ b/deploy/lite/readme_ch.md
+- [端侧部署](#端侧部署)
+  - [1. 准备环境](#1-准备环境)
+    - [运行准备](#运行准备)
+    - [1.1 准备交叉编译环境](#11-准备交叉编译环境)
+    - [1.2 准备预测库](#12-准备预测库)
+  - [2 开始运行](#2-开始运行)
+    - [2.1 模型优化](#21-模型优化)
+    - [2.2 与手机联调](#22-与手机联调)
+      - [注意：](#注意)
+  - [FAQ](#faq)
+# 端侧部署
+本教程将介绍基于[Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite) 在移动端部署PaddleOCR超轻量中文检测、识别模型的详细步骤。
+Paddle Lite是飞桨轻量化推理引擎，为手机、IOT端提供高效推理能力，并广泛整合跨平台硬件，为端侧部署及应用落地问题提供轻量化的部署方案。
+## 1. 准备环境
+### 运行准备
+- 电脑（编译Paddle Lite）
+- 安卓手机（armv7或armv8）
+### 1.1 准备交叉编译环境
+交叉编译环境用于编译 Paddle Lite 和 PaddleOCR 的C++ demo。
+支持多种开发环境，不同开发环境的编译流程请参考对应文档。
+1. [Docker](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#docker)
+2. [Linux](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#linux)
+3. [MAC OS](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#mac-os)
+### 1.2 准备预测库
+预测库有两种获取方式：
+- 1. 直接下载，预测库下载链接如下：
+      | 平台 | 预测库下载链接 |
+      |---|---|
+      |Android|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.android.armv7.gcc.c++_shared.with_extra.with_cv.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.android.armv8.gcc.c++_shared.with_extra.with_cv.tar.gz)|
+      |IOS|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.ios.armv7.with_cv.with_extra.with_log.tiny_publish.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.ios.armv8.with_cv.with_extra.with_log.tiny_publish.tar.gz)|
+      注：1. 上述预测库为PaddleLite 2.10分支编译得到，有关PaddleLite 2.10 详细信息可参考 [链接](https://github.com/PaddlePaddle/Paddle-Lite/releases/tag/v2.10) 。
+- 2. [推荐]编译Paddle-Lite得到预测库，Paddle-Lite的编译方式如下：
+```
+git clone https://github.com/PaddlePaddle/Paddle-Lite.git
+cd Paddle-Lite
+# 切换到Paddle-Lite release/v2.10 稳定分支
+git checkout release/v2.10
+./lite/tools/build_android.sh  --arch=armv8  --with_cv=ON --with_extra=ON
+```
+注意：编译Paddle-Lite获得预测库时，需要打开`--with_cv=ON --with_extra=ON`两个选项，`--arch`表示`arm`版本，这里指定为armv8，
+更多编译命令
+介绍请参考 [链接](https://paddle-lite.readthedocs.io/zh/release-v2.10_a/source_compile/linux_x86_compile_android.html) 。
+直接下载预测库并解压后，可以得到`inference_lite_lib.android.armv8/`文件夹，通过编译Paddle-Lite得到的预测库位于
+`Paddle-Lite/build.lite.android.armv8.gcc/inference_lite_lib.android.armv8/`文件夹下。
+预测库的文件目录如下：
+```
+inference_lite_lib.android.armv8/
+|-- cxx                                        C++ 预测库和头文件
+|   |-- include                                C++ 头文件
+|   |   |-- paddle_api.h
+|   |   |-- paddle_image_preprocess.h
+|   |   |-- paddle_lite_factory_helper.h
+|   |   |-- paddle_place.h
+|   |   |-- paddle_use_kernels.h
+|   |   |-- paddle_use_ops.h
+|   |   `-- paddle_use_passes.h
+|   `-- lib                                           C++预测库
+|       |-- libpaddle_api_light_bundled.a             C++静态库
+|       `-- libpaddle_light_api_shared.so             C++动态库
+|-- java                                     Java预测库
+|   |-- jar
+|   |   `-- PaddlePredictor.jar
+|   |-- so
+|   |   `-- libpaddle_lite_jni.so
+|   `-- src
+|-- demo                                     C++和Java示例代码
+|   |-- cxx                                  C++  预测库demo
+|   `-- java                                 Java 预测库demo
+```
+## 2 开始运行
+### 2.1 模型优化
+Paddle-Lite 提供了多种策略来自动优化原始的模型，其中包括量化、子图融合、混合调度、Kernel优选等方法，使用Paddle-lite的opt工具可以自动
+对inference模型进行优化，优化后的模型更轻量，模型运行速度更快。
+如果已经准备好了 `.nb` 结尾的模型文件，可以跳过此步骤。
+下述表格中也提供了一系列中文移动端模型：
+|模型版本|模型简介|模型大小|检测模型|文本方向分类模型|识别模型|Paddle-Lite版本|
+|---|---|---|---|---|---|---|
+|PP-OCRv2|蒸馏版超轻量中文OCR移动端模型|11M|[下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_PP-OCRv2_det_infer_opt.nb)|[下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_ppocr_mobile_v2.0_cls_infer_opt.nb)|[下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_PP-OCRv2_rec_infer_opt.nb)|v2.10|
+|PP-OCRv2(slim)|蒸馏版超轻量中文OCR移动端模型|4.6M|[下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_PP-OCRv2_det_slim_opt.nb)|[下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_ppocr_mobile_v2.0_cls_slim_opt.nb)|[下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_PP-OCRv2_rec_slim_opt.nb)|v2.10|
+如果直接使用上述表格中的模型进行部署，可略过下述步骤，直接阅读 [2.2节](#2.2与手机联调)。
+如果要部署的模型不在上述表格中，则需要按照如下步骤获得优化后的模型。
+模型优化需要Paddle-Lite的opt可执行文件，可以通过编译Paddle-Lite源码获得，编译步骤如下：
+```
+# 如果准备环境时已经clone了Paddle-Lite，则不用重新clone Paddle-Lite
+git clone https://github.com/PaddlePaddle/Paddle-Lite.git
+cd Paddle-Lite
+git checkout release/v2.10
+# 启动编译
+./lite/tools/build.sh build_optimize_tool
+```
+编译完成后，opt文件位于`build.opt/lite/api/`下，可通过如下方式查看opt的运行选项和使用方式；
+```
+cd build.opt/lite/api/
+./opt
+```
+|选项|说明|
+|---|---|
+|--model_dir|待优化的PaddlePaddle模型（非combined形式）的路径|
+|--model_file|待优化的PaddlePaddle模型（combined形式）的网络结构文件路径|
+|--param_file|待优化的PaddlePaddle模型（combined形式）的权重文件路径|
+|--optimize_out_type|输出模型类型，目前支持两种类型：protobuf和naive_buffer，其中naive_buffer是一种更轻量级的序列化/反序列化实现。若您需要在mobile端执行模型预测，请将此选项设置为naive_buffer。默认为protobuf|
+|--optimize_out|优化模型的输出路径|
+|--valid_targets|指定模型可执行的backend，默认为arm。目前可支持x86、arm、opencl、npu、xpu，可以同时指定多个backend(以空格分隔)，Model Optimize Tool将会自动选择最佳方式。如果需要支持华为NPU（Kirin 810/990 Soc搭载的达芬奇架构NPU），应当设置为npu, arm|
+|--record_tailoring_info|当使用 根据模型裁剪库文件 功能时，则设置该选项为true，以记录优化后模型含有的kernel和OP信息，默认为false|
+`--model_dir`适用于待优化的模型是非combined方式，PaddleOCR的inference模型是combined方式，即模型结构和模型参数使用单独一个文件存储。
+下面以PaddleOCR的超轻量中文模型为例，介绍使用编译好的opt文件完成inference模型到Paddle-Lite优化模型的转换。
+```
+# 【推荐】 下载 PP-OCRv2版本的中英文 inference模型
+wget  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_slim_quant_infer.tar && tar xf  ch_PP-OCRv2_det_slim_quant_infer.tar
+wget  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_slim_quant_infer.tar && tar xf  ch_PP-OCRv2_rec_slim_quant_infer.tar
+wget  https://paddleocr.bj.bcebos.com/dygraph_v2.0/slim/ch_ppocr_mobile_v2.0_cls_slim_infer.tar && tar xf  ch_ppocr_mobile_v2.0_cls_slim_infer.tar
+# 转换检测模型
+./opt --model_file=./ch_PP-OCRv2_det_slim_quant_infer/inference.pdmodel  --param_file=./ch_PP-OCRv2_det_slim_quant_infer/inference.pdiparams  --optimize_out=./ch_PP-OCRv2_det_slim_opt --valid_targets=arm  --optimize_out_type=naive_buffer
+# 转换识别模型
+./opt --model_file=./ch_PP-OCRv2_rec_slim_quant_infer/inference.pdmodel  --param_file=./ch_PP-OCRv2_rec_slim_quant_infer/inference.pdiparams  --optimize_out=./ch_PP-OCRv2_rec_slim_opt --valid_targets=arm  --optimize_out_type=naive_buffer
+# 转换方向分类器模型
+./opt --model_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdmodel  --param_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdiparams  --optimize_out=./ch_ppocr_mobile_v2.0_cls_slim_opt --valid_targets=arm  --optimize_out_type=naive_buffer
+```
+转换成功后，inference模型目录下会多出`.nb`结尾的文件，即是转换成功的模型文件。
+注意：使用paddle-lite部署时，需要使用opt工具优化后的模型。 opt 工具的输入模型是paddle保存的inference模型
+<a name="2.2与手机联调"></a>
+### 2.2 与手机联调
+首先需要进行一些准备工作。
+ 1. 准备一台arm8的安卓手机，如果编译的预测库和opt文件是armv7，则需要arm7的手机，并修改Makefile中`ARM_ABI = arm7`。
+ 2. 打开手机的USB调试选项，选择文件传输模式，连接电脑。
+ 3. 电脑上安装adb工具，用于调试。 adb安装方式如下：
+    3.1. MAC电脑安装ADB:
+    ```
+    brew cask install android-platform-tools
+    ```
+    3.2. Linux安装ADB
+    ```
+    sudo apt update
+    sudo apt install -y wget adb
+    ```
+    3.3. Window安装ADB
+    win上安装需要去谷歌的安卓平台下载adb软件包进行安装：[链接](https://developer.android.com/studio)
+    打开终端，手机连接电脑，在终端中输入
+    ```
+    adb devices
+    ```
+    如果有device输出，则表示安装成功。
+    ```
+       List of devices attached
+       744be294    device
+    ```
+ 4. 准备优化后的模型、预测库文件、测试图像和使用的字典文件。
+ ```
+ git clone https://github.com/PaddlePaddle/PaddleOCR.git
+ cd PaddleOCR/deploy/lite/
+ # 运行prepare.sh，准备预测库文件、测试图像和使用的字典文件，并放置在预测库中的demo/cxx/ocr文件夹下
+ sh prepare.sh /{lite prediction library path}/inference_lite_lib.android.armv8
+ # 进入OCR demo的工作目录
+ cd /{lite prediction library path}/inference_lite_lib.android.armv8/
+ cd demo/cxx/ocr/
+ # 将C++预测动态库so文件复制到debug文件夹中
+ cp ../../../cxx/lib/libpaddle_light_api_shared.so ./debug/
+ ```
+ 准备测试图像，以`PaddleOCR/doc/imgs/11.jpg`为例，将测试的图像复制到`demo/cxx/ocr/debug/`文件夹下。
+ 准备lite opt工具优化后的模型文件，比如使用`ch_PP-OCRv2_det_slim_opt.ch_PP-OCRv2_rec_slim_rec.nb, ch_ppocr_mobile_v2.0_cls_slim_opt.nb`，模型文件放置在`demo/cxx/ocr/debug/`文件夹下。
+ 执行完成后，ocr文件夹下将有如下文件格式：
+```
+demo/cxx/ocr/
+|-- debug/  
+|   |--ch_PP-OCRv2_det_slim_opt.nb           优化后的检测模型文件
+|   |--ch_PP-OCRv2_rec_slim_opt.nb           优化后的识别模型文件
+|   |--ch_ppocr_mobile_v2.0_cls_slim_opt.nb           优化后的文字方向分类器模型文件
+|   |--11.jpg                           待测试图像
+|   |--ppocr_keys_v1.txt                中文字典文件
+|   |--libpaddle_light_api_shared.so    C++预测库文件
+|   |--config.txt                       超参数配置
+|-- config.txt                  超参数配置
+|-- cls_process.cc              方向分类器的预处理和后处理文件
+|-- cls_process.h
+|-- crnn_process.cc             识别模型CRNN的预处理和后处理文件
+|-- crnn_process.h
+|-- db_post_process.cc          检测模型DB的后处理文件
+|-- db_post_process.h
+|-- Makefile                    编译文件
+|-- ocr_db_crnn.cc              C++预测源文件
+```
+#### 注意：
+1. ppocr_keys_v1.txt是中文字典文件，如果使用的 nb 模型是英文数字或其他语言的模型，需要更换为对应语言的字典。
+PaddleOCR 在ppocr/utils/下存放了多种字典，包括：
+```
+dict/french_dict.txt     # 法语字典
+dict/german_dict.txt     # 德语字典
+ic15_dict.txt       # 英文字典
+dict/japan_dict.txt      # 日语字典
+dict/korean_dict.txt     # 韩语字典
+ppocr_keys_v1.txt   # 中文字典
+...
+```
+2.  `config.txt` 包含了检测器、分类器的超参数，如下：
+```
+max_side_len  960         # 输入图像长宽大于960时，等比例缩放图像，使得图像最长边为960
+det_db_thresh  0.3        # 用于过滤DB预测的二值化图像，设置为0.-0.3对结果影响不明显
+det_db_box_thresh  0.5    # DB后处理过滤box的阈值，如果检测存在漏框情况，可酌情减小
+det_db_unclip_ratio  1.6  # 表示文本框的紧致程度，越小则文本框更靠近文本
+use_direction_classify  0  # 是否使用方向分类器，0表示不使用，1表示使用
+```
+ 5. 启动调试
+ 上述步骤完成后就可以使用adb将文件push到手机上运行，步骤如下：
+ ```
+ # 执行编译，得到可执行文件ocr_db_crnn, 第一次执行此命令会下载opencv等依赖库，下载完成后，需要再执行一次
+ make -j
+ # 将编译的可执行文件移动到debug文件夹中
+ mv ocr_db_crnn ./debug/
+ # 将debug文件夹push到手机上
+ adb push debug /data/local/tmp/
+ adb shell
+ cd /data/local/tmp/debug
+ export LD_LIBRARY_PATH=${PWD}:$LD_LIBRARY_PATH
+ # 开始使用，ocr_db_crnn可执行文件的使用方式为:
+ # ./ocr_db_crnn  检测模型文件 方向分类器模型文件  识别模型文件  测试图像路径  字典文件路径
+ ./ocr_db_crnn ch_PP-OCRv2_det_slim_opt.nb  ch_PP-OCRv2_rec_slim_opt.nb  ch_ppocr_mobile_v2.0_cls_slim_opt.nb  ./11.jpg  ppocr_keys_v1.txt
+ ```
+ 如果对代码做了修改，则需要重新编译并push到手机上。
+ 运行效果如下：
+<div align="center">
+    <img src="imgs/lite_demo.png" width="600">
+</div>
+## FAQ
+Q1：如果想更换模型怎么办，需要重新按照流程走一遍吗？
+A1：如果已经走通了上述步骤，更换模型只需要替换 .nb 模型文件即可，同时要注意更新字典
+Q2：换一个图测试怎么做？
+A2：替换debug下的.jpg测试图像为你想要测试的图像，adb push 到手机上即可
+Q3：如何封装到手机APP中？
+A3：此demo旨在提供能在手机上运行OCR的核心算法部分，PaddleOCR/deploy/android_demo是将这个demo封装到手机app的示例，供参考
--- a/deploy/lite/readme_en.md
+++ b/deploy/lite/readme_en.md
- [Tutorial of PaddleOCR Mobile deployment](#tutorial-of-paddleocr-mobile-deployment)
-  - [1. Preparation](#1-preparation)
-    - [Preparation environment](#preparation-environment)
-    - [1.1 Prepare the cross-compilation environment](#11-prepare-the-cross-compilation-environment)
-    - [1.2 Prepare Paddle-Lite library](#12-prepare-paddle-lite-library)
-  - [2 Run](#2-run)
-    - [2.1 Inference Model Optimization](#21-inference-model-optimization)
-    - [2.2 Run optimized model on Phone](#22-run-optimized-model-on-phone)
-      - [注意：](#注意)
-  - [FAQ](#faq)
-# Tutorial of PaddleOCR Mobile deployment
-This tutorial will introduce how to use [Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite) to deploy PaddleOCR ultra-lightweight Chinese and English detection models on mobile phones.
-paddle-lite is a lightweight inference engine for PaddlePaddle. It provides efficient inference capabilities for mobile phones and IoT, and extensively integrates cross-platform hardware to provide lightweight deployment solutions for end-side deployment issues.
-## 1. Preparation
-### Preparation environment
- Computer (for Compiling Paddle Lite)
- Mobile phone (arm7 or arm8)
-### 1.1 Prepare the cross-compilation environment
-The cross-compilation environment is used to compile C++ demos of Paddle Lite and PaddleOCR.
-Supports multiple development environments.
-For the compilation process of different development environments, please refer to the corresponding documents.
-1. [Docker](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#docker)
-2. [Linux](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#linux)
-3. [MAC OS](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#mac-os)
-### 1.2 Prepare Paddle-Lite library
-There are two ways to obtain the Paddle-Lite library：
- 1. Download directly, the download link of the Paddle-Lite library is as follows：
-      | Platform | Paddle-Lite library download link |
-      |---|---|
-      |Android|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.android.armv7.gcc.c++_shared.with_extra.with_cv.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.android.armv8.gcc.c++_shared.with_extra.with_cv.tar.gz)|
-      |IOS|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.ios.armv7.with_cv.with_extra.with_log.tiny_publish.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.ios.armv8.with_cv.with_extra.with_log.tiny_publish.tar.gz)|
-      Note: 1. The above Paddle-Lite library is compiled from the Paddle-Lite 2.10 branch. For more information about Paddle-Lite 2.10, please refer to [link](https://github.com/PaddlePaddle/Paddle-Lite/releases/tag/v2.10).
- 2. [Recommended] Compile Paddle-Lite to get the prediction library. The compilation method of Paddle-Lite is as follows：
-```
-git clone https://github.com/PaddlePaddle/Paddle-Lite.git
-cd Paddle-Lite
-# Switch to Paddle-Lite release/v2.10 stable branch
-git checkout release/v2.10
-./lite/tools/build_android.sh  --arch=armv8  --with_cv=ON --with_extra=ON
-```
-Note: When compiling Paddle-Lite to obtain the Paddle-Lite library, you need to turn on the two options `--with_cv=ON --with_extra=ON`, `--arch` means the `arm` version, here is designated as armv8,
-More compilation commands refer to the introduction [link](https://paddle-lite.readthedocs.io/zh/release-v2.10_a/source_compile/linux_x86_compile_android.html) 。
-After directly downloading the Paddle-Lite library and decompressing it, you can get the `inference_lite_lib.android.armv8/` folder, and the Paddle-Lite library obtained by compiling Paddle-Lite is located
-`Paddle-Lite/build.lite.android.armv8.gcc/inference_lite_lib.android.armv8/` folder.
-The structure of the prediction library is as follows:
-```
-inference_lite_lib.android.armv8/
-|-- cxx                                        C++ prebuild library
-|   |-- include                                C++
-|   |   |-- paddle_api.h
-|   |   |-- paddle_image_preprocess.h
-|   |   |-- paddle_lite_factory_helper.h
-|   |   |-- paddle_place.h
-|   |   |-- paddle_use_kernels.h
-|   |   |-- paddle_use_ops.h
-|   |   `-- paddle_use_passes.h
-|   `-- lib                                           C++ library
-|       |-- libpaddle_api_light_bundled.a             C++ static library
-|       `-- libpaddle_light_api_shared.so             C++ dynamic library
-|-- java                                     Java library
-|   |-- jar
-|   |   `-- PaddlePredictor.jar
-|   |-- so
-|   |   `-- libpaddle_lite_jni.so
-|   `-- src
-|-- demo                                     C++ and Java demo
-|   |-- cxx                                  C++ demo
-|   `-- java                                 Java demo
-```
-## 2 Run
-### 2.1 Inference Model Optimization
-Paddle Lite provides a variety of strategies to automatically optimize the original training model, including quantization, sub-graph fusion, hybrid scheduling, Kernel optimization and so on. In order to make the optimization process more convenient and easy to use, Paddle Lite provide opt tools to automatically complete the optimization steps and output a lightweight, optimal executable model.
-If you have prepared the model file ending in .nb, you can skip this step.
-The following table also provides a series of models that can be deployed on mobile phones to recognize Chinese. You can directly download the optimized model.
-|Version|Introduction|Model size|Detection model|Text Direction model|Recognition model|Paddle-Lite branch|
-|---|---|---|---|---|---|---|
-|PP-OCRv2|extra-lightweight chinese OCR optimized model|11M|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_PP-OCRv2_det_infer_opt.nb)|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_ppocr_mobile_v2.0_cls_infer_opt.nb)|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_PP-OCRv2_rec_infer_opt.nb)|v2.10|
-|PP-OCRv2(slim)|extra-lightweight chinese OCR optimized model|4.6M|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_PP-OCRv2_det_slim_opt.nb)|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_ppocr_mobile_v2.0_cls_slim_opt.nb)|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_PP-OCRv2_rec_slim_opt.nb)|v2.10|
-If you directly use the model in the above table for deployment, you can skip the following steps and directly read [Section 2.2](#2.2-Run-optimized-model-on-Phone).
-If the model to be deployed is not in the above table, you need to follow the steps below to obtain the optimized model.
-The `opt` tool can be obtained by compiling Paddle Lite.
-```
-git clone https://github.com/PaddlePaddle/Paddle-Lite.git
-cd Paddle-Lite
-git checkout release/v2.10
-./lite/tools/build.sh build_optimize_tool
-```
-After the compilation is complete, the opt file is located under build.opt/lite/api/, You can view the operating options and usage of opt in the following ways:
-```
-cd build.opt/lite/api/
-./opt
-```
-|Options|Description|
-|---|---|
-|--model_dir|The path of the PaddlePaddle model to be optimized (non-combined form)|
-|--model_file|The network structure file path of the PaddlePaddle model (combined form) to be optimized|
-|--param_file|The weight file path of the PaddlePaddle model (combined form) to be optimized|
-|--optimize_out_type|Output model type, currently supports two types: protobuf and naive_buffer, among which naive_buffer is a more lightweight serialization/deserialization implementation. If you need to perform model prediction on the mobile side, please set this option to naive_buffer. The default is protobuf|
-|--optimize_out|The output path of the optimized model|
-|--valid_targets|The executable backend of the model, the default is arm. Currently it supports x86, arm, opencl, npu, xpu, multiple backends can be specified at the same time (separated by spaces), and Model Optimize Tool will automatically select the best method. If you need to support Huawei NPU (DaVinci architecture NPU equipped with Kirin 810/990 Soc), it should be set to npu, arm|
-|--record_tailoring_info|When using the function of cutting library files according to the model, set this option to true to record the kernel and OP information contained in the optimized model. The default is false|
-`--model_dir` is suitable for the non-combined mode of the model to be optimized, and the inference model of PaddleOCR is the combined mode, that is, the model structure and model parameters are stored in a single file.
-The following takes the ultra-lightweight Chinese model of PaddleOCR as an example to introduce the use of the compiled opt file to complete the conversion of the inference model to the Paddle-Lite optimized model
-```
-# 【[Recommendation] Download the Chinese and English inference model of PP-OCRv2
-wget  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_slim_quant_infer.tar && tar xf  ch_PP-OCRv2_det_slim_quant_infer.tar
-wget  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_slim_quant_infer.tar && tar xf  ch_PP-OCRv2_rec_slim_quant_infer.tar
-wget  https://paddleocr.bj.bcebos.com/dygraph_v2.0/slim/ch_ppocr_mobile_v2.0_cls_slim_infer.tar && tar xf  ch_ppocr_mobile_v2.0_cls_slim_infer.tar
-# Convert detection model
-./opt --model_file=./ch_PP-OCRv2_det_slim_quant_infer/inference.pdmodel  --param_file=./ch_PP-OCRv2_det_slim_quant_infer/inference.pdiparams  --optimize_out=./ch_PP-OCRv2_det_slim_opt --valid_targets=arm  --optimize_out_type=naive_buffer
-# Convert recognition model
-./opt --model_file=./ch_PP-OCRv2_rec_slim_quant_infer/inference.pdmodel  --param_file=./ch_PP-OCRv2_rec_slim_quant_infer/inference.pdiparams  --optimize_out=./ch_PP-OCRv2_rec_slim_opt --valid_targets=arm  --optimize_out_type=naive_buffer
-# Convert angle classifier model
-./opt --model_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdmodel  --param_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdiparams  --optimize_out=./ch_ppocr_mobile_v2.0_cls_slim_opt --valid_targets=arm  --optimize_out_type=naive_buffer
-```
-After the conversion is successful, there will be more files ending with `.nb` in the inference model directory, which is the successfully converted model file.
-<a name="2.2-Run-optimized-model-on-Phone"></a>
-### 2.2 Run optimized model on Phone
-Some preparatory work is required first.
- 1. Prepare an Android phone with arm8. If the compiled prediction library and opt file are armv7, you need an arm7 phone and modify ARM_ABI = arm7 in the Makefile.
- 2. Make sure the phone is connected to the computer, open the USB debugging option of the phone, and select the file transfer mode.
- 3. Install the adb tool on the computer.
-    3.1. Install ADB for MAC:
-    ```
-    brew cask install android-platform-tools
-    ```
-    3.2. Install ADB for Linux
-    ```
-    sudo apt update
-    sudo apt install -y wget adb
-    ```
-    3.3. Install ADB for windows
-    To install on win, you need to go to Google's Android platform to download the adb package for installation：[link](https://developer.android.com/studio)
-    Verify whether adb is installed successfully
-     ```
-    adb devices
-    ```
-    If there is device output, it means the installation is successful。
-    ```
-       List of devices attached
-       744be294    device
-    ```
- 4. Prepare optimized models, prediction library files, test images and dictionary files used.
- ```
- git clone https://github.com/PaddlePaddle/PaddleOCR.git
- cd PaddleOCR/deploy/lite/
- # run prepare.sh
- sh prepare.sh /{lite prediction library path}/inference_lite_lib.android.armv8
- #
- cd /{lite prediction library path}/inference_lite_lib.android.armv8/
- cd demo/cxx/ocr/
- # copy paddle-lite C++ .so file to debug/ directory
- cp ../../../cxx/lib/libpaddle_light_api_shared.so ./debug/
- cd inference_lite_lib.android.armv8/demo/cxx/ocr/
- cp ../../../cxx/lib/libpaddle_light_api_shared.so ./debug/
- ```
-Prepare the test image, taking PaddleOCR/doc/imgs/11.jpg as an example, copy the image file to the demo/cxx/ocr/debug/ folder. Prepare the model files optimized by the lite opt tool, ch_det_mv3_db_opt.nb, ch_rec_mv3_crnn_opt.nb, and place them under the demo/cxx/ocr/debug/ folder.
-The structure of the OCR demo is as follows after the above command is executed:
-```
-demo/cxx/ocr/
-|-- debug/  
-|   |--ch_PP-OCRv2_det_slim_opt.nb           Detection model
-|   |--ch_PP-OCRv2_rec_slim_opt.nb           Recognition model
-|   |--ch_ppocr_mobile_v2.0_cls_slim_opt.nb           Text direction classification model
-|   |--11.jpg                           Image for OCR
-|   |--ppocr_keys_v1.txt                Dictionary file
-|   |--libpaddle_light_api_shared.so    C++ .so file
-|   |--config.txt                       Config file
-|-- config.txt                  Config file
-|-- cls_process.cc              Pre-processing and post-processing files for the angle classifier
-|-- cls_process.h
-|-- crnn_process.cc             Pre-processing and post-processing files for the CRNN model
-|-- crnn_process.h
-|-- db_post_process.cc          Pre-processing and post-processing files for the DB model
-|-- db_post_process.h
-|-- Makefile  
-|-- ocr_db_crnn.cc              C++ main code
-```
-#### 注意：
-1. `ppocr_keys_v1.txt` is a Chinese dictionary file. If the nb model is used for English recognition or other language recognition, dictionary file should be replaced with a dictionary of the corresponding language. PaddleOCR provides a variety of dictionaries under ppocr/utils/, including:
-```
-dict/french_dict.txt     # french
-dict/german_dict.txt     # german
-ic15_dict.txt       # english
-dict/japan_dict.txt      # japan
-dict/korean_dict.txt     # korean
-ppocr_keys_v1.txt   # chinese
-```
-2.  `config.txt` of the detector and classifier, as shown below:
-```
-max_side_len  960         #  Limit the maximum image height and width to 960
-det_db_thresh  0.3        # Used to filter the binarized image of DB prediction, setting 0.-0.3 has no obvious effect on the result
-det_db_box_thresh  0.5    # DDB post-processing filter box threshold, if there is a missing box detected, it can be reduced as appropriate
-det_db_unclip_ratio  1.6  # Indicates the compactness of the text box, the smaller the value, the closer the text box to the text
-use_direction_classify  0  # Whether to use the direction classifier, 0 means not to use, 1 means to use
-```
- 5. Run Model on phone
-After the above steps are completed, you can use adb to push the file to the phone to run, the steps are as follows:
- ```
- # Execute the compilation and get the executable file ocr_db_crnn
- # The first execution of this command will download dependent libraries such as opencv. After the download is complete, you need to execute it again
- make -j
- # Move the compiled executable file to the debug folder
- mv ocr_db_crnn ./debug/
- # Push the debug folder to the phone
- adb push debug /data/local/tmp/
- adb shell
- cd /data/local/tmp/debug
- export LD_LIBRARY_PATH=${PWD}:$LD_LIBRARY_PATH
- # The use of ocr_db_crnn is:
- # ./ocr_db_crnn Detection model file Orientation classifier model file Recognition model file Test image path Dictionary file path
- ./ocr_db_crnn ch_PP-OCRv2_det_slim_opt.nb  ch_PP-OCRv2_rec_slim_opt.nb  ch_ppocr_mobile_v2.0_cls_opt.nb  ./11.jpg  ppocr_keys_v1.txt
- ```
-If you modify the code, you need to recompile and push to the phone.
-The outputs are as follows:
-<div align="center">
-    <img src="imgs/lite_demo.png" width="600">
-</div>
-## FAQ
-Q1: What if I want to change the model, do I need to run it again according to the process?
-A1: If you have performed the above steps, you only need to replace the .nb model file to complete the model replacement.
-Q2: How to test with another picture?
-A2: Replace the .jpg test image under ./debug with the image you want to test, and run adb push to push new image to the phone.
-Q3: How to package it into the mobile APP?
-A3: This demo aims to provide the core algorithm part that can run OCR on mobile phones. Further, PaddleOCR/deploy/android_demo is an example of encapsulating this demo into a mobile app for reference.
--- a/deploy/paddle2onnx/readme_ch.md
+++ b/deploy/paddle2onnx/readme_ch.md
+# Paddle2ONNX模型转化与预测
+本章节介绍 PaddleOCR 模型如何转化为 ONNX 模型，并基于 ONNXRuntime 引擎预测。
+## 1. 环境准备
+需要准备 PaddleOCR、Paddle2ONNX 模型转化环境，和 ONNXRuntime 预测环境
+###  PaddleOCR
+克隆PaddleOCR的仓库，使用release/2.4分支，并进行安装，由于PaddleOCR仓库比较大，git clone速度比较慢，所以本教程已下载
+```
+git clone  -b release/2.4 https://github.com/PaddlePaddle/PaddleOCR.git
+cd PaddleOCR && python3.7 setup.py install
+```
+###  Paddle2ONNX
+Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式，算子目前稳定支持导出 ONNX Opset 9~11，部分Paddle算子支持更低的ONNX Opset转换。
+更多细节可参考 [Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX/blob/develop/README_zh.md)
+- 安装 Paddle2ONNX
+```
+python3.7 -m pip install paddle2onnx
+```
+- 安装 ONNXRuntime
+```
+# 建议安装 1.9.0 版本，可根据环境更换版本号
+python3.7 -m pip install onnxruntime==1.9.0
+```
+## 2. 模型转换
+- Paddle 模型下载
+有两种方式获取Paddle静态图模型：在 [model_list](../../doc/doc_ch/models_list.md) 中下载PaddleOCR提供的预测模型；
+参考[模型导出说明](../../doc/doc_ch/inference.md#训练模型转inference模型)把训练好的权重转为 inference_model。
+以 ppocr 中文检测、识别、分类模型为例：
+```
+wget -nc  -P ./inference https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar
+cd ./inference && tar xf ch_PP-OCRv2_det_infer.tar && cd ..
+wget -nc  -P ./inference https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar
+cd ./inference && tar xf ch_PP-OCRv2_rec_infer.tar && cd ..
+wget -nc  -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
+cd ./inference && tar xf ch_ppocr_mobile_v2.0_cls_infer.tar && cd ..
+```
+- 模型转换
+使用 Paddle2ONNX 将Paddle静态图模型转换为ONNX模型格式：
+```
+paddle2onnx --model_dir ./inference/ch_PP-OCRv2_det_infer \
+--model_filename inference.pdmodel \
+--params_filename inference.pdiparams \
+--save_file ./inference/det_onnx/model.onnx \
+--opset_version 10 \
+--input_shape_dict="{'x':[-1,3,-1,-1]}" \
+--enable_onnx_checker True
+paddle2onnx --model_dir ./inference/ch_PP-OCRv2_rec_infer \
+--model_filename inference.pdmodel \
+--params_filename inference.pdiparams \
+--save_file ./inference/rec_onnx/model.onnx \
+--opset_version 10 \
+--input_shape_dict="{'x':[-1,3,-1,-1]}" \
+--enable_onnx_checker True
+paddle2onnx --model_dir ./inference/ch_ppocr_mobile_v2.0_cls_infer \
+--model_filename ch_ppocr_mobile_v2.0_cls_infer/inference.pdmodel \
+--params_filename ch_ppocr_mobile_v2.0_cls_infer/inference.pdiparams \
+--save_file ./inferencecls_onnx/model.onnx \
+--opset_version 10 \
+--input_shape_dict="{'x':[-1,3,-1,-1]}" \
+--enable_onnx_checker True
+```
+执行完毕后，ONNX 模型会被分别保存在 `./inference/det_onnx/`，`./inference/rec_onnx/`，`./inference/cls_onnx/`路径下
+* 注意：对于OCR模型，转化过程中必须采用动态shape的形式，即加入选项--input_shape_dict="{'x': [-1, 3, -1, -1]}"，否则预测结果可能与直接使用Paddle预测有细微不同。
+  另外，以下几个模型暂不支持转换为 ONNX 模型：
+  NRTR、SAR、RARE、SRN
+## 3. 推理预测
+以中文OCR模型为例，使用 ONNXRuntime 预测可执行如下命令：
+```
+python3.7 tools/infer/predict_system.py --use_gpu=False --use_onnx=True \
+--det_model_dir=./inference/det_onnx/model.onnx  \
+--rec_model_dir=./inference/rec_onnx/model.onnx  \
+--cls_model_dir=./inference/cls_onnx/model.onnx  \
+--image_dir=./deploy/lite/imgs/lite_demo.png
+```
+以中文OCR模型为例，使用 Paddle Inference 预测可执行如下命令：
+```
+python3.7 tools/infer/predict_system.py --use_gpu=False \
+--cls_model_dir=./inference/ch_ppocr_mobile_v2.0_cls_infer \
+--rec_model_dir=./inference/ch_PP-OCRv2_rec_infer \
+--det_model_dir=./inference/ch_PP-OCRv2_det_infer \
+--image_dir=./deploy/lite/imgs/lite_demo.png
+```
+执行命令后在终端会打印出预测的识别信息，并在 `./inference_results/` 下保存可视化结果。
+ONNXRuntime 执行效果：
+<div align="center">
+    <img src="./images/lite_demo_onnx.png" width=800">
+</div>
+Paddle Inference 执行效果：
+<div align="center">
+    <img src="./images/lite_demo_paddle.png" width=800">
+</div>
+使用 ONNXRuntime 预测，终端输出：
+```
+[2022/02/22 17:48:27] root DEBUG: dt_boxes num : 38, elapse : 0.043187856674194336
+[2022/02/22 17:48:27] root DEBUG: rec_res num  : 38, elapse : 0.592170000076294
+[2022/02/22 17:48:27] root DEBUG: 0  Predict time of ./deploy/lite/imgs/lite_demo.png: 0.642s
+[2022/02/22 17:48:27] root DEBUG: The, 0.984
+[2022/02/22 17:48:27] root DEBUG: visualized, 0.882
+[2022/02/22 17:48:27] root DEBUG: etect18片, 0.720
+[2022/02/22 17:48:27] root DEBUG: image saved in./vis.jpg, 0.947
+[2022/02/22 17:48:27] root DEBUG: 纯臻营养护发素0.993604, 0.996
+[2022/02/22 17:48:27] root DEBUG: 产品信息/参数, 0.922
+[2022/02/22 17:48:27] root DEBUG: 0.992728, 0.914
+[2022/02/22 17:48:27] root DEBUG: （45元／每公斤，100公斤起订）, 0.926
+[2022/02/22 17:48:27] root DEBUG: 0.97417, 0.977
+[2022/02/22 17:48:27] root DEBUG: 每瓶22元，1000瓶起订）0.993976, 0.962
+[2022/02/22 17:48:27] root DEBUG: 【品牌】：代加工方式/0EMODM, 0.945
+[2022/02/22 17:48:27] root DEBUG: 0.985133, 0.980
+[2022/02/22 17:48:27] root DEBUG: 【品名】：纯臻营养护发素, 0.921
+[2022/02/22 17:48:27] root DEBUG: 0.995007, 0.883
+[2022/02/22 17:48:27] root DEBUG: 【产品编号】：YM-X-30110.96899, 0.955
+[2022/02/22 17:48:27] root DEBUG: 【净含量】：220ml, 0.943
+[2022/02/22 17:48:27] root DEBUG: Q.996577, 0.932
+[2022/02/22 17:48:27] root DEBUG: 【适用人群】：适合所有肤质, 0.913
+[2022/02/22 17:48:27] root DEBUG: 0.995842, 0.969
+[2022/02/22 17:48:27] root DEBUG: 【主要成分】：鲸蜡硬脂醇、燕麦B-葡聚, 0.883
+[2022/02/22 17:48:27] root DEBUG: 0.961928, 0.964
+[2022/02/22 17:48:27] root DEBUG: 10, 0.812
+[2022/02/22 17:48:27] root DEBUG: 糖、椰油酰胺丙基甜菜碱、泛醒, 0.866
+[2022/02/22 17:48:27] root DEBUG: 0.925898, 0.943
+[2022/02/22 17:48:27] root DEBUG: （成品包材）, 0.974
+[2022/02/22 17:48:27] root DEBUG: 0.972573, 0.961
+[2022/02/22 17:48:27] root DEBUG: 【主要功能】：可紧致头发磷层，从而达到, 0.936
+[2022/02/22 17:48:27] root DEBUG: 0.994448, 0.952
+[2022/02/22 17:48:27] root DEBUG: 13, 0.998
+[2022/02/22 17:48:27] root DEBUG: 即时持久改善头发光泽的效果，给干燥的头, 0.994
+[2022/02/22 17:48:27] root DEBUG: 0.990198, 0.975
+[2022/02/22 17:48:27] root DEBUG: 14, 0.977
+[2022/02/22 17:48:27] root DEBUG: 发足够的滋养, 0.991
+[2022/02/22 17:48:27] root DEBUG: 0.997668, 0.918
+[2022/02/22 17:48:27] root DEBUG: 花费了0.457335秒, 0.901
+[2022/02/22 17:48:27] root DEBUG: The visualized image saved in ./inference_results/lite_demo.png
+[2022/02/22 17:48:27] root INFO: The predict total time is 0.7003889083862305
+```
+使用 Paddle Inference 预测，终端输出：
+```
+[2022/02/22 17:47:25] root DEBUG: dt_boxes num : 38, elapse : 0.11791276931762695
+[2022/02/22 17:47:27] root DEBUG: rec_res num  : 38, elapse : 2.6206860542297363
+[2022/02/22 17:47:27] root DEBUG: 0  Predict time of ./deploy/lite/imgs/lite_demo.png: 2.746s
+[2022/02/22 17:47:27] root DEBUG: The, 0.984
+[2022/02/22 17:47:27] root DEBUG: visualized, 0.882
+[2022/02/22 17:47:27] root DEBUG: etect18片, 0.720
+[2022/02/22 17:47:27] root DEBUG: image saved in./vis.jpg, 0.947
+[2022/02/22 17:47:27] root DEBUG: 纯臻营养护发素0.993604, 0.996
+[2022/02/22 17:47:27] root DEBUG: 产品信息/参数, 0.922
+[2022/02/22 17:47:27] root DEBUG: 0.992728, 0.914
+[2022/02/22 17:47:27] root DEBUG: （45元／每公斤，100公斤起订）, 0.926
+[2022/02/22 17:47:27] root DEBUG: 0.97417, 0.977
+[2022/02/22 17:47:27] root DEBUG: 每瓶22元，1000瓶起订）0.993976, 0.962
+[2022/02/22 17:47:27] root DEBUG: 【品牌】：代加工方式/0EMODM, 0.945
+[2022/02/22 17:47:27] root DEBUG: 0.985133, 0.980
+[2022/02/22 17:47:27] root DEBUG: 【品名】：纯臻营养护发素, 0.921
+[2022/02/22 17:47:27] root DEBUG: 0.995007, 0.883
+[2022/02/22 17:47:27] root DEBUG: 【产品编号】：YM-X-30110.96899, 0.955
+[2022/02/22 17:47:27] root DEBUG: 【净含量】：220ml, 0.943
+[2022/02/22 17:47:27] root DEBUG: Q.996577, 0.932
+[2022/02/22 17:47:27] root DEBUG: 【适用人群】：适合所有肤质, 0.913
+[2022/02/22 17:47:27] root DEBUG: 0.995842, 0.969
+[2022/02/22 17:47:27] root DEBUG: 【主要成分】：鲸蜡硬脂醇、燕麦B-葡聚, 0.883
+[2022/02/22 17:47:27] root DEBUG: 0.961928, 0.964
+[2022/02/22 17:47:27] root DEBUG: 10, 0.812
+[2022/02/22 17:47:27] root DEBUG: 糖、椰油酰胺丙基甜菜碱、泛醒, 0.866
+[2022/02/22 17:47:27] root DEBUG: 0.925898, 0.943
+[2022/02/22 17:47:27] root DEBUG: （成品包材）, 0.974
+[2022/02/22 17:47:27] root DEBUG: 0.972573, 0.961
+[2022/02/22 17:47:27] root DEBUG: 【主要功能】：可紧致头发磷层，从而达到, 0.936
+[2022/02/22 17:47:27] root DEBUG: 0.994448, 0.952
+[2022/02/22 17:47:27] root DEBUG: 13, 0.998
+[2022/02/22 17:47:27] root DEBUG: 即时持久改善头发光泽的效果，给干燥的头, 0.994
+[2022/02/22 17:47:27] root DEBUG: 0.990198, 0.975
+[2022/02/22 17:47:27] root DEBUG: 14, 0.977
+[2022/02/22 17:47:27] root DEBUG: 发足够的滋养, 0.991
+[2022/02/22 17:47:27] root DEBUG: 0.997668, 0.918
+[2022/02/22 17:47:27] root DEBUG: 花费了0.457335秒, 0.901
+[2022/02/22 17:47:27] root DEBUG: The visualized image saved in ./inference_results/lite_demo.png
+[2022/02/22 17:47:27] root INFO: The predict total time is 2.8338775634765625
+```
--- a/deploy/paddlejs/README.md
+++ b/deploy/paddlejs/README.md
+English| [简体中文](README_ch.md) 
+# Paddle.js
+[Paddle.js](https://github.com/PaddlePaddle/Paddle.js) is a web project for Baidu PaddlePaddle, which is an open source deep learning framework running in the browser. Paddle.js can either load a pre-trained model, or transforming a model from paddle-hub with model transforming tools provided by Paddle.js. It could run in every browser with WebGL/WebGPU/WebAssembly supported. It could also run in Baidu Smartprogram and WX miniprogram.
+- [Online experience](https://paddlejs.baidu.com/ocr)
+- [Tutorial](https://github.com/PaddlePaddle/Paddle.js/blob/release/v2.2.3/packages/paddlejs-models/ocr/README_cn.md)
+- Visualization：
+<div align="center">
+    <img src="./paddlejs_demo.gif" width="800">
+</div>
\ No newline at end of file
--- a/deploy/paddlejs/README_ch.md
+++ b/deploy/paddlejs/README_ch.md
+[English](README.md) | 简体中文
+# Paddle.js 网页前端部署
+[Paddle.js](https://github.com/PaddlePaddle/Paddle.js) 是百度 PaddlePaddle 的 web 方向子项目，是一个运行在浏览器中的开源深度学习框架。Paddle.js 可以加载提前训练好的 paddle 模型，通过 Paddle.js 的模型转换工具 paddlejs-converter 变成浏览器友好的模型进行在线推理预测使用。目前，Paddle.js 可以在支持 WebGL/WebGPU/WebAssembly 的浏览器中运行，也可以在百度小程序和微信小程序环境下运行。
+- [在线体验](https://paddlejs.baidu.com/ocr)
+- [直达教程](https://github.com/PaddlePaddle/Paddle.js/blob/release/v2.2.3/packages/paddlejs-models/ocr/README_cn.md)
+- 效果：
+<div align="center">
+    <img src="./paddlejs_demo.gif" width="800">
+</div>
\ No newline at end of file
--- a/deploy/paddlejs/paddlejs_demo.gif
+++ b/deploy/paddlejs/paddlejs_demo.gif
--- a/deploy/slim/quantization/README.md
+++ b/deploy/slim/quantization/README.md
@@ -42,7 +42,7 @@ python deploy/slim/quantization/quant.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3
 # 比如下载提供的训练模型
 wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_train.tar
 tar -xf ch_ppocr_mobile_v2.0_det_train.tar
-python deploy/slim/quantization/quant.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o Global.pretrained_model=./ch_ppocr_mobile_v2.0_det_train/best_accuracy   Global.save_model_dir=./output/quant_inference_model
+python deploy/slim/quantization/quant.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o Global.pretrained_model=./ch_ppocr_mobile_v2.0_det_train/best_accuracy   Global.save_model_dir=./output/quant_model
 ```
 如果要训练识别模型的量化，修改配置文件和加载的模型参数即可。

--- a/doc/PPOCR.pdf
+++ b/doc/PPOCR.pdf
--- a/doc/deployment.png
+++ b/doc/deployment.png
--- a/doc/deployment_en.png
+++ b/doc/deployment_en.png
--- a/doc/doc_ch/algorithm.md
+++ b/doc/doc_ch/algorithm.md
+# 前沿算法与模型
+PaddleOCR将**持续新增**支持OCR领域前沿算法与模型，已支持的模型与使用教程可点击下方列表查看：
+- [文本检测算法](./algorithm_overview.md#11-%E6%96%87%E6%9C%AC%E6%A3%80%E6%B5%8B%E7%AE%97%E6%B3%95)
+- [文本识别算法](./algorithm_overview.md#12-%E6%96%87%E6%9C%AC%E8%AF%86%E5%88%AB%E7%AE%97%E6%B3%95)
+- [端到端算法](./algorithm_overview.md#2-%E6%96%87%E6%9C%AC%E8%AF%86%E5%88%AB%E7%AE%97%E6%B3%95)
+**欢迎广大开发者合作共建，贡献更多算法，合入有奖🎁！具体可查看[社区常规赛](https://github.com/PaddlePaddle/PaddleOCR/issues/4982)。**
+新增算法可参考如下教程：
+- [使用PaddleOCR架构添加新算法](./add_new_algorithm.md)
\ No newline at end of file
--- a/doc/doc_ch/algorithm_det_db.md
+++ b/doc/doc_ch/algorithm_det_db.md
+# DB
+- [1. 算法简介](#1)
+- [2. 环境配置](#2)
+- [3. 模型训练、评估、预测](#3)
+    - [3.1 训练](#3-1)
+    - [3.2 评估](#3-2)
+    - [3.3 预测](#3-3)
+- [4. 推理部署](#4)
+    - [4.1 Python推理](#4-1)
+    - [4.2 C++推理](#4-2)
+    - [4.3 Serving服务化部署](#4-3)
+    - [4.4 更多推理部署](#4-4)
+- [5. FAQ](#5)
+<a name="1"></a>
+## 1. 算法简介
+论文信息：
+> [Real-time Scene Text Detection with Differentiable Binarization](https://arxiv.org/abs/1911.08947)
+> Liao, Minghui and Wan, Zhaoyi and Yao, Cong and Chen, Kai and Bai, Xiang
+> AAAI, 2020
+在ICDAR2015文本检测公开数据集上，算法复现效果如下：
+|模型|骨干网络|配置文件|precision|recall|Hmean|下载链接|
+| --- | --- | --- | --- | --- | --- | --- |
+|DB|ResNet50_vd|configs/det/det_r50_vd_db.yml|86.41%|78.72%|82.38%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_db_v2.0_train.tar)|
+|DB|MobileNetV3|configs/det/det_mv3_db.yml|77.29%|73.08%|75.12%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_mv3_db_v2.0_train.tar)|
+<a name="2"></a>
+## 2. 环境配置
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目代码。
+<a name="3"></a>
+## 3. 模型训练、评估、预测
+请参考[文本检测训练教程](./detection.md)。PaddleOCR对代码进行了模块化，训练不同的检测模型只需要**更换配置文件**即可。
+<a name="4"></a>
+## 4. 推理部署
+<a name="4-1"></a>
+### 4.1 Python推理
+首先将DB文本检测训练过程中保存的模型，转换成inference model。以基于Resnet50_vd骨干网络，在ICDAR2015英文数据集训练的模型为例（ [模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_db_v2.0_train.tar) )，可以使用如下命令进行转换：
+```
+python3 tools/export_model.py -c configs/det/det_r50_vd_db.yml -o Global.pretrained_model=./det_r50_vd_db_v2.0_train/best_accuracy  Global.save_inference_dir=./inference/det_db
+```
+DB文本检测模型推理，可以执行如下命令：
+```
+python3 tools/infer/predict_det.py --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_db/"
+```
+可视化文本检测结果默认保存到`./inference_results`文件夹里面，结果文件的名称前缀为'det_res'。结果示例如下：
+![](../imgs_results/det_res_img_10_db.jpg)
+**注意**：由于ICDAR2015数据集只有1000张训练图像，且主要针对英文场景，所以上述模型对中文文本图像检测效果会比较差。
+<a name="4-2"></a>
+### 4.2 C++推理
+敬请期待
+<a name="4-3"></a>
+### 4.3 Serving服务化部署
+敬请期待
+<a name="4-4"></a>
+### 4.4 更多推理部署
+敬请期待
+<a name="5"></a>
+## 5. FAQ
+## 引用
+```bibtex
+@inproceedings{liao2020real,
+  title={Real-time scene text detection with differentiable binarization},
+  author={Liao, Minghui and Wan, Zhaoyi and Yao, Cong and Chen, Kai and Bai, Xiang},
+  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
+  volume={34},
+  number={07},
+  pages={11474--11481},
+  year={2020}
+}
+```
\ No newline at end of file
--- a/doc/doc_ch/pgnet.md
+++ b/doc/doc_ch/pgnet.md
@@ -43,7 +43,7 @@ PGNet算法细节详见[论文](https://www.aaai.org/AAAI21Papers/AAAI-2885.Wang
 <a name="环境配置"></a>
 ## 二、环境配置
-请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《PaddleOCR全景图与项目克隆》](./paddleOCR_overview.md)克隆项目
+请先参考[《运行环境准备》](./environment.md)配置PaddleOCR运行环境，参考[《项目克隆》](./clone.md)克隆项目
 <a name="快速使用"></a>
 ## 三、快速使用

--- a/doc/doc_ch/inference.md
+++ b/doc/doc_ch/inference.md
@@ -14,12 +14,14 @@ inference 模型（`paddle.jit.save`保存的模型）
    - [识别模型转inference模型](#识别模型转inference模型)  
    - [方向分类模型转inference模型](#方向分类模型转inference模型)
 - [二、文本检测模型推理](#文本检测模型推理)
    - [1. 超轻量中文检测模型推理](#超轻量中文检测模型推理)
    - [2. DB文本检测模型推理](#DB文本检测模型推理)
    - [3. EAST文本检测模型推理](#EAST文本检测模型推理)
    - [4. SAST文本检测模型推理](#SAST文本检测模型推理)  
 - [三、文本识别模型推理](#文本识别模型推理)
    - [1. 超轻量中文识别模型推理](#超轻量中文识别模型推理)
    - [2. 基于CTC损失的识别模型推理](#基于CTC损失的识别模型推理)
@@ -27,15 +29,19 @@ inference 模型（`paddle.jit.save`保存的模型）
    - [4. 自定义文本识别字典的推理](#自定义文本识别字典的推理)
    - [5. 多语言模型的推理](#多语言模型的推理)
 - [四、方向分类模型推理](#方向识别模型推理)
    - [1. 方向分类模型推理](#方向分类模型推理)
 - [五、文本检测、方向分类和文字识别串联推理](#文本检测、方向分类和文字识别串联推理)
    - [1. 超轻量中文OCR模型推理](#超轻量中文OCR模型推理)
    - [2. 其他模型推理](#其他模型推理)
 - [六、参数解释](#参数解释)
 - [七、FAQ](#FAQ)

--- a/doc/doc_ch/algorithm_overview.md
+++ b/doc/doc_ch/algorithm_overview.md
-# 两阶段算法
+# OCR算法
- [两阶段算法](#两阶段算法)
+- [1. 两阶段算法](#1-两阶段算法)
-  - [1. 算法介绍](#1-算法介绍)
    - [1.1 文本检测算法](#11-文本检测算法)
    - [1.2 文本识别算法](#12-文本识别算法)
-  - [2. 模型训练](#2-模型训练)
+- [2. 端到端算法](#2-端到端算法)
-  - [3. 模型推理](#3-模型推理)
+本文给出了PaddleOCR已支持的OCR算法列表，以及每个算法在**英文公开数据集**上的模型和指标，主要用于算法简介和算法性能对比，更多包括中文在内的其他数据集上的模型请参考[PP-OCR v2.0 系列模型下载](./models_list.md)。
 <a name="1"></a>
-## 1. 算法介绍
+## 1. 两阶段算法
-本文给出了PaddleOCR已支持的文本检测算法和文本识别算法列表，以及每个算法在**英文公开数据集**上的模型和指标，主要用于算法简介和算法性能对比，更多包括中文在内的其他数据集上的模型请参考[PP-OCR v2.0 系列模型下载](./models_list.md)。
 <a name="11"></a>
 ### 1.1 文本检测算法
-PaddleOCR开源的文本检测算法列表：
+已支持的文本检测算法列表（戳链接获取使用教程）：
- [x]  DB([paper]( https://arxiv.org/abs/1911.08947)) [2]（ppocr推荐）
+- [x]  [DB](./algorithm_det_db.md)
- [x]  EAST([paper](https://arxiv.org/abs/1704.03155))[1]
+- [x]  [EAST](./algorithm_det_east.md)
- [x]  SAST([paper](https://arxiv.org/abs/1908.05498))[4]
+- [x]  [SAST](./algorithm_det_sast.md)
- [x]  PSENet([paper](https://arxiv.org/abs/1903.12473v2)）
+- [x]  [PSENet](./algorithm_det_psenet.md)
- [x]  FCENet([paper](https://arxiv.org/abs/2104.10442))
+- [x]  [FCENet](./algorithm_det_fcenet.md)
 在ICDAR2015文本检测公开数据集上，算法效果如下：
 |模型|骨干网络|precision|recall|Hmean|下载链接|
 | --- | --- | --- | --- | --- | --- |
 |EAST|ResNet50_vd|88.71%|81.36%|84.88%|[训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_east_v2.0_train.tar)|
@@ -50,19 +51,20 @@ PaddleOCR开源的文本检测算法列表：
 * [百度云地址](https://pan.baidu.com/s/12cPnZcVuV1zn5DOd4mqjVw) (提取码: 2bpi)
 * [Google Drive下载地址](https://drive.google.com/drive/folders/1ll2-XEVyCQLpJjawLDiRlvo_i4BqHCJe?usp=sharing)
 <a name="12"></a>
 ### 1.2 文本识别算法
-PaddleOCR基于动态图开源的文本识别算法列表：
+已支持的文本识别算法列表（戳链接获取使用教程）：
- [x]  CRNN([paper](https://arxiv.org/abs/1507.05717))[7]（ppocr推荐）
+- [x]  [CRNN](./algorithm_rec_crnn.md)
- [x]  Rosetta([paper](https://arxiv.org/abs/1910.05085))[10]
+- [x]  [Rosetta](./algorithm_rec_rosetta.md)
- [x]  STAR-Net([paper](http://www.bmva.org/bmvc/2016/papers/paper043/index.html))[11]
+- [x]  [STAR-Net](./algorithm_rec_starnet.md)
- [x]  RARE([paper](https://arxiv.org/abs/1603.03915v1))[12]
+- [x]  [RARE](./algorithm_rec_rare.md)
- [x]  SRN([paper](https://arxiv.org/abs/2003.12294))[5]
+- [x]  [SRN](./algorithm_rec_srn.md)
- [x]  NRTR([paper](https://arxiv.org/abs/1806.00926v2))[13]
+- [x]  [NRTR](./algorithm_rec_nrtr.md)
- [x]  SAR([paper](https://arxiv.org/abs/1811.00751v2))
+- [x]  [SAR](./algorithm_rec_sar.md)
- [x] SEED([paper](https://arxiv.org/pdf/2005.10977.pdf))
+- [x]  [SEED](./algorithm_rec_seed.md)
 参考[DTRB](https://arxiv.org/abs/1904.01906)[3]文字识别训练和评估流程，使用MJSynth和SynthText两个文字识别数据集训练，在IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE数据集上进行评估，算法效果如下：
@@ -81,14 +83,12 @@ PaddleOCR基于动态图开源的文本识别算法列表：
 |SAR|Resnet31| 87.20% | rec_r31_sar | [训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.1/rec/rec_r31_sar_train.tar) |
 |SEED|Aster_Resnet| 85.35% | rec_resnet_stn_bilstm_att | [训练模型](https://paddleocr.bj.bcebos.com/dygraph_v2.1/rec/rec_resnet_stn_bilstm_att.tar) |
-<a name="2"></a>
-## 2. 模型训练
+<a name="2"></a>
-PaddleOCR文本检测算法的训练和使用请参考文档教程中[模型训练/评估中的文本检测部分](./detection.md)。文本识别算法的训练和使用请参考文档教程中[模型训练/评估中的文本识别部分](./recognition.md)。
+## 2. 端到端算法
-<a name="3"></a>
+已支持的端到端OCR算法列表（戳链接获取使用教程）：
+- [x]  [PGNet](./algorithm_e2e_pgnet.md)
-## 3. 模型推理
-上述模型中除PP-OCR系列模型以外，其余模型仅支持基于Python引擎的推理，具体内容可参考[基于Python预测引擎推理](./inference.md)
--- a/doc/doc_ch/android_demo.md
+++ b/doc/doc_ch/android_demo.md
-# Android Demo 快速测试
-### 1. 安装最新版本的Android Studio
-可以从 https://developer.android.com/studio 下载。本Demo使用是4.0版本Android Studio编写。
-### 2. 创建新项目
-Demo测试的时候使用的是NDK 20b版本，20版本以上均可以支持编译成功。
-如果您是初学者，可以用以下方式安装和测试NDK编译环境。
-点击 File -> New ->New Project，  新建  "Native C++" project
-1. Start a new Android Studio project
-    在项目模版中选择 Native C++ 选择PaddleOCR/deploy/android_demo 路径
-    进入项目后会自动编译，第一次编译会花费较长的时间，建议添加代理加速下载。
-**代理添加：**
-选择 Android Studio -> Preferences -> Appearance & Behavior -> System Settings -> HTTP Proxy -> Manual proxy configuration
-![](../demo/proxy.png)
-2. 开始编译
-点击编译按钮，连接手机，跟着Android Studio的引导完成操作。
-在 Android Studio 里看到下图，表示编译完成：
-![](../demo/build.png)
-**提示:** 此时如果出现下列找不到OpenCV的报错信息，请重新点击编译，编译完成后退出项目，再次进入。
-![](../demo/error.png)
-### 3. 发送到手机端
-完成编译，点击运行，在手机端查看效果。
-### 4. 如何自定义demo图片
-1. 图片存放路径:android_demo/app/src/main/assets/images
-   将自定义图片放置在该路径下
-2. 配置文件: android_demo/app/src/main/res/values/strings.xml
-   修改 IMAGE_PATH_DEFAULT 为自定义图片名即可
-# 获得更多支持
-前往[端计算模型生成平台EasyEdge](https://ai.baidu.com/easyedge/app/open_source_demo?referrerUrl=paddlelite)，获得更多开发支持：
- Demo APP：可使用手机扫码安装，方便手机端快速体验文字识别
- SDK：模型被封装为适配不同芯片硬件和操作系统SDK，包括完善的接口，方便进行二次开发
--- a/doc/doc_ch/application.md
+++ b/doc/doc_ch/application.md
+# 场景应用
\ No newline at end of file
--- a/doc/doc_ch/paddleOCR_overview.md
+++ b/doc/doc_ch/paddleOCR_overview.md
-# PaddleOCR全景图与项目克隆
+# 项目克隆
-## 1. PaddleOCR全景图
+## 1. 克隆PaddleOCR repo代码
-PaddleOCR包含丰富的文本检测、文本识别以及端到端算法。结合实际测试与产业经验，PaddleOCR选择DB和CRNN作为基础的检测和识别模型，经过一系列优化策略提出面向产业应用的PP-OCR模型。PP-OCR模型针对通用场景，根据不同语种形成了PP-OCR模型库。基于PP-OCR的能力，PaddleOCR针对文档场景任务发布PP-Structure工具库，包含版面分析和表格识别两大任务。为了打通产业落地的全流程，PaddleOCR提供了规模化的数据生产工具和多种预测部署工具，助力开发者快速落地。
-<div align="center">
-    <img src="../overview.png">
-</div>
-## 2. 项目克隆
-### **2.1 克隆PaddleOCR repo代码**
 ```
 【推荐】git clone https://github.com/PaddlePaddle/PaddleOCR
@@ -24,7 +14,7 @@ git clone https://gitee.com/paddlepaddle/PaddleOCR
 注：码云托管代码可能无法实时同步本github项目更新，存在3~5天延时，请优先使用推荐方式。
-### **2.2 安装第三方库**
+## 2. 安装第三方库
 ```
 cd PaddleOCR