{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# 1 Introduction and Overview of Inference Deployment\n",
    "\n",
    "\n",
    "This chapter mainly introduces the high-performance inference method, service deployment method and end-side deployment method of the PP-OCRv2 system. Through the study of this chapter, you can learn:\n",
    "\n",
    "* Choose the appropriate predictive deployment method according to different scenarios\n",
    "* Inference methods of PP-OCRv2 series models in different scenarios\n",
    "* Inference deployment development methods of Paddle Inference, Paddle Serving, and Paddle Lite\n",
    "\n",
    "## 1.1 Introduction\n",
    "\n",
    "In the previous chapters, we obtained the trained model through the method of model training. When using it to predict, we first need to define the model, then load the trained model, and then preprocess the data Send it to the network for prediction and post-processing, and get the final result. Using this method to make predictions is convenient for debugging, but the prediction efficiency is relatively low.\n",
    "\n",
    "In view of the above problems, for the trained model, there are generally the following two offline prediction methods.\n",
    "\n",
    "1. Inference based on the training engine, that is, using the same set of engine as the training, it is convenient for debugging, which is convenient for us to quickly locate the problem and verify the correctness. Mostly Python language.\n",
    "2. Based on the prediction engine inference, the trained model is transformed, and the parts that are not related to the inference are removed. Using this method can speed up the prediction. Most of them are Python language or C++.\n",
    "\n",
    "The specific differences between the two are shown below.\n",
    "<center>\n",
    "\n",
    "|      | Inference based on training engine    | Inference based on prediction engine    |\n",
    "|:------:|:------|:------|\n",
    "| Features   |1. Use the same set of engine as training</br>2. The network model needs to be defined when inferencing</br>3. Not suitable for system integration| 1. The model needs to be converted, and the parts that are not related to inference are removed</br>2. * *There is no need to define a network model when inferencing**</br>3. Suitable for system integration|\n",
    "| Programming language | More as Python                                     | Python or C++                     |\n",
    "| Prediction step | 1. Define the network structure on the Python side</br>2. Prepare the input data</br>3. Load the training model</br>4. Perform prediction| 1. Prepare the input data</br>2. Load the model structure and model Parameters</br>3. Perform prediction|\n",
    "\n",
    "</center>\n",
    "\n",
    "In actual offline prediction deployment, it is more recommended to perform inference based on the prediction engine.\n",
    "\n",
    "From the perspective of application scenarios, the flying paddle provides the following inference deployment solutions for different application scenarios.\n",
    "\n",
    "<div align=\"center\">\n",
    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/d95a9b662f944af589cf3934b7012f54bfb1ef610c0447568712618fdb6216a2\"  width = \"1200\" />\n",
    "</div>\n",
    "<center>Figure 1: Different deployment options for paddles</center>\n",
    "\n",
    "Specifically, PaddleOCR provides three predictive deployment solutions for different application scenarios.\n",
    "\n",
    "* Inference offline prediction. This method is mainly used in scenarios where the timeliness of the prediction response is not high, especially in scenes that require a large number of picture predictions. Such as document digitization, advertising information extraction, etc. Although the inference request cannot be responded to in time, there is no network delay, the calculation efficiency is relatively high, and the data security is high.\n",
    "* Serving service deployment, this method is mainly used in scenarios that require high timeliness of predictive response, such as the API interface of commercial OCR, real-time photo translation, and topic shooting. Although this method can respond to inference demand in a timely manner, the network is time-consuming and expensive, GPU utilization is often low, and there are data security risks.\n",
    "* Lite end-side deployment, this method mainly hopes to deploy the model to end-side devices such as mobile phones and robots, mainly considering the convenience of deployment and data security, such as ID card recognition in mobile APP, bank card recognition, and instrument monitoring in industrial application scenarios Recognition etc. This method is more sensitive to the size of the OCR model. Although there is no network delay and data security is high, the prediction efficiency is not high due to the limitation of computing power.\n",
    "\n",
    "Based on PP-OCRv2, this chapter introduces the process of text detection, recognition, and system series prediction, inference and deployment.\n",
    "\n",
    "## 1.2 Environmental preparation\n",
    "\n",
    "To experience the content of this chapter, you need to download the PaddleOCR code first and install related dependencies. The specific commands are as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cloning into 'PaddleOCR'...\n",
      "remote: Enumerating objects: 2335, done.\u001b[K\n",
      "remote: Counting objects: 100% (2335/2335), done.\u001b[K\n",
      "remote: Compressing objects: 100% (937/937), done.\u001b[K\n",
      "remote: Total 27956 (delta 1569), reused 2078 (delta 1382), pack-reused 25621\u001b[K\n",
      "Receiving objects: 100% (27956/27956), 245.96 MiB | 4.68 MiB/s, done.\n",
      "Resolving deltas: 100% (19430/19430), done.\n",
      "Checking connectivity... done.\n",
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Collecting pip\n",
      "\u001b[?25l  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl (1.7MB)\n",
      "\u001b[K     |████████████████████████████████| 1.7MB 17.9MB/s eta 0:00:01\n",
      "\u001b[?25hInstalling collected packages: pip\n",
      "  Found existing installation: pip 19.2.3\n",
      "    Uninstalling pip-19.2.3:\n",
      "      Successfully uninstalled pip-19.2.3\n",
      "Successfully installed pip-21.3.1\n",
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Collecting shapely\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ae/20/33ce377bd24d122a4d54e22ae2c445b9b1be8240edb50040b40add950cd9/Shapely-1.8.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB)\n",
      "     |████████████████████████████████| 1.1 MB 27.3 MB/s            \n",
      "\u001b[?25hCollecting scikit-image\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/9a/44/8f8c7f9c9de7fde70587a656d7df7d056e6f05192a74491f7bc074a724d0/scikit_image-0.19.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (13.3 MB)\n",
      "     |████████████████████████████████| 13.3 MB 74.2 MB/s            \n",
      "\u001b[?25hCollecting imgaug==0.4.0\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/66/b1/af3142c4a85cba6da9f4ebb5ff4e21e2616309552caca5e8acefe9840622/imgaug-0.4.0-py2.py3-none-any.whl (948 kB)\n",
      "     |████████████████████████████████| 948 kB 53.5 MB/s            \n",
      "\u001b[?25hCollecting pyclipper\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/c5/fa/2c294127e4f88967149a68ad5b3e43636e94e3721109572f8f17ab15b772/pyclipper-1.3.0.post2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (603 kB)\n",
      "     |████████████████████████████████| 603 kB 55.1 MB/s            \n",
      "\u001b[?25hCollecting lmdb\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2e/dd/ada2fd91cd7832979069c556607903f274470c3d3d2274e0a848908272e8/lmdb-1.2.1-cp37-cp37m-manylinux2010_x86_64.whl (299 kB)\n",
      "     |████████████████████████████████| 299 kB 59.7 MB/s            \n",
      "\u001b[?25hRequirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 6)) (4.27.0)\n",
      "Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 7)) (1.20.3)\n",
      "Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 8)) (2.2.0)\n",
      "Collecting python-Levenshtein\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/2a/dc/97f2b63ef0fa1fd78dcb7195aca577804f6b2b51e712516cc0e902a9a201/python-Levenshtein-0.12.2.tar.gz (50 kB)\n",
      "     |████████████████████████████████| 50 kB 16.0 MB/s            \n",
      "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hCollecting opencv-contrib-python==4.4.0.46\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/08/51/1e0a206dd5c70fea91084e6f43979dc13e8eb175760cc7a105083ec3eb68/opencv_contrib_python-4.4.0.46-cp37-cp37m-manylinux2014_x86_64.whl (55.7 MB)\n",
      "     |████████████████████████████████| 55.7 MB 49 kB/s             \n",
      "\u001b[?25hRequirement already satisfied: cython in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 11)) (0.29)\n",
      "Collecting lxml\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7b/01/16a9b80c8ce4339294bb944f08e157dbfcfbb09ba9031bde4ddf7e3e5499/lxml-4.7.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (6.4 MB)\n",
      "     |████████████████████████████████| 6.4 MB 5.1 MB/s            \n",
      "\u001b[?25hCollecting premailer\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b1/07/4e8d94f94c7d41ca5ddf8a9695ad87b888104e2fd41a35546c1dc9ca74ac/premailer-3.10.0-py2.py3-none-any.whl (19 kB)\n",
      "Requirement already satisfied: openpyxl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from -r requirements.txt (line 14)) (3.0.5)\n",
      "Collecting fasttext==0.9.1\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/10/61/2e01f1397ec533756c1d893c22d9d5ed3fce3a6e4af1976e0d86bb13ea97/fasttext-0.9.1.tar.gz (57 kB)\n",
      "     |████████████████████████████████| 57 kB 12.8 MB/s            \n",
      "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hRequirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (2.2.3)\n",
      "Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (1.15.0)\n",
      "Requirement already satisfied: opencv-python in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (4.1.1.26)\n",
      "Requirement already satisfied: imageio in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (2.6.1)\n",
      "Requirement already satisfied: scipy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (1.6.3)\n",
      "Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->-r requirements.txt (line 3)) (7.1.2)\n",
      "Collecting pybind11>=2.2\n",
      "  Using cached https://pypi.tuna.tsinghua.edu.cn/packages/a8/3b/fc246e1d4c7547a7a07df830128e93c6215e9b93dcb118b2a47a70726153/pybind11-2.8.1-py2.py3-none-any.whl (208 kB)\n",
      "Requirement already satisfied: setuptools>=0.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fasttext==0.9.1->-r requirements.txt (line 15)) (56.2.0)\n",
      "Collecting PyWavelets>=1.1.1\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/a1/9c/564511b6e1c4e1d835ed2d146670436036960d09339a8fa2921fe42dad08/PyWavelets-1.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (6.1 MB)\n",
      "     |████████████████████████████████| 6.1 MB 49.8 MB/s            \n",
      "\u001b[?25hRequirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 2)) (20.9)\n",
      "Collecting tifffile>=2019.7.26\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d8/38/85ae5ed77598ca90558c17a2f79ddaba33173b31cf8d8f545d34d9134f0d/tifffile-2021.11.2-py3-none-any.whl (178 kB)\n",
      "     |████████████████████████████████| 178 kB 56.8 MB/s            \n",
      "\u001b[?25hRequirement already satisfied: networkx>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->-r requirements.txt (line 2)) (2.4)\n",
      "Requirement already satisfied: flask>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.1.1)\n",
      "Requirement already satisfied: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.21.0)\n",
      "Requirement already satisfied: shellcheck-py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (0.7.1.1)\n",
      "Requirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (3.14.0)\n",
      "Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (0.8.53)\n",
      "Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (2.22.0)\n",
      "Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.1.5)\n",
      "Requirement already satisfied: Flask-Babel>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (1.0.0)\n",
      "Requirement already satisfied: flake8>=3.7.9 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->-r requirements.txt (line 8)) (3.8.2)\n",
      "Collecting cssutils\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/24/c4/9db28fe567612896d360ab28ad02ee8ae107d0e92a22db39affd3fba6212/cssutils-2.3.0-py3-none-any.whl (404 kB)\n",
      "     |████████████████████████████████| 404 kB 43.6 MB/s            \n",
      "\u001b[?25hCollecting cssselect\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/3b/d4/3b5c17f00cce85b9a1e6f91096e1cc8e8ede2e1be8e96b87ce1ed09e92c5/cssselect-1.1.0-py2.py3-none-any.whl (16 kB)\n",
      "Requirement already satisfied: cachetools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->-r requirements.txt (line 13)) (4.0.0)\n",
      "Requirement already satisfied: jdcal in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->-r requirements.txt (line 14)) (1.4.1)\n",
      "Requirement already satisfied: et-xmlfile in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->-r requirements.txt (line 14)) (1.0.1)\n",
      "Requirement already satisfied: importlib-metadata in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (0.23)\n",
      "Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (0.6.1)\n",
      "Requirement already satisfied: pycodestyle<2.7.0,>=2.6.0a1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (2.6.0)\n",
      "Requirement already satisfied: pyflakes<2.3.0,>=2.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (2.2.0)\n",
      "Requirement already satisfied: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (7.0)\n",
      "Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (1.1.0)\n",
      "Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (2.11.0)\n",
      "Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (0.16.0)\n",
      "Requirement already satisfied: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->-r requirements.txt (line 8)) (2019.3)\n",
      "Requirement already satisfied: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->-r requirements.txt (line 8)) (2.8.0)\n",
      "Requirement already satisfied: decorator>=4.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from networkx>=2.2->scikit-image->-r requirements.txt (line 2)) (4.4.2)\n",
      "Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging>=20.0->scikit-image->-r requirements.txt (line 2)) (2.4.2)\n",
      "Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->-r requirements.txt (line 8)) (0.18.0)\n",
      "Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->-r requirements.txt (line 8)) (3.9.9)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->-r requirements.txt (line 3)) (2.8.0)\n",
      "Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->-r requirements.txt (line 3)) (0.10.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->-r requirements.txt (line 3)) (1.1.0)\n",
      "Requirement already satisfied: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (1.4.10)\n",
      "Requirement already satisfied: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (1.3.4)\n",
      "Requirement already satisfied: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (16.7.9)\n",
      "Requirement already satisfied: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (5.1.2)\n",
      "Requirement already satisfied: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (0.10.0)\n",
      "Requirement already satisfied: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (1.3.0)\n",
      "Requirement already satisfied: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->-r requirements.txt (line 8)) (2.0.1)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (3.0.4)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (2019.9.11)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (2.8)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->visualdl->-r requirements.txt (line 8)) (1.25.6)\n",
      "Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.1->visualdl->-r requirements.txt (line 8)) (1.1.1)\n",
      "Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata->flake8>=3.7.9->visualdl->-r requirements.txt (line 8)) (3.6.0)\n",
      "Building wheels for collected packages: fasttext, python-Levenshtein\n",
      "  Building wheel for fasttext (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for fasttext: filename=fasttext-0.9.1-cp37-cp37m-linux_x86_64.whl size=2585528 sha256=e9fa045840a7c64a81fede292ad670651aa5fe46380135f54ee64b17d9030594\n",
      "  Stored in directory: /home/aistudio/.cache/pip/wheels/a1/cb/b3/a25a8ce16c1a4ff102c1e40d6eaa4dfc9d5695b92d57331b36\n",
      "  Building wheel for python-Levenshtein (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for python-Levenshtein: filename=python_Levenshtein-0.12.2-cp37-cp37m-linux_x86_64.whl size=171687 sha256=8acb827b504424dfde1dae12b21c2d0ebbc5df8b46d73086c5c14cae7bb908fd\n",
      "  Stored in directory: /home/aistudio/.cache/pip/wheels/38/b9/a4/3729726160fb103833de468adb5ce019b58543ae41d0b0e446\n",
      "Successfully built fasttext python-Levenshtein\n",
      "Installing collected packages: tifffile, PyWavelets, shapely, scikit-image, pybind11, lxml, cssutils, cssselect, python-Levenshtein, pyclipper, premailer, opencv-contrib-python, lmdb, imgaug, fasttext\n",
      "Successfully installed PyWavelets-1.2.0 cssselect-1.1.0 cssutils-2.3.0 fasttext-0.9.1 imgaug-0.4.0 lmdb-1.2.1 lxml-4.7.1 opencv-contrib-python-4.4.0.46 premailer-3.10.0 pybind11-2.8.1 pyclipper-1.3.0.post2 python-Levenshtein-0.12.2 scikit-image-0.19.1 shapely-1.8.0 tifffile-2021.11.2\n",
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Collecting paddlenlp==2.2.1\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/dd/cb/f15e5406718079a201b869b46705ef0882be2ffb6932472e44efc3d16a75/paddlenlp-2.2.1-py3-none-any.whl (1.0 MB)\n",
      "     |████████████████████████████████| 1.0 MB 11.9 MB/s            \n",
      "\u001b[?25hRequirement already satisfied: multiprocess in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (0.70.11.1)\n",
      "Requirement already satisfied: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (1.2.2)\n",
      "Requirement already satisfied: h5py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (2.9.0)\n",
      "Requirement already satisfied: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (4.1.0)\n",
      "Requirement already satisfied: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (0.4.4)\n",
      "Requirement already satisfied: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp==2.2.1) (0.42.1)\n",
      "Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from h5py->paddlenlp==2.2.1) (1.15.0)\n",
      "Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from h5py->paddlenlp==2.2.1) (1.20.3)\n",
      "Requirement already satisfied: dill>=0.3.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from multiprocess->paddlenlp==2.2.1) (0.3.3)\n",
      "Requirement already satisfied: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from seqeval->paddlenlp==2.2.1) (0.24.2)\n",
      "Requirement already satisfied: scipy>=0.19.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.2.1) (1.6.3)\n",
      "Requirement already satisfied: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.2.1) (0.14.1)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp==2.2.1) (2.1.0)\n",
      "Installing collected packages: paddlenlp\n",
      "  Attempting uninstall: paddlenlp\n",
      "    Found existing installation: paddlenlp 2.1.1\n",
      "    Uninstalling paddlenlp-2.1.1:\n",
      "      Successfully uninstalled paddlenlp-2.1.1\n",
      "Successfully installed paddlenlp-2.2.1\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "os.chdir(\"/home/aistudio\")\n",
    "# Download code\n",
    "!git clone https://gitee.com/paddlepaddle/PaddleOCR.git\n",
    "os.chdir(\"/home/aistudio/PaddleOCR\")\n",
    "# Install and run the required whl package\n",
    "!pip install -U pip\n",
    "!pip install -r requirements.txt\n",
    "# This library is needed in VQA tasks\n",
    "!pip install paddlenlp==2.2.1\n",
    "\n",
    "# Import some libraries\n",
    "import cv2\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# 2 Python Inference Based on Paddle Inference\n",
    "\n",
    "## 2.1 Introduction\n",
    "\n",
    "In the project, the inference performance of the model directly affects the cost of the project, so we hope that a model of a trained model can have a faster inference speed. The prediction is directly based on the training engine, and the model contains operators related to training, so **the efficiency is generally low**; and the model needs to be defined, which is **difficult to decouple from the training code**. Paddle Inference came into being. It is the native inference library of Flying Paddle, which acts on the server and the cloud to provide high-performance inference capabilities. Since the ability is directly based on the training operator of the flying paddle, Paddle Inference can universally support all models trained by the flying paddle.\n",
    "\n",
    "Considering that everyone’s usage scenarios are very different, Paddle Inference has carried out in-depth adaptation and optimization for different platforms and different application scenarios, achieving high throughput and low latency, ensuring that the flying paddle model is ready to use on the server side, and is fast deploy.\n",
    "\n",
    "This chapter mainly introduces the prediction inference process of **PP-OCRv2** based on Paddle Inference. For more introduction about Paddle Inference, please refer to: [Paddle Inference Introduction](https://paddleinference.paddlepaddle.org.cn/product_introduction/inference_intro.html).\n",
    "\n",
    "When performing model inference based on Paddle Inference, there are generally the following steps.\n",
    "<div align=\"center\">\n",
    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/18090cbc0733401caeb868f7bf7759436771ba71bce444a8a4ff5dbc5eece03c\"  width = \"600\" />\n",
    "</div>\n",
    "<center>Figure 2: Paddle Inference process</center>\n",
    "\n",
    "The PP-OCRv2 system includes three models: text detection, direction classifier, and text recognition. The following describes the inference process of these three models based on Paddle Inference."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 2.2 PP-OCRv2 Text Detection Model Inference\n",
    "\n",
    "In PaddleOCR, when inference is based on the text detection model, the parameter `image_dir` specifies the path of a single image or image collection, and the parameter `det_model_dir` specifies the path of the detected `inference` model.\n",
    "\n",
    "The following is the latest ultra-lightweight text detection model inference actual combat. For more models and usage methods, please refer to [Text Detection Prediction Tutorial](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/doc/doc_ch/inference_ppocr.md).\n",
    "\n",
    "For more introduction to the hyperparameters of other algorithms, please refer to [PaddleOCR Inference related parameters introduction](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/doc/doc_ch/inference.md#%E5%85%AD%E5%8F%82%E6%95%B0%E8%A7%A3%E9%87%8A).\n",
    "\n",
    "\n",
    "### 2.2.1 Prepare Data and Environment\n",
    "\n",
    "At the very beginning, Paddle and the corresponding dependencies have been installed, and the environment here is ready.\n",
    "\n",
    "The test sample data is under the `doc/imgs` folder, and part of the data is shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "00006737.jpg  00056221.jpg  00111002.jpg  1.jpg\t\tjapan_1.jpg\n",
      "00009282.jpg  00057937.jpg  00207393.jpg  french_0.jpg\tjapan_2.jpg\n",
      "00015504.jpg  00059985.jpg  11.jpg\t  ger_1.jpg\tkorean_1.jpg\n",
      "00018069.jpg  00077949.jpg  12.jpg\t  ger_2.jpg\tmodel_prod_flow_ch.png\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 433396 characters]",
      "text/plain": [
       "<Figure size 1080x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Switch directory\n",
    "os.chdir(\"/home/aistudio/PaddleOCR\")\n",
    "\n",
    "# View data\n",
    "!ls doc/imgs/\n",
    "\n",
    "# Choose 2 images to visualize\n",
    "img1 = cv2.imread(\"doc/imgs/00006737.jpg\")\n",
    "img2 = cv2.imread(\"doc/imgs/00056221.jpg\")\n",
    "plt.figure(figsize=(15, 6))\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.imshow(img1[:,:,::-1])\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.imshow(img2[:,:,::-1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.2 Prepare the inference model\n",
    "\n",
    "Download the inference model, unzip it, and place it under the `inference` directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-12-25 14:55:13--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar\n",
      "Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6\n",
      "Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 3190272 (3.0M) [application/x-tar]\n",
      "Saving to: ‘ch_PP-OCRv2_det_infer.tar’\n",
      "\n",
      "ch_PP-OCRv2_det_inf 100%[===================>]   3.04M  --.-KB/s    in 0.07s   \n",
      "\n",
      "2021-12-25 14:55:13 (42.2 MB/s) - ‘ch_PP-OCRv2_det_infer.tar’ saved [3190272/3190272]\n",
      "\n",
      "inference/ch_PP-OCRv2_det_infer\n",
      "├── [2.2M]  inference.pdiparams\n",
      "├── [ 23K]  inference.pdiparams.info\n",
      "└── [845K]  inference.pdmodel\n",
      "\n",
      "0 directories, 3 files\n"
     ]
    }
   ],
   "source": [
    "# Download model\n",
    "!mkdir inference\n",
    "!cd inference && wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar -O ch_PP-OCRv2_det_infer.tar && tar -xf ch_PP-OCRv2_det_infer.tar\n",
    "!tree -h inference/ch_PP-OCRv2_det_infer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* If you want to export the model you have trained and deploy it using Paddle Inference, you can use the following command to convert the pre-trained model into an inference model using dynamic and static methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-12-25 14:55:21--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_distill_train.tar\n",
      "Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6\n",
      "Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 63830016 (61M) [application/x-tar]\n",
      "Saving to: ‘ch_PP-OCRv2_det_distill_train.tar’\n",
      "\n",
      "ch_PP-OCRv2_det_dis 100%[===================>]  60.87M  81.4MB/s    in 0.7s    \n",
      "\n",
      "2021-12-25 14:55:22 (81.4 MB/s) - ‘ch_PP-OCRv2_det_distill_train.tar’ saved [63830016/63830016]\n",
      "\n",
      "W1225 14:55:24.746377  1078 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1\n",
      "W1225 14:55:24.749907  1078 device_context.cc:465] device: 0, cuDNN Version: 7.6.\n",
      "[2021/12/25 14:55:30] root INFO: load pretrain successful from ch_PP-OCRv2_det_distill_train/best_accuracy\n",
      "[2021/12/25 14:55:31] root INFO: inference model is saved to ./my_model/Teacher/inference\n",
      "[2021/12/25 14:55:33] root INFO: inference model is saved to ./my_model/Student/inference\n",
      "[2021/12/25 14:55:34] root INFO: inference model is saved to ./my_model/Student2/inference\n",
      "my_model\n",
      "├── [4.0K]  Student\n",
      "│   ├── [2.2M]  inference.pdiparams\n",
      "│   ├── [ 23K]  inference.pdiparams.info\n",
      "│   └── [961K]  inference.pdmodel\n",
      "├── [4.0K]  Student2\n",
      "│   ├── [2.2M]  inference.pdiparams\n",
      "│   ├── [ 23K]  inference.pdiparams.info\n",
      "│   └── [962K]  inference.pdmodel\n",
      "└── [4.0K]  Teacher\n",
      "    ├── [ 47M]  inference.pdiparams\n",
      "    ├── [ 12K]  inference.pdiparams.info\n",
      "    └── [568K]  inference.pdmodel\n",
      "\n",
      "3 directories, 9 files\n"
     ]
    }
   ],
   "source": [
    "# Reference Code\n",
    "# https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/export_model.py\n",
    "# Download the pre-trained model\n",
    "!wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_distill_train.tar && tar -xf ch_PP-OCRv2_det_distill_train.tar && rm ch_PP-OCRv2_det_distill_train.tar\n",
    "# Derive inference model\n",
    "!python tools/export_model.py -c configs/det/ch_PP-OCRv2/ch_PP-OCRv2_det_cml.yml \\\n",
    "    -o Global.pretrained_model=\"ch_PP-OCRv2_det_distill_train/best_accuracy\" \\\n",
    "    Global.save_inference_dir=\"./my_model\"\n",
    "# The PP-OCRv2 detection model contains 3 sub-networks: teacher, student, and student 2. Therefore, when exporting, it contains 3 sub-files. In actual inference, one of the student networks can be used for inference.\n",
    "!tree -h my_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2.3 A Preliminary Study of Text Detection Function\n",
    "\n",
    "Let's first take a look at the predicted results of the loading inference model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/25 14:56:01] root INFO: 00018069.jpg\t[[[378, 249], [411, 252], [409, 266], [377, 265]], [[300, 250], [344, 250], [344, 265], [300, 265]], [[233, 250], [268, 250], [268, 265], [233, 265]], [[101, 249], [184, 249], [184, 266], [101, 266]], [[25, 250], [52, 250], [52, 266], [25, 266]], [[375, 233], [409, 233], [409, 247], [375, 247]], [[285, 233], [362, 233], [362, 247], [285, 247]], [[234, 233], [269, 233], [269, 247], [234, 247]], [[24, 234], [53, 234], [53, 247], [24, 247]], [[101, 230], [184, 230], [184, 248], [101, 248]], [[23, 214], [51, 214], [51, 229], [23, 229]], [[378, 213], [410, 213], [410, 228], [378, 228]], [[295, 211], [348, 214], [347, 229], [294, 226]], [[231, 213], [268, 213], [268, 228], [231, 228]], [[100, 212], [185, 212], [185, 229], [100, 229]], [[309, 195], [338, 195], [338, 209], [309, 209]], [[231, 194], [268, 194], [268, 209], [231, 209]], [[23, 194], [51, 194], [51, 210], [23, 210]], [[378, 193], [409, 193], [409, 209], [378, 209]], [[99, 193], [188, 193], [188, 208], [99, 208]], [[376, 175], [413, 175], [413, 190], [376, 190]], [[22, 175], [52, 175], [52, 192], [22, 192]], [[288, 173], [360, 173], [360, 191], [288, 191]], [[237, 174], [264, 174], [264, 190], [237, 190]], [[100, 173], [185, 173], [185, 191], [100, 191]], [[22, 156], [52, 156], [52, 172], [22, 172]], [[284, 154], [358, 154], [358, 172], [284, 172]], [[235, 155], [267, 155], [267, 171], [235, 171]], [[119, 155], [167, 155], [167, 170], [119, 170]], [[378, 138], [410, 135], [412, 152], [379, 155]], [[294, 136], [357, 136], [357, 153], [294, 153]], [[232, 137], [271, 137], [271, 152], [232, 152]], [[118, 137], [166, 137], [166, 151], [118, 151]], [[22, 137], [52, 137], [52, 152], [22, 152]], [[379, 119], [406, 119], [406, 135], [379, 135]], [[229, 118], [268, 118], [268, 134], [229, 134]], [[23, 118], [52, 118], [52, 134], [23, 134]], [[294, 114], [351, 117], [350, 135], [293, 132]], [[117, 113], [168, 116], [167, 135], [116, 132]], [[377, 99], [407, 99], [407, 115], [377, 115]], [[294, 99], [349, 99], [349, 114], [294, 114]], [[223, 99], [272, 99], [272, 114], [223, 114]], [[24, 99], [48, 99], [48, 115], [24, 115]], [[120, 98], [166, 98], [166, 114], [120, 114]], [[364, 80], [425, 80], [425, 95], [364, 95]], [[294, 81], [362, 81], [362, 93], [294, 93]], [[20, 80], [54, 80], [54, 95], [20, 95]], [[234, 79], [267, 79], [267, 94], [234, 94]], [[100, 76], [184, 78], [184, 96], [99, 94]], [[20, 62], [53, 62], [53, 77], [20, 77]], [[366, 62], [420, 59], [421, 75], [366, 77]], [[303, 60], [340, 60], [340, 75], [303, 75]], [[235, 60], [265, 60], [265, 76], [235, 76]], [[103, 60], [182, 60], [182, 75], [103, 75]], [[19, 43], [55, 43], [55, 58], [19, 58]], [[366, 43], [421, 40], [422, 55], [366, 58]], [[305, 42], [341, 42], [341, 57], [305, 57]], [[233, 42], [268, 42], [268, 57], [233, 57]], [[110, 42], [176, 42], [176, 56], [110, 56]], [[22, 24], [53, 24], [53, 40], [22, 40]], [[376, 23], [412, 23], [412, 38], [376, 38]], [[299, 23], [345, 23], [345, 38], [299, 38]], [[233, 23], [268, 23], [268, 39], [233, 39]], [[101, 21], [184, 21], [184, 40], [101, 40]], [[295, 3], [347, 3], [347, 20], [295, 20]], [[373, 1], [414, 1], [414, 20], [373, 20]], [[231, 1], [270, 1], [270, 20], [231, 20]], [[123, 1], [159, 1], [159, 21], [123, 21]], [[18, 1], [54, 1], [54, 21], [18, 21]]]\n",
      "\n",
      "[2021/12/25 14:56:01] root INFO: The predict time of ./doc/imgs/00018069.jpg: 0.6686491966247559\n",
      "[2021/12/25 14:56:01] root INFO: The visualized image saved in ./inference_results/det_res_00018069.jpg\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 402904 characters]",
      "text/plain": [
       "<Figure size 1440x576 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Reference Code\n",
    "# https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/predict_det.py\n",
    "# prediction\n",
    "!python tools/infer/predict_det.py --image_dir=\"./doc/imgs/00018069.jpg\" --det_model_dir=\"./inference/ch_PP-OCRv2_det_infer\" --use_gpu=False\n",
    "\n",
    "# Read the image and display it, display the result\n",
    "plt.figure(figsize=(20, 8))\n",
    "img_ori = cv2.imread(\"./doc/imgs/00018069.jpg\")\n",
    "img_out = cv2.imread(\"./inference_results/det_res_00018069.jpg\")\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.imshow(img_ori[:,:,::-1])\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.imshow(img_out[:,:,::-1])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So how does it work? The following is a detailed explanation of the inference model loading and inference code of PP-OCRv2.\n",
    "\n",
    "First, you need to define the parameter settings as shown below. For more parameter introduction, please refer to: [PaddleOCR inference process parameter introduction](https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.4/doc/doc_ch/inference.md#%E5%85%AD%E5%8F%82%E6%95%B0%E8%A7%A3%E9%87%8A)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "# Reference Code\n",
    "# https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/utility.py\n",
    "import argparse\n",
    "import os\n",
    "import sys\n",
    "import cv2\n",
    "import numpy as np\n",
    "import paddle\n",
    "from PIL import Image, ImageDraw, ImageFont\n",
    "import math\n",
    "from paddle import inference\n",
    "import time\n",
    "from ppocr.utils.logging import get_logger\n",
    "\n",
    "\n",
    "def str2bool(v):\n",
    "    return v.lower() in (\"true\", \"t\", \"1\")\n",
    "\n",
    "\n",
    "def init_args():\n",
    "    parser = argparse.ArgumentParser()\n",
    "    # params for prediction engine\n",
    "    parser.add_argument(\"--use_gpu\", type=str2bool, default=True)\n",
    "    parser.add_argument(\"--ir_optim\", type=str2bool, default=True)\n",
    "    parser.add_argument(\"--use_tensorrt\", type=str2bool, default=False)\n",
    "    parser.add_argument(\"--min_subgraph_size\", type=int, default=15)\n",
    "    parser.add_argument(\"--precision\", type=str, default=\"fp32\")\n",
    "    parser.add_argument(\"--gpu_mem\", type=int, default=500)\n",
    "\n",
    "    # params for text detector\n",
    "    parser.add_argument(\"--image_dir\", type=str)\n",
    "    parser.add_argument(\"--det_algorithm\", type=str, default='DB')\n",
    "    parser.add_argument(\"--det_model_dir\", type=str)\n",
    "    parser.add_argument(\"--det_limit_side_len\", type=float, default=960)\n",
    "    parser.add_argument(\"--det_limit_type\", type=str, default='max')\n",
    "\n",
    "    # DB parmas\n",
    "    parser.add_argument(\"--det_db_thresh\", type=float, default=0.3)\n",
    "    parser.add_argument(\"--det_db_box_thresh\", type=float, default=0.6)\n",
    "    parser.add_argument(\"--det_db_unclip_ratio\", type=float, default=1.5)\n",
    "    parser.add_argument(\"--max_batch_size\", type=int, default=10)\n",
    "    parser.add_argument(\"--use_dilation\", type=str2bool, default=False)\n",
    "    parser.add_argument(\"--det_db_score_mode\", type=str, default=\"fast\")\n",
    "    # EAST parmas\n",
    "    parser.add_argument(\"--det_east_score_thresh\", type=float, default=0.8)\n",
    "    parser.add_argument(\"--det_east_cover_thresh\", type=float, default=0.1)\n",
    "    parser.add_argument(\"--det_east_nms_thresh\", type=float, default=0.2)\n",
    "\n",
    "    # SAST parmas\n",
    "    parser.add_argument(\"--det_sast_score_thresh\", type=float, default=0.5)\n",
    "    parser.add_argument(\"--det_sast_nms_thresh\", type=float, default=0.2)\n",
    "    parser.add_argument(\"--det_sast_polygon\", type=str2bool, default=False)\n",
    "\n",
    "    # PSE parmas\n",
    "    parser.add_argument(\"--det_pse_thresh\", type=float, default=0)\n",
    "    parser.add_argument(\"--det_pse_box_thresh\", type=float, default=0.85)\n",
    "    parser.add_argument(\"--det_pse_min_area\", type=float, default=16)\n",
    "    parser.add_argument(\"--det_pse_box_type\", type=str, default='box')\n",
    "    parser.add_argument(\"--det_pse_scale\", type=int, default=1)\n",
    "\n",
    "    # params for text recognizer\n",
    "    parser.add_argument(\"--rec_algorithm\", type=str, default='CRNN')\n",
    "    parser.add_argument(\"--rec_model_dir\", type=str)\n",
    "    parser.add_argument(\"--rec_image_shape\", type=str, default=\"3, 32, 320\")\n",
    "    parser.add_argument(\"--rec_batch_num\", type=int, default=6)\n",
    "    parser.add_argument(\"--max_text_length\", type=int, default=25)\n",
    "    parser.add_argument(\n",
    "        \"--rec_char_dict_path\",\n",
    "        type=str,\n",
    "        default=\"./ppocr/utils/ppocr_keys_v1.txt\")\n",
    "    parser.add_argument(\"--use_space_char\", type=str2bool, default=True)\n",
    "    parser.add_argument(\n",
    "        \"--vis_font_path\", type=str, default=\"./doc/fonts/simfang.ttf\")\n",
    "    parser.add_argument(\"--drop_score\", type=float, default=0.5)\n",
    "\n",
    "    # params for e2e\n",
    "    parser.add_argument(\"--e2e_algorithm\", type=str, default='PGNet')\n",
    "    parser.add_argument(\"--e2e_model_dir\", type=str)\n",
    "    parser.add_argument(\"--e2e_limit_side_len\", type=float, default=768)\n",
    "    parser.add_argument(\"--e2e_limit_type\", type=str, default='max')\n",
    "\n",
    "    # PGNet parmas\n",
    "    parser.add_argument(\"--e2e_pgnet_score_thresh\", type=float, default=0.5)\n",
    "    parser.add_argument(\n",
    "        \"--e2e_char_dict_path\", type=str, default=\"./ppocr/utils/ic15_dict.txt\")\n",
    "    parser.add_argument(\"--e2e_pgnet_valid_set\", type=str, default='totaltext')\n",
    "    parser.add_argument(\"--e2e_pgnet_mode\", type=str, default='fast')\n",
    "\n",
    "    # params for text classifier\n",
    "    parser.add_argument(\"--use_angle_cls\", type=str2bool, default=False)\n",
    "    parser.add_argument(\"--cls_model_dir\", type=str)\n",
    "    parser.add_argument(\"--cls_image_shape\", type=str, default=\"3, 48, 192\")\n",
    "    parser.add_argument(\"--label_list\", type=list, default=['0', '180'])\n",
    "    parser.add_argument(\"--cls_batch_num\", type=int, default=6)\n",
    "    parser.add_argument(\"--cls_thresh\", type=float, default=0.9)\n",
    "\n",
    "    parser.add_argument(\"--enable_mkldnn\", type=str2bool, default=False)\n",
    "    parser.add_argument(\"--cpu_threads\", type=int, default=10)\n",
    "    parser.add_argument(\"--use_pdserving\", type=str2bool, default=False)\n",
    "    parser.add_argument(\"--warmup\", type=str2bool, default=False)\n",
    "\n",
    "    #\n",
    "    parser.add_argument(\n",
    "        \"--draw_img_save_dir\", type=str, default=\"./inference_results\")\n",
    "    parser.add_argument(\"--save_crop_res\", type=str2bool, default=False)\n",
    "    parser.add_argument(\"--crop_res_save_dir\", type=str, default=\"./output\")\n",
    "\n",
    "    # multi-process\n",
    "    parser.add_argument(\"--use_mp\", type=str2bool, default=False)\n",
    "    parser.add_argument(\"--total_process_num\", type=int, default=1)\n",
    "    parser.add_argument(\"--process_id\", type=int, default=0)\n",
    "\n",
    "    parser.add_argument(\"--benchmark\", type=str2bool, default=False)\n",
    "    parser.add_argument(\"--save_log_path\", type=str, default=\"./log_output/\")\n",
    "\n",
    "    parser.add_argument(\"--show_log\", type=str2bool, default=True)\n",
    "    parser.add_argument(\"--use_onnx\", type=str2bool, default=False)\n",
    "    # It should be noted here that this is added because if you parse directly in the notebook, sys.argv will add the following content at the back, causing the parsing to fail\n",
    "    # '-f', '/home/aistudio/.local/share/jupyter/runtime/kernel-e1221262-c656-4129-896f-1b197b6b782c.json'\n",
    "    parser.add_argument(\"-f\", type=str, default=None)\n",
    "    return parser\n",
    "\n",
    "\n",
    "def parse_args():\n",
    "    parser = init_args()\n",
    "    return parser.parse_args()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at the specific code for text detection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/25 15:10:56] root INFO: 00018069.jpg\t[[[378, 249], [411, 252], [409, 266], [377, 265]], [[300, 250], [344, 250], [344, 265], [300, 265]], [[233, 250], [268, 250], [268, 265], [233, 265]], [[101, 249], [184, 249], [184, 266], [101, 266]], [[25, 250], [52, 250], [52, 266], [25, 266]], [[375, 233], [409, 233], [409, 247], [375, 247]], [[285, 233], [362, 233], [362, 247], [285, 247]], [[234, 233], [269, 233], [269, 247], [234, 247]], [[24, 234], [53, 234], [53, 247], [24, 247]], [[101, 230], [184, 230], [184, 248], [101, 248]], [[23, 214], [51, 214], [51, 229], [23, 229]], [[378, 213], [410, 213], [410, 228], [378, 228]], [[295, 211], [348, 214], [347, 229], [294, 226]], [[231, 213], [268, 213], [268, 228], [231, 228]], [[100, 212], [185, 212], [185, 229], [100, 229]], [[309, 195], [338, 195], [338, 209], [309, 209]], [[231, 194], [268, 194], [268, 209], [231, 209]], [[23, 194], [51, 194], [51, 210], [23, 210]], [[378, 193], [409, 193], [409, 209], [378, 209]], [[99, 193], [188, 193], [188, 208], [99, 208]], [[376, 175], [413, 175], [413, 190], [376, 190]], [[22, 175], [52, 175], [52, 192], [22, 192]], [[288, 173], [360, 173], [360, 191], [288, 191]], [[237, 174], [264, 174], [264, 190], [237, 190]], [[100, 173], [185, 173], [185, 191], [100, 191]], [[22, 156], [52, 156], [52, 172], [22, 172]], [[284, 154], [358, 154], [358, 172], [284, 172]], [[235, 155], [267, 155], [267, 171], [235, 171]], [[119, 155], [167, 155], [167, 170], [119, 170]], [[378, 138], [410, 135], [412, 152], [379, 155]], [[294, 136], [357, 136], [357, 153], [294, 153]], [[232, 137], [271, 137], [271, 152], [232, 152]], [[118, 137], [166, 137], [166, 151], [118, 151]], [[22, 137], [52, 137], [52, 152], [22, 152]], [[379, 119], [406, 119], [406, 135], [379, 135]], [[229, 118], [268, 118], [268, 134], [229, 134]], [[23, 118], [52, 118], [52, 134], [23, 134]], [[294, 114], [351, 117], [350, 135], [293, 132]], [[117, 113], [168, 116], [167, 135], [116, 132]], [[377, 99], [407, 99], [407, 115], [377, 115]], [[294, 99], [349, 99], [349, 114], [294, 114]], [[223, 99], [272, 99], [272, 114], [223, 114]], [[24, 99], [48, 99], [48, 115], [24, 115]], [[120, 98], [166, 98], [166, 114], [120, 114]], [[364, 80], [425, 80], [425, 95], [364, 95]], [[294, 81], [362, 81], [362, 93], [294, 93]], [[20, 80], [54, 80], [54, 95], [20, 95]], [[234, 79], [267, 79], [267, 94], [234, 94]], [[100, 76], [184, 78], [184, 96], [99, 94]], [[20, 62], [53, 62], [53, 77], [20, 77]], [[366, 62], [420, 59], [421, 75], [366, 77]], [[303, 60], [340, 60], [340, 75], [303, 75]], [[235, 60], [265, 60], [265, 76], [235, 76]], [[103, 60], [182, 60], [182, 75], [103, 75]], [[19, 43], [55, 43], [55, 58], [19, 58]], [[366, 43], [421, 40], [422, 55], [366, 58]], [[305, 42], [341, 42], [341, 57], [305, 57]], [[233, 42], [268, 42], [268, 57], [233, 57]], [[110, 42], [176, 42], [176, 56], [110, 56]], [[22, 24], [53, 24], [53, 40], [22, 40]], [[376, 23], [412, 23], [412, 38], [376, 38]], [[299, 23], [345, 23], [345, 38], [299, 38]], [[233, 23], [268, 23], [268, 39], [233, 39]], [[101, 21], [184, 21], [184, 40], [101, 40]], [[295, 3], [347, 3], [347, 20], [295, 20]], [[373, 1], [414, 1], [414, 20], [373, 20]], [[231, 1], [270, 1], [270, 20], [231, 20]], [[123, 1], [159, 1], [159, 21], [123, 21]], [[18, 1], [54, 1], [54, 21], [18, 21]]]\n",
      "\n",
      "[2021/12/25 15:10:56] root INFO: The predict time of ./doc/imgs/00018069.jpg: 0.04716134071350098\n",
      "[2021/12/25 15:10:56] root INFO: The visualized image saved in ./inference_results/det_res_00018069.jpg\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 126796 characters]",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Reference Code\n",
    "# https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/predict_det.py\n",
    "import os\n",
    "import sys\n",
    "import cv2\n",
    "import numpy as np\n",
    "import time\n",
    "\n",
    "import tools.infer.utility as utility\n",
    "from ppocr.utils.logging import get_logger\n",
    "from ppocr.utils.utility import get_image_file_list, check_and_read_gif\n",
    "from ppocr.data import create_operators, transform\n",
    "from ppocr.postprocess import build_post_process\n",
    "import json\n",
    "logger = get_logger()\n",
    "\n",
    "# Text detection class\n",
    "class TextDetector(object):\n",
    "    def __init__(self, args):\n",
    "        self.args = args\n",
    "        self.det_algorithm = args.det_algorithm\n",
    "        pre_process_list = [{\n",
    "            'DetResizeForTest': {\n",
    "                'limit_side_len': args.det_limit_side_len,\n",
    "                'limit_type': args.det_limit_type,\n",
    "            }\n",
    "        }, {\n",
    "            'NormalizeImage': {\n",
    "                'std': [0.229, 0.224, 0.225],\n",
    "                'mean': [0.485, 0.456, 0.406],\n",
    "                'scale': '1./255.',\n",
    "                'order': 'hwc'\n",
    "            }\n",
    "        }, {\n",
    "            'ToCHWImage': None\n",
    "        }, {\n",
    "            'KeepKeys': {\n",
    "                'keep_keys': ['image', 'shape']\n",
    "            }\n",
    "        }]\n",
    "        postprocess_params = {}\n",
    "        if self.det_algorithm == \"DB\":\n",
    "            postprocess_params['name'] = 'DBPostProcess'\n",
    "            postprocess_params[\"thresh\"] = args.det_db_thresh\n",
    "            postprocess_params[\"box_thresh\"] = args.det_db_box_thresh\n",
    "            postprocess_params[\"max_candidates\"] = 1000\n",
    "            postprocess_params[\"unclip_ratio\"] = args.det_db_unclip_ratio\n",
    "            postprocess_params[\"use_dilation\"] = args.use_dilation\n",
    "            postprocess_params[\"score_mode\"] = args.det_db_score_mode\n",
    "        else:\n",
    "            logger.info(\"unknown det_algorithm:{}\".format(self.det_algorithm))\n",
    "            sys.exit(0)\n",
    "        # Initialize the prediction engine\n",
    "        self.predictor, self.input_tensor, self.output_tensors, self.config = utility.create_predictor(\n",
    "            args, 'det', logger)\n",
    "        # Construct preprocessing operator\n",
    "        self.preprocess_op = create_operators(pre_process_list)\n",
    "        # Post-construction operator\n",
    "        self.postprocess_op = build_post_process(postprocess_params)\n",
    "        \n",
    "\n",
    "    def order_points_clockwise(self, pts):\n",
    "        \"\"\"\n",
    "        Refer: https://github.com/jrosebr1/imutils/blob/master/imutils/perspective.py\n",
    "       Sort the detected points clockwise\n",
    "        \"\"\"\n",
    "        xSorted = pts[np.argsort(pts[:, 0]), :]\n",
    "\n",
    "        leftMost = xSorted[:2, :]\n",
    "        rightMost = xSorted[2:, :]\n",
    "\n",
    "        leftMost = leftMost[np.argsort(leftMost[:, 1]), :]\n",
    "        (tl, bl) = leftMost\n",
    "\n",
    "        rightMost = rightMost[np.argsort(rightMost[:, 1]), :]\n",
    "        (tr, br) = rightMost\n",
    "\n",
    "        rect = np.array([tl, tr, br, bl], dtype=\"float32\")\n",
    "        return rect\n",
    "\n",
    "    def clip_det_res(self, points, img_height, img_width):\n",
    "        # Limit the detection results according to the width and height to prevent exceeding the image boundary\n",
    "        for pno in range(points.shape[0]):\n",
    "            points[pno, 0] = int(min(max(points[pno, 0], 0), img_width - 1))\n",
    "            points[pno, 1] = int(min(max(points[pno, 1], 0), img_height - 1))\n",
    "        return points\n",
    "\n",
    "    def filter_tag_det_res(self, dt_boxes, image_shape):\n",
    "        # Remove test results smaller than a certain size\n",
    "        img_height, img_width = image_shape[0:2]\n",
    "        dt_boxes_new = []\n",
    "        for box in dt_boxes:\n",
    "            box = self.order_points_clockwise(box)\n",
    "            box = self.clip_det_res(box, img_height, img_width)\n",
    "            rect_width = int(np.linalg.norm(box[0] - box[1]))\n",
    "            rect_height = int(np.linalg.norm(box[0] - box[3]))\n",
    "            if rect_width <= 3 or rect_height <= 3:\n",
    "                continue\n",
    "            dt_boxes_new.append(box)\n",
    "        dt_boxes = np.array(dt_boxes_new)\n",
    "        return dt_boxes\n",
    "\n",
    "    def filter_tag_det_res_only_clip(self, dt_boxes, image_shape):\n",
    "        # Limit only the boundary of the detection result\n",
    "        img_height, img_width = image_shape[0:2]\n",
    "        dt_boxes_new = []\n",
    "        for box in dt_boxes:\n",
    "            box = self.clip_det_res(box, img_height, img_width)\n",
    "            dt_boxes_new.append(box)\n",
    "        dt_boxes = np.array(dt_boxes_new)\n",
    "        return dt_boxes\n",
    "\n",
    "    def __call__(self, img):\n",
    "        ori_im = img.copy()\n",
    "        data = {'image': img}\n",
    "\n",
    "        st = time.time()\n",
    "        \n",
    "        # Data preprocessing\n",
    "        data = transform(data, self.preprocess_op)\n",
    "        img, shape_list = data\n",
    "        if img is None:\n",
    "            return None, 0\n",
    "        # Extended bs dimension: CHW -> NCHW\n",
    "        img = np.expand_dims(img, axis=0)\n",
    "        shape_list = np.expand_dims(shape_list, axis=0)\n",
    "        img = img.copy()\n",
    "        # Copy the data to the prediction engine\n",
    "        self.input_tensor.copy_from_cpu(img)\n",
    "        # Automatic inference\n",
    "        self.predictor.run()\n",
    "        outputs = []\n",
    "        # Copy the returned result from the prediction engine back to the CPU\n",
    "        for output_tensor in self.output_tensors:\n",
    "            output = output_tensor.copy_to_cpu()\n",
    "            outputs.append(output)\n",
    "\n",
    "        preds = {}\n",
    "        if self.det_algorithm in ['DB', 'PSE']:\n",
    "            preds['maps'] = outputs[0]\n",
    "        else:\n",
    "            raise NotImplementedError\n",
    "\n",
    "        # Post-processing\n",
    "        post_result = self.postprocess_op(preds, shape_list)\n",
    "        dt_boxes = post_result[0]['points']\n",
    "        dt_boxes = self.filter_tag_det_res(dt_boxes, ori_im.shape)\n",
    "        \n",
    "        et = time.time()\n",
    "        return dt_boxes, et - st\n",
    "\n",
    "# Setting parameters\n",
    "args = parse_args()\n",
    "args.det_model_dir = \"./inference/ch_PP-OCRv2_det_infer\"\n",
    "args.image_dir = \"./doc/imgs/00018069.jpg\"\n",
    "\n",
    "# Get a list of pictures\n",
    "image_file_list = get_image_file_list(args.image_dir)\n",
    "# Create a text detector object\n",
    "text_detector = TextDetector(args)\n",
    "\n",
    "count = 0\n",
    "total_time = 0\n",
    "draw_img_save = \"./inference_results\"\n",
    "\n",
    "if not os.path.exists(draw_img_save):\n",
    "    os.makedirs(draw_img_save)\n",
    "save_results = []\n",
    "for image_file in image_file_list:\n",
    "    img = cv2.imread(image_file)\n",
    "    if img is None:\n",
    "        logger.info(\"error in loading image:{}\".format(image_file))\n",
    "        continue\n",
    "    st = time.time()\n",
    "    dt_boxes, _ = text_detector(img)\n",
    "    elapse = time.time() - st\n",
    "    if count > 0:\n",
    "        total_time += elapse\n",
    "    count += 1\n",
    "    save_pred = os.path.basename(image_file) + \"\\t\" + str(\n",
    "        json.dumps(np.array(dt_boxes).astype(np.int32).tolist())) + \"\\n\"\n",
    "    save_results.append(save_pred)\n",
    "    logger.info(save_pred)\n",
    "    logger.info(\"The predict time of {}: {}\".format(image_file, elapse))\n",
    "    src_im = utility.draw_text_det_res(dt_boxes, image_file)\n",
    "    img_name_pure = os.path.split(image_file)[-1]\n",
    "    img_path = os.path.join(draw_img_save,\n",
    "                            \"det_res_{}\".format(img_name_pure))\n",
    "    cv2.imwrite(img_path, src_im)\n",
    "    logger.info(\"The visualized image saved in {}\".format(img_path))\n",
    "    \n",
    "    break\n",
    "\n",
    "with open(os.path.join(draw_img_save, \"det_results.txt\"), 'w') as f:\n",
    "    f.writelines(save_results)\n",
    "    f.close()\n",
    "\n",
    "plt.figure(figsize=(10, 10))\n",
    "plt.imshow(src_im[:, :, ::-1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above completes the complete text detection process.\n",
    "\n",
    "## 2.3 PP-OCRv2 - Directional Classifier Model Inference\n",
    "\n",
    "Similarly, for the direction classifier model, we can also use the following commands to quickly experience its functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-12-25 15:40:13--  https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar\n",
      "Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6\n",
      "Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1454080 (1.4M) [application/x-tar]\n",
      "Saving to: ‘ch_ppocr_mobile_v2.0_cls_infer.tar’\n",
      "\n",
      "ch_ppocr_mobile_v2. 100%[===================>]   1.39M  --.-KB/s    in 0.04s   \n",
      "\n",
      "2021-12-25 15:40:13 (33.6 MB/s) - ‘ch_ppocr_mobile_v2.0_cls_infer.tar’ saved [1454080/1454080]\n",
      "\n",
      "[2021/12/25 15:40:15] root INFO: Predicts of ./doc/imgs_words/ch/word_1.jpg:['0', 0.9998784]\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 62244 characters]",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Download model\n",
    "!cd inference && wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar -O ch_ppocr_mobile_v2.0_cls_infer.tar && tar -xf ch_ppocr_mobile_v2.0_cls_infer.tar\n",
    "# predict\n",
    "!python tools/infer/predict_cls.py \\\n",
    "    --image_dir=\"./doc/imgs_words/ch/word_1.jpg\" \\\n",
    "    --cls_model_dir=\"./inference/ch_ppocr_mobile_v2.0_cls_infer\" \\\n",
    "    --use_gpu=False\n",
    "# Drawing\n",
    "img = cv2.imread(\"./doc/imgs_words/ch/word_1.jpg\")\n",
    "plt.imshow(img[:,:,::-1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The direction of the picture is forward horizontal text, and the prediction result is correct.\n",
    "\n",
    "Specifically, the specific implementation code of the direction classifier is as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/25 15:43:28] root INFO: Predicts of ./doc/imgs_words/ch/word_4.jpg:['0', 0.9999982]\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 31152 characters]",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Reference Code\n",
    "# https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/predict_cls.py\n",
    "import copy\n",
    "\n",
    "# Class implemented by the direction classifier\n",
    "class TextClassifier(object):\n",
    "    def __init__(self, args):\n",
    "        self.cls_image_shape = [int(v) for v in args.cls_image_shape.split(\",\")]\n",
    "        self.cls_batch_num = args.cls_batch_num\n",
    "        self.cls_thresh = args.cls_thresh\n",
    "        postprocess_params = {\n",
    "            'name': 'ClsPostProcess',\n",
    "            \"label_list\": args.label_list,\n",
    "        }\n",
    "        # Post-processing operator\n",
    "        self.postprocess_op = build_post_process(postprocess_params)\n",
    "        # Initialize the prediction engine\n",
    "        self.predictor, self.input_tensor, self.output_tensors, _ = \\\n",
    "            utility.create_predictor(args, 'cls', logger)\n",
    "\n",
    "    # Resize and normalize the image\n",
    "    def resize_norm_img(self, img):\n",
    "        imgC, imgH, imgW = self.cls_image_shape\n",
    "        h = img.shape[0]\n",
    "        w = img.shape[1]\n",
    "        ratio = w / float(h)\n",
    "        if math.ceil(imgH * ratio) > imgW:\n",
    "            resized_w = imgW\n",
    "        else:\n",
    "            resized_w = int(math.ceil(imgH * ratio))\n",
    "        resized_image = cv2.resize(img, (resized_w, imgH))\n",
    "        resized_image = resized_image.astype('float32')\n",
    "        if self.cls_image_shape[0] == 1:\n",
    "            resized_image = resized_image / 255\n",
    "            resized_image = resized_image[np.newaxis, :]\n",
    "        else:\n",
    "            resized_image = resized_image.transpose((2, 0, 1)) / 255\n",
    "        resized_image -= 0.5\n",
    "        resized_image /= 0.5\n",
    "        padding_im = np.zeros((imgC, imgH, imgW), dtype=np.float32)\n",
    "        padding_im[:, :, 0:resized_w] = resized_image\n",
    "        return padding_im\n",
    "\n",
    "    def __call__(self, img_list):\n",
    "        img_list = copy.deepcopy(img_list)\n",
    "        img_num = len(img_list)\n",
    "        # Record aspect ratio\n",
    "        width_list = []\n",
    "        for img in img_list:\n",
    "            width_list.append(img.shape[1] / float(img.shape[0]))\n",
    "        # Sort to speed up the subsequent pre-processing process\n",
    "        indices = np.argsort(np.array(width_list))\n",
    "\n",
    "        cls_res = [['', 0.0]] * img_num\n",
    "        batch_num = self.cls_batch_num\n",
    "        elapse = 0\n",
    "        for beg_img_no in range(0, img_num, batch_num):\n",
    "            end_img_no = min(img_num, beg_img_no + batch_num)\n",
    "            norm_img_batch = []\n",
    "            max_wh_ratio = 0\n",
    "            starttime = time.time()\n",
    "            # Preprocess data, group batch\n",
    "            for ino in range(beg_img_no, end_img_no):\n",
    "                h, w = img_list[indices[ino]].shape[0:2]\n",
    "                wh_ratio = w * 1.0 / h\n",
    "                max_wh_ratio = max(max_wh_ratio, wh_ratio)\n",
    "            for ino in range(beg_img_no, end_img_no):\n",
    "                norm_img = self.resize_norm_img(img_list[indices[ino]])\n",
    "                norm_img = norm_img[np.newaxis, :]\n",
    "                norm_img_batch.append(norm_img)\n",
    "            norm_img_batch = np.concatenate(norm_img_batch)\n",
    "            norm_img_batch = norm_img_batch.copy()\n",
    "            # Copy the data to the prediction engine\n",
    "            self.input_tensor.copy_from_cpu(norm_img_batch)\n",
    "            # Automatic inference process\n",
    "            self.predictor.run()\n",
    "            # Copy the data back to the CPU\n",
    "            prob_out = self.output_tensors[0].copy_to_cpu()\n",
    "            # Post-processing\n",
    "            cls_result = self.postprocess_op(prob_out)\n",
    "            elapse += time.time() - starttime\n",
    "            for rno in range(len(cls_result)):\n",
    "                label, score = cls_result[rno]\n",
    "                cls_res[indices[beg_img_no + rno]] = [label, score]\n",
    "                if '180' in label and score > self.cls_thresh:\n",
    "                    img_list[indices[beg_img_no + rno]] = cv2.rotate(\n",
    "                        img_list[indices[beg_img_no + rno]], 1)\n",
    "        return img_list, cls_res, elapse\n",
    "\n",
    "args = parse_args()\n",
    "args.cls_model_dir = \"./inference/ch_ppocr_mobile_v2.0_cls_infer\"\n",
    "args.image_dir = \"./doc/imgs_words/ch/word_4.jpg\"\n",
    "\n",
    "image_file_list = get_image_file_list(args.image_dir)\n",
    "text_classifier = TextClassifier(args)\n",
    "valid_image_file_list = []\n",
    "img_list = []\n",
    "for image_file in image_file_list:\n",
    "    img = cv2.imread(image_file)\n",
    "    # Rotate the image 180 degrees before prediction\n",
    "    # img = cv2.rotate(img, cv2.ROTATE_180)\n",
    "    if img is None:\n",
    "        logger.info(\"error in loading image:{}\".format(image_file))\n",
    "        continue\n",
    "    valid_image_file_list.append(image_file)\n",
    "    img_list.append(img)\n",
    "img_list, cls_res, predict_time = text_classifier(img_list)\n",
    "for ino in range(len(img_list)):\n",
    "    logger.info(\"Predicts of {}:{}\".format(valid_image_file_list[ino],\n",
    "                                            cls_res[ino]))\n",
    "\n",
    "plt.imshow(img[:,:,::-1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we can also rotate the image 180 degrees to see the classification effect of the direction classifier.\n",
    "\n",
    "The above completes the inference process of the complete direction classifier.\n",
    "\n",
    "\n",
    "## 2.4 PP-OCRv2 - Text Recognition Model Inference\n",
    "\n",
    "For the text recognition model, we can also use the following commands to quickly experience its functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-12-25 15:43:40--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar\n",
      "Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6\n",
      "Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 8875520 (8.5M) [application/x-tar]\n",
      "Saving to: ‘ch_PP-OCRv2_rec_infer.tar’\n",
      "\n",
      "ch_PP-OCRv2_rec_inf 100%[===================>]   8.46M  --.-KB/s    in 0.1s    \n",
      "\n",
      "2021-12-25 15:43:40 (64.5 MB/s) - ‘ch_PP-OCRv2_rec_infer.tar’ saved [8875520/8875520]\n",
      "\n",
      "[2021/12/25 15:43:42] root INFO: Predicts of ./doc/imgs_words/ch/word_4.jpg:('实力活力', 0.9409585)\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 31152 characters]",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Download model\n",
    "!cd inference && wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar -O ch_PP-OCRv2_rec_infer.tar &&  tar -xf ch_PP-OCRv2_rec_infer.tar\n",
    "# predict\n",
    "!python tools/infer/predict_rec.py \\\n",
    "    --image_dir=\"./doc/imgs_words/ch/word_4.jpg\" \\\n",
    "    --rec_model_dir=\"./inference/ch_PP-OCRv2_rec_infer\" \\\n",
    "    --use_gpu=False\n",
    "\n",
    "# Read the image and display\n",
    "img = cv2.imread(\"./doc/imgs_words/ch/word_4.jpg\")\n",
    "plt.imshow(img[:,:,::-1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The specific code for text recognition is shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/25 15:51:51] root INFO: Predicts of ./doc/imgs_words/ch/word_4.jpg:('实力活力', 0.9409561)\n"
     ]
    }
   ],
   "source": [
    "# Reference Code\n",
    "# https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/tools/infer/predict_rec.py\n",
    "class TextRecognizer(object):\n",
    "    def __init__(self, args):\n",
    "        self.rec_image_shape = [int(v) for v in args.rec_image_shape.split(\",\")]\n",
    "        self.rec_batch_num = args.rec_batch_num\n",
    "        self.rec_algorithm = args.rec_algorithm\n",
    "        postprocess_params = {\n",
    "            'name': 'CTCLabelDecode',\n",
    "            \"character_dict_path\": args.rec_char_dict_path,\n",
    "            \"use_space_char\": args.use_space_char\n",
    "        }\n",
    "        # Initialize the prediction engine\n",
    "        self.predictor, self.input_tensor, self.output_tensors, self.config = \\\n",
    "            utility.create_predictor(args, 'rec', logger)\n",
    "        # Post-initialization process\n",
    "        self.postprocess_op = build_post_process(postprocess_params)\n",
    "\n",
    "    # Preprocessing core logic\n",
    "    def resize_norm_img(self, img, max_wh_ratio):\n",
    "        imgC, imgH, imgW = self.rec_image_shape\n",
    "        assert imgC == img.shape[2]\n",
    "        imgW = int((32 * max_wh_ratio))\n",
    "        h, w = img.shape[:2]\n",
    "        ratio = w / float(h)\n",
    "        if math.ceil(imgH * ratio) > imgW:\n",
    "            resized_w = imgW\n",
    "        else:\n",
    "            resized_w = int(math.ceil(imgH * ratio))\n",
    "        resized_image = cv2.resize(img, (resized_w, imgH))\n",
    "        resized_image = resized_image.astype('float32')\n",
    "        # [0, 255] -> [0, 1]\n",
    "        resized_image = resized_image.transpose((2, 0, 1)) / 255\n",
    "        # [0, 1] -> [-0.5, 0.5]\n",
    "        resized_image -= 0.5\n",
    "        # [-0.5, 0.5] -> [-1, 1]\n",
    "        resized_image /= 0.5\n",
    "        padding_im = np.zeros((imgC, imgH, imgW), dtype=np.float32)\n",
    "        padding_im[:, :, 0:resized_w] = resized_image\n",
    "        return padding_im\n",
    "\n",
    "    # Process the image list\n",
    "    def __call__(self, img_list):\n",
    "        img_num = len(img_list)\n",
    "        # Record aspect ratio\n",
    "        width_list = []\n",
    "        for img in img_list:\n",
    "            width_list.append(img.shape[1] / float(img.shape[0]))\n",
    "        # Sort, speed up the process\n",
    "        indices = np.argsort(np.array(width_list))\n",
    "        rec_res = [['', 0.0]] * img_num\n",
    "        batch_num = self.rec_batch_num\n",
    "        st = time.time()\n",
    "        for beg_img_no in range(0, img_num, batch_num):\n",
    "            end_img_no = min(img_num, beg_img_no + batch_num)\n",
    "            norm_img_batch = []\n",
    "            max_wh_ratio = 0\n",
    "            for ino in range(beg_img_no, end_img_no):\n",
    "                h, w = img_list[indices[ino]].shape[0:2]\n",
    "                wh_ratio = w * 1.0 / h\n",
    "                max_wh_ratio = max(max_wh_ratio, wh_ratio)\n",
    "            # Call preprocessing method and group batch\n",
    "            for ino in range(beg_img_no, end_img_no):\n",
    "                norm_img = self.resize_norm_img(img_list[indices[ino]],\n",
    "                                                max_wh_ratio)\n",
    "                norm_img = norm_img[np.newaxis, :]\n",
    "                norm_img_batch.append(norm_img)\n",
    "            norm_img_batch = np.concatenate(norm_img_batch)\n",
    "            norm_img_batch = norm_img_batch.copy()\n",
    "            \n",
    "            # Copy the data to the prediction engine\n",
    "            self.input_tensor.copy_from_cpu(norm_img_batch)\n",
    "            # Automated inference process\n",
    "            self.predictor.run()\n",
    "            outputs = []\n",
    "            # Copy data to CPU\n",
    "            for output_tensor in self.output_tensors:\n",
    "                output = output_tensor.copy_to_cpu()\n",
    "                outputs.append(output)\n",
    "            if len(outputs) != 1:\n",
    "                preds = outputs\n",
    "            else:\n",
    "                preds = outputs[0]\n",
    "            # Post-processing\n",
    "            rec_result = self.postprocess_op(preds)\n",
    "            for rno in range(len(rec_result)):\n",
    "                rec_res[indices[beg_img_no + rno]] = rec_result[rno]\n",
    "        return rec_res, time.time() - st\n",
    "\n",
    "\n",
    "# Define parameters\n",
    "args = parse_args()\n",
    "args.rec_model_dir = \"./inference/ch_PP-OCRv2_rec_infer\"\n",
    "args.image_dir = \"./doc/imgs_words/ch/word_4.jpg\"\n",
    "img_list = []\n",
    "\n",
    "image_file_list = get_image_file_list(args.image_dir)\n",
    "text_recognizer = TextRecognizer(args)\n",
    "valid_image_file_list = []\n",
    "for image_file in image_file_list:\n",
    "    img = cv2.imread(image_file)\n",
    "    if img is None:\n",
    "        logger.info(\"error in loading image:{}\".format(image_file))\n",
    "        continue\n",
    "    valid_image_file_list.append(image_file)\n",
    "    img_list.append(img)\n",
    "rec_res, _ = text_recognizer(img_list)\n",
    "for ino in range(len(img_list)):\n",
    "    logger.info(\"Predicts of {}:{}\".format(valid_image_file_list[ino],\n",
    "                                            rec_res[ino]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.5 PP-OCRv2 - System Series Prediction and Inference\n",
    "\n",
    "The previous content introduced you to the separate inference process of detection, direction classifier, and recognition model in the PP-OCRv2 system. In order to facilitate everyone's end-to-end use, we connected these three modules in series to form the PP-OCRv2 system, and provided the corresponding prediction script.\n",
    "\n",
    "When performing system inference of PP-OCRv2, you need to specify the path of a single image or image collection through the parameter `image_dir`, and the parameters `det_model_dir`, `cls_model_dir` and `rec_model_dir` to specify the `inference` for detection, direction classification, and recognition, respectively Model path. The parameter `use_angle_cls` is used to control whether to enable the direction classification model. `use_mp` indicates whether to use multiple processes. `total_process_num` indicates the number of processes when using multiple processes.\n",
    "\n",
    "\n",
    "Taking the image file `./doc/imgs/00018069.jpg` as an example, the predicted original image is as follows.\n",
    "\n",
    "<div align=\"center\">\n",
    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/5b00f13167014bb1b5b94af65c287509e645043507504ae99f912832e7998c5f\"  width = \"600\" />\n",
    "</div>\n",
    "<center>Figure 3: The original image</center>\n",
    "\n",
    "\n",
    "If you use a direction classifier for tandem prediction, you can use the following command to make predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/25 15:53:46] root DEBUG: dt_boxes num : 69, elapse : 2.86064076423645\n",
      "[2021/12/25 15:53:46] root DEBUG: cls num  : 69, elapse : 0.05832171440124512\n",
      "[2021/12/25 15:53:46] root DEBUG: rec_res num  : 69, elapse : 0.11343073844909668\n",
      "[2021/12/25 15:53:46] root DEBUG: 0  Predict time of ./doc/imgs/00018069.jpg: 3.039s\n",
      "[2021/12/25 15:53:46] root DEBUG: 代号, 0.951\n",
      "[2021/12/25 15:53:46] root DEBUG: 项目, 0.981\n",
      "[2021/12/25 15:53:46] root DEBUG: 结果, 0.994\n",
      "[2021/12/25 15:53:46] root DEBUG: 参考值, 0.999\n",
      "[2021/12/25 15:53:46] root DEBUG: 单位, 0.994\n",
      "[2021/12/25 15:53:46] root DEBUG: 谷丙转氨酶, 0.982\n",
      "[2021/12/25 15:53:46] root DEBUG: 25.6, 0.936\n",
      "[2021/12/25 15:53:46] root DEBUG: 0--40, 0.993\n",
      "[2021/12/25 15:53:46] root DEBUG: ALT, 0.999\n",
      "[2021/12/25 15:53:46] root DEBUG: U/L, 0.961\n",
      "[2021/12/25 15:53:46] root DEBUG: 总胆红素, 0.970\n",
      "[2021/12/25 15:53:46] root DEBUG: 11.2, 0.980\n",
      "[2021/12/25 15:53:46] root DEBUG: TBIL, 0.987\n",
      "[2021/12/25 15:53:46] root DEBUG: <20, 0.976\n",
      "[2021/12/25 15:53:46] root DEBUG: UMOL/L, 0.754\n",
      "[2021/12/25 15:53:46] root DEBUG: 直接胆红素, 0.954\n",
      "[2021/12/25 15:53:46] root DEBUG: 3.3, 0.954\n",
      "[2021/12/25 15:53:46] root DEBUG: DBIL, 0.993\n",
      "[2021/12/25 15:53:46] root DEBUG: 01-7, 0.839\n",
      "[2021/12/25 15:53:46] root DEBUG: UMOL/L, 0.786\n",
      "[2021/12/25 15:53:46] root DEBUG: 间接胆红素, 0.961\n",
      "[2021/12/25 15:53:46] root DEBUG: IBIL, 0.996\n",
      "[2021/12/25 15:53:46] root DEBUG: 7.9, 0.868\n",
      "[2021/12/25 15:53:46] root DEBUG: 1.5--15, 0.887\n",
      "[2021/12/25 15:53:46] root DEBUG: UMOL/L, 0.864\n",
      "[2021/12/25 15:53:46] root DEBUG: TP, 0.927\n",
      "[2021/12/25 15:53:46] root DEBUG: 总蛋白, 0.983\n",
      "[2021/12/25 15:53:46] root DEBUG: +6 89, 0.801\n",
      "[2021/12/25 15:53:46] root DEBUG: 60--80, 0.996\n",
      "[2021/12/25 15:53:46] root DEBUG: 8/L, 0.991\n",
      "[2021/12/25 15:53:46] root DEBUG: 白蛋白, 0.984\n",
      "[2021/12/25 15:53:46] root DEBUG: ALB, 0.998\n",
      "[2021/12/25 15:53:46] root DEBUG: 35.1, 0.956\n",
      "[2021/12/25 15:53:46] root DEBUG: 33--55, 0.898\n",
      "[2021/12/25 15:53:46] root DEBUG: 8/L, 0.962\n",
      "[2021/12/25 15:53:46] root DEBUG: GLO, 0.985\n",
      "[2021/12/25 15:53:46] root DEBUG: 球蛋白, 0.981\n",
      "[2021/12/25 15:53:46] root DEBUG: 23.8, 0.962\n",
      "[2021/12/25 15:53:46] root DEBUG: 20--30, 0.986\n",
      "[2021/12/25 15:53:46] root DEBUG: 8/L, 0.821\n",
      "[2021/12/25 15:53:46] root DEBUG: 白球比, 0.992\n",
      "[2021/12/25 15:53:46] root DEBUG: 1.5, 0.993\n",
      "[2021/12/25 15:53:46] root DEBUG: A/G, 0.978\n",
      "[2021/12/25 15:53:46] root DEBUG: 11.5--2.5, 0.920\n",
      "[2021/12/25 15:53:46] root DEBUG: 碱性磷酸酶, 0.965\n",
      "[2021/12/25 15:53:46] root DEBUG: 93, 0.940\n",
      "[2021/12/25 15:53:46] root DEBUG: ALP, 0.998\n",
      "[2021/12/25 15:53:46] root DEBUG: 15--112, 0.964\n",
      "[2021/12/25 15:53:46] root DEBUG: IU/L, 0.895\n",
      "[2021/12/25 15:53:46] root DEBUG: 谷氨酰转肽酶, 0.902\n",
      "[2021/12/25 15:53:46] root DEBUG: GGT, 0.858\n",
      "[2021/12/25 15:53:46] root DEBUG: 14.3, 0.957\n",
      "[2021/12/25 15:53:46] root DEBUG: <50, 0.938\n",
      "[2021/12/25 15:53:46] root DEBUG: U/L, 0.928\n",
      "[2021/12/25 15:53:46] root DEBUG: 谷草转氨酶, 0.975\n",
      "[2021/12/25 15:53:46] root DEBUG: 16.3, 0.991\n",
      "[2021/12/25 15:53:46] root DEBUG: 8--40, 0.978\n",
      "[2021/12/25 15:53:46] root DEBUG: AST, 0.996\n",
      "[2021/12/25 15:53:46] root DEBUG: U/L, 0.872\n",
      "[2021/12/25 15:53:46] root DEBUG: 乳酸脱氢酶, 0.939\n",
      "[2021/12/25 15:53:46] root DEBUG: 167, 0.984\n",
      "[2021/12/25 15:53:46] root DEBUG: 114--240, 0.980\n",
      "[2021/12/25 15:53:46] root DEBUG: LDH, 0.990\n",
      "[2021/12/25 15:53:46] root DEBUG: U/L, 0.989\n",
      "[2021/12/25 15:53:46] root DEBUG: 腺甘脱氨酶, 0.975\n",
      "[2021/12/25 15:53:46] root DEBUG: ADA, 0.990\n",
      "[2021/12/25 15:53:46] root DEBUG: 12.6, 0.984\n",
      "[2021/12/25 15:53:46] root DEBUG: 4--24, 0.896\n",
      "[2021/12/25 15:53:46] root DEBUG: U/L, 0.877\n",
      "[2021/12/25 15:53:46] root DEBUG: The visualized image saved in ./inference_results/00018069.jpg\n",
      "[2021/12/25 15:53:46] root INFO: The predict total time is 3.074248790740967\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 467416 characters]",
      "text/plain": [
       "<Figure size 1440x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Use the direction classifier to run the PP-OCRv2 system\n",
    "!python tools/infer/predict_system.py \\\n",
    "    --image_dir=\"./doc/imgs/00018069.jpg\" \\\n",
    "    --det_model_dir=\"./inference/ch_PP-OCRv2_det_infer/\" \\\n",
    "    --cls_model_dir=\"./inference/ch_ppocr_mobile_v2.0_cls_infer/\" \\\n",
    "    --rec_model_dir=\"./inference/ch_PP-OCRv2_rec_infer/\" \\\n",
    "    --use_angle_cls=True\n",
    "\n",
    "# Visualization\n",
    "img = cv2.imread(\"./inference_results/00018069.jpg\")\n",
    "plt.figure(figsize=(20, 8))\n",
    "plt.imshow(img[..., ::-1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The visual recognition results are saved in the `./inference_results` folder by default.\n",
    "\n",
    "The detection frame and recognition result are visualized in the image, and the specific recognition file and file reading path information are also printed in the above notebook.\n",
    "  \n",
    "If you want to save the cropped recognition results, you can set the `save_crop_res` parameter to True, and the final result is saved in the `output` directory. Some of the cropped images are shown below. The saved results can be used for subsequent labeling and training of recognition models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/22 00:23:26] root DEBUG: dt_boxes num : 69, elapse : 1.6756877899169922\n",
      "[2021/12/22 00:23:26] root DEBUG: cls num  : 69, elapse : 0.07216906547546387\n",
      "[2021/12/22 00:23:26] root DEBUG: rec_res num  : 69, elapse : 0.10237336158752441\n",
      "[2021/12/22 00:23:26] root DEBUG: 0, ('代号', 0.9509918)\n",
      "[2021/12/22 00:23:26] root DEBUG: 1, ('项目', 0.98095936)\n",
      "[2021/12/22 00:23:26] root DEBUG: 2, ('结果', 0.99443346)\n",
      "[2021/12/22 00:23:26] root DEBUG: 3, ('参考值', 0.99930096)\n",
      "[2021/12/22 00:23:26] root DEBUG: 4, ('单位', 0.99435925)\n",
      "[2021/12/22 00:23:26] root DEBUG: 5, ('谷丙转氨酶', 0.9817702)\n",
      "[2021/12/22 00:23:26] root DEBUG: 6, ('25.6', 0.9356477)\n",
      "[2021/12/22 00:23:26] root DEBUG: 7, ('0--40', 0.99340856)\n",
      "[2021/12/22 00:23:26] root DEBUG: 8, ('ALT', 0.9987676)\n",
      "[2021/12/22 00:23:26] root DEBUG: 9, ('U/L', 0.96127224)\n",
      "[2021/12/22 00:23:26] root DEBUG: 10, ('总胆红素', 0.9696524)\n",
      "[2021/12/22 00:23:26] root DEBUG: 11, ('11.2', 0.98009694)\n",
      "[2021/12/22 00:23:26] root DEBUG: 12, ('TBIL', 0.9866674)\n",
      "[2021/12/22 00:23:26] root DEBUG: 13, ('<20', 0.97562915)\n",
      "[2021/12/22 00:23:26] root DEBUG: 14, ('UMOL/L', 0.753596)\n",
      "[2021/12/22 00:23:26] root DEBUG: 15, ('直接胆红素', 0.95369786)\n",
      "[2021/12/22 00:23:26] root DEBUG: 16, ('3.3', 0.9539189)\n",
      "[2021/12/22 00:23:26] root DEBUG: 17, ('DBIL', 0.992847)\n",
      "[2021/12/22 00:23:26] root DEBUG: 18, ('01-7', 0.8393893)\n",
      "[2021/12/22 00:23:26] root DEBUG: 19, ('UMOL/L', 0.78601867)\n",
      "[2021/12/22 00:23:26] root DEBUG: 20, ('间接胆红素', 0.9609308)\n",
      "[2021/12/22 00:23:26] root DEBUG: 21, ('IBIL', 0.9956264)\n",
      "[2021/12/22 00:23:26] root DEBUG: 22, ('7.9', 0.8684632)\n",
      "[2021/12/22 00:23:26] root DEBUG: 23, ('1.5--15', 0.8868044)\n",
      "[2021/12/22 00:23:26] root DEBUG: 24, ('UMOL/L', 0.86376244)\n",
      "[2021/12/22 00:23:26] root DEBUG: 25, ('TP', 0.92697346)\n",
      "[2021/12/22 00:23:26] root DEBUG: 26, ('总蛋白', 0.9828675)\n",
      "[2021/12/22 00:23:26] root DEBUG: 27, ('+6 89', 0.8005167)\n",
      "[2021/12/22 00:23:26] root DEBUG: 28, ('60--80', 0.9963381)\n",
      "[2021/12/22 00:23:26] root DEBUG: 29, ('8/L', 0.9908847)\n",
      "[2021/12/22 00:23:26] root DEBUG: 30, ('白蛋白', 0.9840359)\n",
      "[2021/12/22 00:23:26] root DEBUG: 31, ('ALB', 0.997701)\n",
      "[2021/12/22 00:23:26] root DEBUG: 32, ('35.1', 0.9561434)\n",
      "[2021/12/22 00:23:26] root DEBUG: 33, ('33--55', 0.89755243)\n",
      "[2021/12/22 00:23:26] root DEBUG: 34, ('8/L', 0.96184856)\n",
      "[2021/12/22 00:23:26] root DEBUG: 35, ('GLO', 0.98490185)\n",
      "[2021/12/22 00:23:26] root DEBUG: 36, ('球蛋白', 0.9813283)\n",
      "[2021/12/22 00:23:26] root DEBUG: 37, ('23.8', 0.9624181)\n",
      "[2021/12/22 00:23:26] root DEBUG: 38, ('20--30', 0.98609096)\n",
      "[2021/12/22 00:23:26] root DEBUG: 39, ('8/L', 0.821223)\n",
      "[2021/12/22 00:23:26] root DEBUG: 40, ('白球比', 0.9924216)\n",
      "[2021/12/22 00:23:26] root DEBUG: 41, ('1.5', 0.9929995)\n",
      "[2021/12/22 00:23:26] root DEBUG: 42, ('A/G', 0.9776535)\n",
      "[2021/12/22 00:23:26] root DEBUG: 43, ('11.5--2.5', 0.92048424)\n",
      "[2021/12/22 00:23:26] root DEBUG: 44, ('碱性磷酸酶', 0.9653308)\n",
      "[2021/12/22 00:23:26] root DEBUG: 45, ('93', 0.9396907)\n",
      "[2021/12/22 00:23:26] root DEBUG: 46, ('ALP', 0.9981435)\n",
      "[2021/12/22 00:23:26] root DEBUG: 47, ('15--112', 0.96404344)\n",
      "[2021/12/22 00:23:26] root DEBUG: 48, ('IU/L', 0.89462817)\n",
      "[2021/12/22 00:23:26] root DEBUG: 49, ('谷氨酰转肽酶', 0.90209746)\n",
      "[2021/12/22 00:23:26] root DEBUG: 50, ('GGT', 0.85804963)\n",
      "[2021/12/22 00:23:26] root DEBUG: 51, ('14.3', 0.9572656)\n",
      "[2021/12/22 00:23:26] root DEBUG: 52, ('<50', 0.9381611)\n",
      "[2021/12/22 00:23:26] root DEBUG: 53, ('U/L', 0.9277277)\n",
      "[2021/12/22 00:23:26] root DEBUG: 54, ('谷草转氨酶', 0.9747366)\n",
      "[2021/12/22 00:23:26] root DEBUG: 55, ('16.3', 0.99106205)\n",
      "[2021/12/22 00:23:26] root DEBUG: 56, ('8--40', 0.9781853)\n",
      "[2021/12/22 00:23:26] root DEBUG: 57, ('AST', 0.99589247)\n",
      "[2021/12/22 00:23:26] root DEBUG: 58, ('U/L', 0.87197894)\n",
      "[2021/12/22 00:23:26] root DEBUG: 59, ('乳酸脱氢酶', 0.938639)\n",
      "[2021/12/22 00:23:26] root DEBUG: 60, ('167', 0.9842878)\n",
      "[2021/12/22 00:23:26] root DEBUG: 61, ('114--240', 0.980333)\n",
      "[2021/12/22 00:23:26] root DEBUG: 62, ('LDH', 0.99032784)\n",
      "[2021/12/22 00:23:26] root DEBUG: 63, ('U/L', 0.9886821)\n",
      "[2021/12/22 00:23:26] root DEBUG: 64, ('腺甘脱氨酶', 0.9749187)\n",
      "[2021/12/22 00:23:26] root DEBUG: 65, ('ADA', 0.99024695)\n",
      "[2021/12/22 00:23:26] root DEBUG: 66, ('12.6', 0.98363185)\n",
      "[2021/12/22 00:23:26] root DEBUG: 67, ('4--24', 0.8959759)\n",
      "[2021/12/22 00:23:26] root DEBUG: 68, ('U/L', 0.8774794)\n",
      "[2021/12/22 00:23:26] root DEBUG: 0  Predict time of ./doc/imgs/00018069.jpg: 1.868s\n",
      "[2021/12/22 00:23:26] root DEBUG: 代号, 0.951\n",
      "[2021/12/22 00:23:26] root DEBUG: 项目, 0.981\n",
      "[2021/12/22 00:23:26] root DEBUG: 结果, 0.994\n",
      "[2021/12/22 00:23:26] root DEBUG: 参考值, 0.999\n",
      "[2021/12/22 00:23:26] root DEBUG: 单位, 0.994\n",
      "[2021/12/22 00:23:26] root DEBUG: 谷丙转氨酶, 0.982\n",
      "[2021/12/22 00:23:26] root DEBUG: 25.6, 0.936\n",
      "[2021/12/22 00:23:26] root DEBUG: 0--40, 0.993\n",
      "[2021/12/22 00:23:26] root DEBUG: ALT, 0.999\n",
      "[2021/12/22 00:23:26] root DEBUG: U/L, 0.961\n",
      "[2021/12/22 00:23:26] root DEBUG: 总胆红素, 0.970\n",
      "[2021/12/22 00:23:26] root DEBUG: 11.2, 0.980\n",
      "[2021/12/22 00:23:26] root DEBUG: TBIL, 0.987\n",
      "[2021/12/22 00:23:26] root DEBUG: <20, 0.976\n",
      "[2021/12/22 00:23:26] root DEBUG: UMOL/L, 0.754\n",
      "[2021/12/22 00:23:26] root DEBUG: 直接胆红素, 0.954\n",
      "[2021/12/22 00:23:26] root DEBUG: 3.3, 0.954\n",
      "[2021/12/22 00:23:26] root DEBUG: DBIL, 0.993\n",
      "[2021/12/22 00:23:26] root DEBUG: 01-7, 0.839\n",
      "[2021/12/22 00:23:26] root DEBUG: UMOL/L, 0.786\n",
      "[2021/12/22 00:23:26] root DEBUG: 间接胆红素, 0.961\n",
      "[2021/12/22 00:23:26] root DEBUG: IBIL, 0.996\n",
      "[2021/12/22 00:23:26] root DEBUG: 7.9, 0.868\n",
      "[2021/12/22 00:23:26] root DEBUG: 1.5--15, 0.887\n",
      "[2021/12/22 00:23:26] root DEBUG: UMOL/L, 0.864\n",
      "[2021/12/22 00:23:26] root DEBUG: TP, 0.927\n",
      "[2021/12/22 00:23:26] root DEBUG: 总蛋白, 0.983\n",
      "[2021/12/22 00:23:26] root DEBUG: +6 89, 0.801\n",
      "[2021/12/22 00:23:26] root DEBUG: 60--80, 0.996\n",
      "[2021/12/22 00:23:26] root DEBUG: 8/L, 0.991\n",
      "[2021/12/22 00:23:26] root DEBUG: 白蛋白, 0.984\n",
      "[2021/12/22 00:23:26] root DEBUG: ALB, 0.998\n",
      "[2021/12/22 00:23:26] root DEBUG: 35.1, 0.956\n",
      "[2021/12/22 00:23:26] root DEBUG: 33--55, 0.898\n",
      "[2021/12/22 00:23:26] root DEBUG: 8/L, 0.962\n",
      "[2021/12/22 00:23:26] root DEBUG: GLO, 0.985\n",
      "[2021/12/22 00:23:26] root DEBUG: 球蛋白, 0.981\n",
      "[2021/12/22 00:23:26] root DEBUG: 23.8, 0.962\n",
      "[2021/12/22 00:23:26] root DEBUG: 20--30, 0.986\n",
      "[2021/12/22 00:23:26] root DEBUG: 8/L, 0.821\n",
      "[2021/12/22 00:23:26] root DEBUG: 白球比, 0.992\n",
      "[2021/12/22 00:23:26] root DEBUG: 1.5, 0.993\n",
      "[2021/12/22 00:23:26] root DEBUG: A/G, 0.978\n",
      "[2021/12/22 00:23:26] root DEBUG: 11.5--2.5, 0.920\n",
      "[2021/12/22 00:23:26] root DEBUG: 碱性磷酸酶, 0.965\n",
      "[2021/12/22 00:23:26] root DEBUG: 93, 0.940\n",
      "[2021/12/22 00:23:26] root DEBUG: ALP, 0.998\n",
      "[2021/12/22 00:23:26] root DEBUG: 15--112, 0.964\n",
      "[2021/12/22 00:23:26] root DEBUG: IU/L, 0.895\n",
      "[2021/12/22 00:23:26] root DEBUG: 谷氨酰转肽酶, 0.902\n",
      "[2021/12/22 00:23:26] root DEBUG: GGT, 0.858\n",
      "[2021/12/22 00:23:26] root DEBUG: 14.3, 0.957\n",
      "[2021/12/22 00:23:26] root DEBUG: <50, 0.938\n",
      "[2021/12/22 00:23:26] root DEBUG: U/L, 0.928\n",
      "[2021/12/22 00:23:26] root DEBUG: 谷草转氨酶, 0.975\n",
      "[2021/12/22 00:23:26] root DEBUG: 16.3, 0.991\n",
      "[2021/12/22 00:23:26] root DEBUG: 8--40, 0.978\n",
      "[2021/12/22 00:23:26] root DEBUG: AST, 0.996\n",
      "[2021/12/22 00:23:26] root DEBUG: U/L, 0.872\n",
      "[2021/12/22 00:23:26] root DEBUG: 乳酸脱氢酶, 0.939\n",
      "[2021/12/22 00:23:26] root DEBUG: 167, 0.984\n",
      "[2021/12/22 00:23:26] root DEBUG: 114--240, 0.980\n",
      "[2021/12/22 00:23:26] root DEBUG: LDH, 0.990\n",
      "[2021/12/22 00:23:26] root DEBUG: U/L, 0.989\n",
      "[2021/12/22 00:23:26] root DEBUG: 腺甘脱氨酶, 0.975\n",
      "[2021/12/22 00:23:26] root DEBUG: ADA, 0.990\n",
      "[2021/12/22 00:23:26] root DEBUG: 12.6, 0.984\n",
      "[2021/12/22 00:23:26] root DEBUG: 4--24, 0.896\n",
      "[2021/12/22 00:23:26] root DEBUG: U/L, 0.877\n",
      "[2021/12/22 00:23:26] root DEBUG: The visualized image saved in ./inference_results/00018069.jpg\n",
      "[2021/12/22 00:23:26] root INFO: The predict total time is 1.9217827320098877\n",
      "mg_crop_0.jpg\tmg_crop_22.jpg\tmg_crop_35.jpg\tmg_crop_48.jpg\tmg_crop_60.jpg\n",
      "mg_crop_10.jpg\tmg_crop_23.jpg\tmg_crop_36.jpg\tmg_crop_49.jpg\tmg_crop_61.jpg\n",
      "mg_crop_11.jpg\tmg_crop_24.jpg\tmg_crop_37.jpg\tmg_crop_4.jpg\tmg_crop_62.jpg\n",
      "mg_crop_12.jpg\tmg_crop_25.jpg\tmg_crop_38.jpg\tmg_crop_50.jpg\tmg_crop_63.jpg\n",
      "mg_crop_13.jpg\tmg_crop_26.jpg\tmg_crop_39.jpg\tmg_crop_51.jpg\tmg_crop_64.jpg\n",
      "mg_crop_14.jpg\tmg_crop_27.jpg\tmg_crop_3.jpg\tmg_crop_52.jpg\tmg_crop_65.jpg\n",
      "mg_crop_15.jpg\tmg_crop_28.jpg\tmg_crop_40.jpg\tmg_crop_53.jpg\tmg_crop_66.jpg\n",
      "mg_crop_16.jpg\tmg_crop_29.jpg\tmg_crop_41.jpg\tmg_crop_54.jpg\tmg_crop_67.jpg\n",
      "mg_crop_17.jpg\tmg_crop_2.jpg\tmg_crop_42.jpg\tmg_crop_55.jpg\tmg_crop_68.jpg\n",
      "mg_crop_18.jpg\tmg_crop_30.jpg\tmg_crop_43.jpg\tmg_crop_56.jpg\tmg_crop_6.jpg\n",
      "mg_crop_19.jpg\tmg_crop_31.jpg\tmg_crop_44.jpg\tmg_crop_57.jpg\tmg_crop_7.jpg\n",
      "mg_crop_1.jpg\tmg_crop_32.jpg\tmg_crop_45.jpg\tmg_crop_58.jpg\tmg_crop_8.jpg\n",
      "mg_crop_20.jpg\tmg_crop_33.jpg\tmg_crop_46.jpg\tmg_crop_59.jpg\tmg_crop_9.jpg\n",
      "mg_crop_21.jpg\tmg_crop_34.jpg\tmg_crop_47.jpg\tmg_crop_5.jpg\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 126636 characters]",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 13332 characters]",
      "text/plain": [
       "<Figure size 1008x288 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Crop the result image of text detection and save it\n",
    "!python tools/infer/predict_system.py \\\n",
    "    --image_dir=\"./doc/imgs/00018069.jpg\" \\\n",
    "    --det_model_dir=\"./inference/ch_PP-OCRv2_det_infer/\" \\\n",
    "    --cls_model_dir=\"./inference/ch_ppocr_mobile_v2.0_cls_infer/\" \\\n",
    "    --rec_model_dir=\"./inference/ch_PP-OCRv2_rec_infer/\" \\\n",
    "    --use_angle_cls=True \\\n",
    "    --save_crop_res=True\n",
    "\n",
    "!ls output\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.imshow(cv2.imread(\"./doc/imgs/00018069.jpg\")[:, :, ::-1])\n",
    "plt.show()\n",
    "plt.figure(figsize=(14, 4))\n",
    "plt.subplot(1,3,1)\n",
    "plt.imshow(cv2.imread(\"output/mg_crop_0.jpg\")[:, :, ::-1])\n",
    "plt.subplot(1,3,2)\n",
    "plt.imshow(cv2.imread(\"output/mg_crop_1.jpg\")[:, :, ::-1])\n",
    "plt.subplot(1,3,3)\n",
    "plt.imshow(cv2.imread(\"output/mg_crop_2.jpg\")[:, :, ::-1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The series prediction is implemented by the `TextSystem` class, and its specific implementation process and function are defined as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/22 00:23:29] root DEBUG: dt_boxes num : 69, elapse : 0.04321765899658203\n",
      "[2021/12/22 00:23:29] root DEBUG: cls num  : 69, elapse : 0.04562950134277344\n",
      "[2021/12/22 00:23:29] root DEBUG: rec_res num  : 69, elapse : 0.09602046012878418\n",
      "[2021/12/22 00:23:29] root DEBUG: 0  Predict time of ./doc/imgs/00018069.jpg: 0.195s\n",
      "[2021/12/22 00:23:29] root DEBUG: 代号, 0.951\n",
      "[2021/12/22 00:23:29] root DEBUG: 项目, 0.981\n",
      "[2021/12/22 00:23:29] root DEBUG: 结果, 0.994\n",
      "[2021/12/22 00:23:29] root DEBUG: 参考值, 0.999\n",
      "[2021/12/22 00:23:29] root DEBUG: 单位, 0.994\n",
      "[2021/12/22 00:23:29] root DEBUG: 谷丙转氨酶, 0.982\n",
      "[2021/12/22 00:23:29] root DEBUG: 25.6, 0.936\n",
      "[2021/12/22 00:23:29] root DEBUG: 0--40, 0.993\n",
      "[2021/12/22 00:23:29] root DEBUG: ALT, 0.999\n",
      "[2021/12/22 00:23:29] root DEBUG: U/L, 0.961\n",
      "[2021/12/22 00:23:29] root DEBUG: 总胆红素, 0.970\n",
      "[2021/12/22 00:23:29] root DEBUG: 11.2, 0.980\n",
      "[2021/12/22 00:23:29] root DEBUG: TBIL, 0.987\n",
      "[2021/12/22 00:23:29] root DEBUG: <20, 0.976\n",
      "[2021/12/22 00:23:29] root DEBUG: UMOL/L, 0.754\n",
      "[2021/12/22 00:23:29] root DEBUG: 直接胆红素, 0.954\n",
      "[2021/12/22 00:23:29] root DEBUG: 3.3, 0.954\n",
      "[2021/12/22 00:23:29] root DEBUG: DBIL, 0.993\n",
      "[2021/12/22 00:23:29] root DEBUG: 01-7, 0.839\n",
      "[2021/12/22 00:23:29] root DEBUG: UMOL/L, 0.786\n",
      "[2021/12/22 00:23:29] root DEBUG: 间接胆红素, 0.961\n",
      "[2021/12/22 00:23:29] root DEBUG: IBIL, 0.996\n",
      "[2021/12/22 00:23:29] root DEBUG: 7.9, 0.868\n",
      "[2021/12/22 00:23:29] root DEBUG: 1.5--15, 0.887\n",
      "[2021/12/22 00:23:29] root DEBUG: UMOL/L, 0.864\n",
      "[2021/12/22 00:23:29] root DEBUG: TP, 0.927\n",
      "[2021/12/22 00:23:29] root DEBUG: 总蛋白, 0.983\n",
      "[2021/12/22 00:23:29] root DEBUG: +6 89, 0.801\n",
      "[2021/12/22 00:23:29] root DEBUG: 60--80, 0.996\n",
      "[2021/12/22 00:23:29] root DEBUG: 8/L, 0.991\n",
      "[2021/12/22 00:23:29] root DEBUG: 白蛋白, 0.984\n",
      "[2021/12/22 00:23:29] root DEBUG: ALB, 0.998\n",
      "[2021/12/22 00:23:29] root DEBUG: 35.1, 0.956\n",
      "[2021/12/22 00:23:29] root DEBUG: 33--55, 0.898\n",
      "[2021/12/22 00:23:29] root DEBUG: 8/L, 0.962\n",
      "[2021/12/22 00:23:29] root DEBUG: GLO, 0.985\n",
      "[2021/12/22 00:23:29] root DEBUG: 球蛋白, 0.981\n",
      "[2021/12/22 00:23:29] root DEBUG: 23.8, 0.962\n",
      "[2021/12/22 00:23:29] root DEBUG: 20--30, 0.986\n",
      "[2021/12/22 00:23:29] root DEBUG: 8/L, 0.821\n",
      "[2021/12/22 00:23:29] root DEBUG: 白球比, 0.992\n",
      "[2021/12/22 00:23:29] root DEBUG: 1.5, 0.993\n",
      "[2021/12/22 00:23:29] root DEBUG: A/G, 0.978\n",
      "[2021/12/22 00:23:29] root DEBUG: 11.5--2.5, 0.920\n",
      "[2021/12/22 00:23:29] root DEBUG: 碱性磷酸酶, 0.965\n",
      "[2021/12/22 00:23:29] root DEBUG: 93, 0.940\n",
      "[2021/12/22 00:23:29] root DEBUG: ALP, 0.998\n",
      "[2021/12/22 00:23:29] root DEBUG: 15--112, 0.964\n",
      "[2021/12/22 00:23:29] root DEBUG: IU/L, 0.895\n",
      "[2021/12/22 00:23:29] root DEBUG: 谷氨酰转肽酶, 0.902\n",
      "[2021/12/22 00:23:29] root DEBUG: GGT, 0.858\n",
      "[2021/12/22 00:23:29] root DEBUG: 14.3, 0.957\n",
      "[2021/12/22 00:23:29] root DEBUG: <50, 0.938\n",
      "[2021/12/22 00:23:29] root DEBUG: U/L, 0.928\n",
      "[2021/12/22 00:23:29] root DEBUG: 谷草转氨酶, 0.975\n",
      "[2021/12/22 00:23:29] root DEBUG: 16.3, 0.991\n",
      "[2021/12/22 00:23:29] root DEBUG: 8--40, 0.978\n",
      "[2021/12/22 00:23:29] root DEBUG: AST, 0.996\n",
      "[2021/12/22 00:23:29] root DEBUG: U/L, 0.872\n",
      "[2021/12/22 00:23:29] root DEBUG: 乳酸脱氢酶, 0.939\n",
      "[2021/12/22 00:23:29] root DEBUG: 167, 0.984\n",
      "[2021/12/22 00:23:29] root DEBUG: 114--240, 0.980\n",
      "[2021/12/22 00:23:29] root DEBUG: LDH, 0.990\n",
      "[2021/12/22 00:23:29] root DEBUG: U/L, 0.989\n",
      "[2021/12/22 00:23:29] root DEBUG: 腺甘脱氨酶, 0.975\n",
      "[2021/12/22 00:23:29] root DEBUG: ADA, 0.990\n",
      "[2021/12/22 00:23:29] root DEBUG: 12.6, 0.984\n",
      "[2021/12/22 00:23:29] root DEBUG: 4--24, 0.896\n",
      "[2021/12/22 00:23:29] root DEBUG: U/L, 0.877\n",
      "[2021/12/22 00:23:29] root DEBUG: The visualized image saved in ./inference_results/00018069.jpg\n",
      "[2021/12/22 00:23:29] root INFO: The predict total time is 0.28207850456237793\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 126636 characters]",
      "text/plain": [
       "<Figure size 576x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 227904 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Reference Code：https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.4/tools/infer/predict_system.py\n",
    "from tools.infer.utility import draw_ocr_box_txt, get_rotate_crop_image\n",
    "from ppocr.utils.utility import get_image_file_list\n",
    "\n",
    "class TextSystem(object):\n",
    "    # Initialization function\n",
    "    def __init__(self, args):\n",
    "        self.args = args\n",
    "        # If you do not want to display log, you can set show_log to False\n",
    "        if not args.show_log:\n",
    "            logger.setLevel(logging.INFO)\n",
    "        # Define text recognition model prediction engine\n",
    "        self.text_detector = TextDetector(args)\n",
    "        # Define text recognition model prediction engine\n",
    "        self.text_recognizer = TextRecognizer(args)\n",
    "        # Whether to use direction classifier\n",
    "        self.use_angle_cls = args.use_angle_cls\n",
    "        # Score threshold, according to the threshold to determine whether the detection and recognition results need to be visualized or returned\n",
    "        self.drop_score = args.drop_score\n",
    "        # Define the direction classifier prediction engine\n",
    "        if self.use_angle_cls:\n",
    "            self.text_classifier = TextClassifier(args)\n",
    "\n",
    "    # Save text detection result image\n",
    "    def draw_crop_rec_res(self, output_dir, img_crop_list, rec_res):\n",
    "        os.makedirs(output_dir, exist_ok=True)\n",
    "        bbox_num = len(img_crop_list)\n",
    "        for bno in range(bbox_num):\n",
    "            cv2.imwrite(\n",
    "                os.path.join(output_dir,\n",
    "                             f\"mg_crop_{bno+self.crop_image_res_index}.jpg\"),\n",
    "                img_crop_list[bno])\n",
    "            logger.debug(f\"{bno}, {rec_res[bno]}\")\n",
    "        self.crop_image_res_index += bbox_num\n",
    "    \n",
    "    # Core prediction function\n",
    "    def __call__(self, img, cls=True):\n",
    "        ori_im = img.copy()\n",
    "        # Get the detection result of the detection text\n",
    "        dt_boxes, elapse = self.text_detector(img)\n",
    "        logger.debug(\"dt_boxes num : {}, elapse : {}\".format(\n",
    "            len(dt_boxes), elapse))\n",
    "        if dt_boxes is None:\n",
    "            return None, None\n",
    "        img_crop_list = []\n",
    "        # Sort the detection boxes, the order is: first from top to bottom, and second from left to right\n",
    "        dt_boxes = sorted_boxes(dt_boxes)\n",
    "        # Perform perspective transformation and correction on detection results\n",
    "        for bno in range(len(dt_boxes)):\n",
    "            tmp_box = copy.deepcopy(dt_boxes[bno])\n",
    "            img_crop = get_rotate_crop_image(ori_im, tmp_box)\n",
    "            img_crop_list.append(img_crop)\n",
    "        # Use the direction classifier to correct the detection result\n",
    "        if self.use_angle_cls and cls:\n",
    "            img_crop_list, angle_list, elapse = self.text_classifier(\n",
    "                img_crop_list)\n",
    "            logger.debug(\"cls num  : {}, elapse : {}\".format(\n",
    "                len(img_crop_list), elapse))\n",
    "        # Get text recognition results\n",
    "        rec_res, elapse = self.text_recognizer(img_crop_list)\n",
    "        logger.debug(\"rec_res num  : {}, elapse : {}\".format(\n",
    "            len(rec_res), elapse))\n",
    "        # Save the corrected text detection image\n",
    "        if self.args.save_crop_res:\n",
    "            self.draw_crop_rec_res(self.args.crop_res_save_dir, img_crop_list,\n",
    "                                   rec_res)\n",
    "        filter_boxes, filter_rec_res = [], []\n",
    "        #Filter the results according to the threshold of the recognition score, if the score is less than the threshold, filter out\n",
    "        for box, rec_reuslt in zip(dt_boxes, rec_res):\n",
    "            text, score = rec_reuslt\n",
    "            if score >= self.drop_score:\n",
    "                filter_boxes.append(box)\n",
    "                filter_rec_res.append(rec_reuslt)\n",
    "        return filter_boxes, filter_rec_res\n",
    "\n",
    "def sorted_boxes(dt_boxes):\n",
    "    # Sort the detection boxes: first from top to bottom, then from left to right\n",
    "    num_boxes = dt_boxes.shape[0]\n",
    "    sorted_boxes = sorted(dt_boxes, key=lambda x: (x[0][1], x[0][0]))\n",
    "    _boxes = list(sorted_boxes)\n",
    "\n",
    "    for i in range(num_boxes - 1):\n",
    "        if abs(_boxes[i + 1][0][1] - _boxes[i][0][1]) < 10 and \\\n",
    "                (_boxes[i + 1][0][0] < _boxes[i][0][0]):\n",
    "            tmp = _boxes[i]\n",
    "            _boxes[i] = _boxes[i + 1]\n",
    "            _boxes[i + 1] = tmp\n",
    "    return _boxes\n",
    "\n",
    "args = parse_args()\n",
    "args.cls_model_dir = \"./inference/ch_ppocr_mobile_v2.0_cls_infer\"\n",
    "args.det_model_dir=\"./inference/ch_PP-OCRv2_det_infer/\"\n",
    "args.rec_model_dir=\"./inference/ch_PP-OCRv2_rec_infer/\"\n",
    "args.image_dir = \"./doc/imgs/00018069.jpg\"\n",
    "args.use_angle_cls=True\n",
    "args.use_gpu=True\n",
    "\n",
    "image_file_list = get_image_file_list(args.image_dir)\n",
    "image_file_list = image_file_list[args.process_id::args.total_process_num]\n",
    "text_sys = TextSystem(args)\n",
    "is_visualize = True\n",
    "font_path = args.vis_font_path\n",
    "drop_score = args.drop_score\n",
    "\n",
    "total_time = 0\n",
    "cpu_mem, gpu_mem, gpu_util = 0, 0, 0\n",
    "_st = time.time()\n",
    "count = 0\n",
    "for idx, image_file in enumerate(image_file_list):\n",
    "    img = cv2.imread(image_file)\n",
    "    if img is None:\n",
    "        logger.debug(\"error in loading image:{}\".format(image_file))\n",
    "        continue\n",
    "    starttime = time.time()\n",
    "    dt_boxes, rec_res = text_sys(img)\n",
    "    elapse = time.time() - starttime\n",
    "    total_time += elapse\n",
    "\n",
    "    logger.debug(\n",
    "        str(idx) + \"  Predict time of %s: %.3fs\" % (image_file, elapse))\n",
    "    for text, score in rec_res:\n",
    "        logger.debug(\"{}, {:.3f}\".format(text, score))\n",
    "\n",
    "    if is_visualize:\n",
    "        image = Image.fromarray(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))\n",
    "        boxes = dt_boxes\n",
    "        txts = [rec_res[i][0] for i in range(len(rec_res))]\n",
    "        scores = [rec_res[i][1] for i in range(len(rec_res))]\n",
    "\n",
    "        draw_img = draw_ocr_box_txt(\n",
    "            image,\n",
    "            boxes,\n",
    "            txts,\n",
    "            scores,\n",
    "            drop_score=drop_score,\n",
    "            font_path=font_path)\n",
    "        draw_img_save_dir = args.draw_img_save_dir\n",
    "        os.makedirs(draw_img_save_dir, exist_ok=True)\n",
    "        cv2.imwrite(\n",
    "            os.path.join(draw_img_save_dir, os.path.basename(image_file)),\n",
    "            draw_img[:, :, ::-1])\n",
    "        logger.debug(\"The visualized image saved in {}\".format(\n",
    "            os.path.join(draw_img_save_dir, os.path.basename(image_file))))\n",
    "\n",
    "logger.info(\"The predict total time is {}\".format(time.time() - _st))\n",
    "\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.imshow(image)\n",
    "plt.show()\n",
    "plt.figure(figsize=(16, 8))\n",
    "plt.imshow(draw_img)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.6 PP-OCRv2 - Use whl Package for Prediction and Inference\n",
    "\n",
    "In order to more conveniently and quickly experience the OCR text detection and recognition model, PaddleOCR provides a whl package based on the Paddle Inference prediction engine, which is convenient for you to install and experience PaddleOCR with one click.\n",
    "\n",
    "### 2.6.1 Install whl Package\n",
    "\n",
    "Use pip to install the whl package of paddleocr, the command is as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Collecting paddleocr==2.3.0.2\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/e1/b6/5486e674ce096667dff247b58bf0fb789c2ce17a10e546c2686a2bb07aec/paddleocr-2.3.0.2-py3-none-any.whl (250 kB)\n",
      "     |████████████████████████████████| 250 kB 13.6 MB/s            \n",
      "\u001b[?25hRequirement already satisfied: imgaug==0.4.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.4.0)\n",
      "Requirement already satisfied: opencv-contrib-python==4.4.0.46 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (4.4.0.46)\n",
      "Requirement already satisfied: python-Levenshtein in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.12.2)\n",
      "Requirement already satisfied: pyclipper in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (1.3.0.post2)\n",
      "Requirement already satisfied: numpy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (1.20.3)\n",
      "Requirement already satisfied: visualdl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (2.2.0)\n",
      "Requirement already satisfied: lmdb in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (1.2.1)\n",
      "Requirement already satisfied: scikit-image in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.19.1)\n",
      "Requirement already satisfied: shapely in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (1.8.0)\n",
      "Requirement already satisfied: cython in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.29)\n",
      "Requirement already satisfied: openpyxl in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (3.0.5)\n",
      "Requirement already satisfied: lxml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (4.7.1)\n",
      "Requirement already satisfied: premailer in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (3.10.0)\n",
      "Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (4.27.0)\n",
      "Requirement already satisfied: fasttext==0.9.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddleocr==2.3.0.2) (0.9.1)\n",
      "Requirement already satisfied: setuptools>=0.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fasttext==0.9.1->paddleocr==2.3.0.2) (56.2.0)\n",
      "Requirement already satisfied: pybind11>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fasttext==0.9.1->paddleocr==2.3.0.2) (2.8.1)\n",
      "Requirement already satisfied: scipy in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (1.6.3)\n",
      "Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (7.1.2)\n",
      "Requirement already satisfied: imageio in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (2.6.1)\n",
      "Requirement already satisfied: matplotlib in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (2.2.3)\n",
      "Requirement already satisfied: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (1.15.0)\n",
      "Requirement already satisfied: opencv-python in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from imgaug==0.4.0->paddleocr==2.3.0.2) (4.1.1.26)\n",
      "Requirement already satisfied: tifffile>=2019.7.26 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->paddleocr==2.3.0.2) (2021.11.2)\n",
      "Requirement already satisfied: networkx>=2.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->paddleocr==2.3.0.2) (2.4)\n",
      "Requirement already satisfied: PyWavelets>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->paddleocr==2.3.0.2) (1.2.0)\n",
      "Requirement already satisfied: packaging>=20.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-image->paddleocr==2.3.0.2) (20.9)\n",
      "Requirement already satisfied: et-xmlfile in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->paddleocr==2.3.0.2) (1.0.1)\n",
      "Requirement already satisfied: jdcal in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from openpyxl->paddleocr==2.3.0.2) (1.4.1)\n",
      "Requirement already satisfied: cssutils in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->paddleocr==2.3.0.2) (2.3.0)\n",
      "Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->paddleocr==2.3.0.2) (2.22.0)\n",
      "Requirement already satisfied: cssselect in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->paddleocr==2.3.0.2) (1.1.0)\n",
      "Requirement already satisfied: cachetools in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from premailer->paddleocr==2.3.0.2) (4.0.0)\n",
      "Requirement already satisfied: pre-commit in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (1.21.0)\n",
      "Requirement already satisfied: bce-python-sdk in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (0.8.53)\n",
      "Requirement already satisfied: shellcheck-py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (0.7.1.1)\n",
      "Requirement already satisfied: pandas in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (1.1.5)\n",
      "Requirement already satisfied: flask>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (1.1.1)\n",
      "Requirement already satisfied: flake8>=3.7.9 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (3.8.2)\n",
      "Requirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (3.14.0)\n",
      "Requirement already satisfied: Flask-Babel>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from visualdl->paddleocr==2.3.0.2) (1.0.0)\n",
      "Requirement already satisfied: importlib-metadata in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (0.23)\n",
      "Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (0.6.1)\n",
      "Requirement already satisfied: pyflakes<2.3.0,>=2.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (2.2.0)\n",
      "Requirement already satisfied: pycodestyle<2.7.0,>=2.6.0a1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (2.6.0)\n",
      "Requirement already satisfied: click>=5.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (7.0)\n",
      "Requirement already satisfied: Jinja2>=2.10.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (2.11.0)\n",
      "Requirement already satisfied: Werkzeug>=0.15 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (0.16.0)\n",
      "Requirement already satisfied: itsdangerous>=0.24 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (1.1.0)\n",
      "Requirement already satisfied: pytz in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->paddleocr==2.3.0.2) (2019.3)\n",
      "Requirement already satisfied: Babel>=2.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Flask-Babel>=1.0.0->visualdl->paddleocr==2.3.0.2) (2.8.0)\n",
      "Requirement already satisfied: decorator>=4.3.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from networkx>=2.2->scikit-image->paddleocr==2.3.0.2) (4.4.2)\n",
      "Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from packaging>=20.0->scikit-image->paddleocr==2.3.0.2) (2.4.2)\n",
      "Requirement already satisfied: future>=0.6.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->paddleocr==2.3.0.2) (0.18.0)\n",
      "Requirement already satisfied: pycryptodome>=3.8.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from bce-python-sdk->visualdl->paddleocr==2.3.0.2) (3.9.9)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->paddleocr==2.3.0.2) (2.8.0)\n",
      "Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->paddleocr==2.3.0.2) (0.10.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from matplotlib->imgaug==0.4.0->paddleocr==2.3.0.2) (1.1.0)\n",
      "Requirement already satisfied: cfgv>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (2.0.1)\n",
      "Requirement already satisfied: toml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (0.10.0)\n",
      "Requirement already satisfied: nodeenv>=0.11.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (1.3.4)\n",
      "Requirement already satisfied: aspy.yaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (1.3.0)\n",
      "Requirement already satisfied: virtualenv>=15.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (16.7.9)\n",
      "Requirement already satisfied: identify>=1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (1.4.10)\n",
      "Requirement already satisfied: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from pre-commit->visualdl->paddleocr==2.3.0.2) (5.1.2)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->premailer->paddleocr==2.3.0.2) (3.0.4)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->premailer->paddleocr==2.3.0.2) (2019.9.11)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->premailer->paddleocr==2.3.0.2) (2.8)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->premailer->paddleocr==2.3.0.2) (1.25.6)\n",
      "Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from Jinja2>=2.10.1->flask>=1.1.1->visualdl->paddleocr==2.3.0.2) (1.1.1)\n",
      "Requirement already satisfied: zipp>=0.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from importlib-metadata->flake8>=3.7.9->visualdl->paddleocr==2.3.0.2) (3.6.0)\n",
      "Installing collected packages: paddleocr\n",
      "Successfully installed paddleocr-2.3.0.2\n"
     ]
    }
   ],
   "source": [
    "!pip install \"paddleocr==2.3.0.2\"\n",
    "\n",
    "# If you want to get the latest features, you can compile and install based on the source code\n",
    "#     python3 setup.py bdist_wheel\n",
    "#     pip3 install dist/paddleocr-x.x.x-py3-none-any.whl # x.x.x Is the version number of paddleocr\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.6.2 Use whl Package Predictive Inference\n",
    "\n",
    "The paddleocr whl package will automatically download the PP-OCRv2 ultra-lightweight model as the default model. It also supports custom model paths, prediction configurations and other parameters. The parameter names are the same as those in the Python prediction based on Paddle Inference.\n",
    "\n",
    "\n",
    "* Perform Testing Separately\n",
    "\n",
    "Run the following code to quickly experience the prediction and effect of the text detection model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/25 16:04:56] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR\n",
      "download https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar to /home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer/ch_PP-OCRv2_det_infer.tar\n",
      "download https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar to /home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer/ch_PP-OCRv2_rec_infer.tar\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 3.19M/3.19M [00:00<00:00, 47.1MiB/s]\n",
      "100%|██████████| 8.88M/8.88M [00:00<00:00, 54.8MiB/s]\n",
      "100%|██████████| 1.45M/1.45M [00:00<00:00, 35.4MiB/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "download https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar to /home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.tar\n",
      "Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/aistudio/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=False, use_dilation=False, use_gpu=False, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)\n",
      "[2021/12/25 16:04:57] root WARNING: Since the angle classifier is not initialized, the angle classifier will not be uesd during the forward process\n",
      "[[27.0, 459.0], [135.0, 459.0], [135.0, 479.0], [27.0, 479.0]]\n",
      "[[29.0, 431.0], [369.0, 431.0], [369.0, 444.0], [29.0, 444.0]]\n",
      "[[26.0, 397.0], [361.0, 397.0], [361.0, 414.0], [26.0, 414.0]]\n",
      "[[369.0, 368.0], [475.0, 368.0], [475.0, 389.0], [369.0, 389.0]]\n",
      "[[28.0, 367.0], [279.0, 367.0], [279.0, 381.0], [28.0, 381.0]]\n",
      "[[27.0, 335.0], [343.0, 335.0], [343.0, 352.0], [27.0, 352.0]]\n",
      "[[26.0, 304.0], [251.0, 304.0], [251.0, 320.0], [26.0, 320.0]]\n",
      "[[24.0, 271.0], [180.0, 269.0], [180.0, 289.0], [24.0, 290.0]]\n",
      "[[24.0, 239.0], [241.0, 238.0], [241.0, 258.0], [24.0, 259.0]]\n",
      "[[413.0, 233.0], [430.0, 233.0], [430.0, 303.0], [413.0, 303.0]]\n",
      "[[25.0, 208.0], [234.0, 208.0], [234.0, 228.0], [25.0, 228.0]]\n",
      "[[25.0, 177.0], [300.0, 177.0], [300.0, 196.0], [25.0, 196.0]]\n",
      "[[26.0, 143.0], [281.0, 144.0], [281.0, 164.0], [26.0, 163.0]]\n",
      "[[28.0, 113.0], [330.0, 113.0], [330.0, 133.0], [28.0, 133.0]]\n",
      "[[27.0, 82.0], [172.0, 82.0], [172.0, 103.0], [27.0, 103.0]]\n",
      "[[28.0, 37.0], [302.0, 39.0], [302.0, 72.0], [28.0, 70.0]]\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 197828 characters]",
      "text/plain": [
       "<Figure size 1080x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from paddleocr import PaddleOCR, draw_ocr\n",
    "\n",
    "ocr = PaddleOCR(use_gpu=False)  # need to run only once to download and load model into memory\n",
    "img_path = '/home/aistudio/PaddleOCR/doc/imgs/11.jpg'\n",
    "result = ocr.ocr(img_path, rec=False)\n",
    "for line in result:\n",
    "    print(line)\n",
    "\n",
    "# show result\n",
    "from PIL import Image\n",
    "\n",
    "image = Image.open(img_path).convert('RGB')\n",
    "im_show = draw_ocr(image, result, txts=None, scores=None, font_path='/home/aistudio/PaddleOCR/doc/fonts/simfang.ttf')\n",
    "plt.figure(figsize=(15, 8))\n",
    "plt.imshow(im_show)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Perform identification separately\n",
    "\n",
    "You can specify `det=False` to run only a single recognition module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/25 16:05:03] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR\n",
      "Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/aistudio/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=False, use_dilation=False, use_gpu=False, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)\n",
      "[2021/12/25 16:05:03] root WARNING: Since the angle classifier is not initialized, the angle classifier will not be uesd during the forward process\n",
      "('韩国小馆', 0.9967349)\n"
     ]
    }
   ],
   "source": [
    "from paddleocr import PaddleOCR\n",
    "\n",
    "ocr = PaddleOCR(use_gpu=False)  # need to run only once to download and load model into memory\n",
    "img_path = '/home/aistudio/PaddleOCR/doc/imgs_words/ch/word_1.jpg'\n",
    "result = ocr.ocr(img_path, det=False)\n",
    "for line in result:\n",
    "    print(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Separately execute the direction classifier\n",
    "\n",
    "You can specify `det=False, rec=False, cls=True` to run only the direction classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/25 16:06:27] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR\n",
      "Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/aistudio/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=True, use_dilation=False, use_gpu=False, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)\n",
      "['0', 0.9998784]\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 62244 characters]",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from paddleocr import PaddleOCR\n",
    "\n",
    "ocr = PaddleOCR(use_angle_cls=True, use_gpu=False)  # need to run only once to download and load model into memory\n",
    "img_path = '/home/aistudio/PaddleOCR/doc/imgs_words/ch/word_1.jpg'\n",
    "result = ocr.ocr(img_path, det=False, rec=False, cls=True)\n",
    "for line in result:\n",
    "    print(line)\n",
    "\n",
    "img = cv2.imread(img_path)\n",
    "plt.imshow(img[...,::-1])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*`Detection + direction classifier + recognition` whole process experience\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2021/12/25 16:07:58] root WARNING: version PP-OCRv2 not support cls models, auto switch to version PP-OCR\n",
      "Namespace(benchmark=False, cls_batch_num=6, cls_image_shape='3, 48, 192', cls_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_thresh=0.9, cpu_threads=10, crop_res_save_dir='./output', det=True, det_algorithm='DB', det_db_box_thresh=0.6, det_db_score_mode='fast', det_db_thresh=0.3, det_db_unclip_ratio=1.5, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_east_score_thresh=0.8, det_limit_side_len=960, det_limit_type='max', det_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/det/ch/ch_PP-OCRv2_det_infer', det_pse_box_thresh=0.85, det_pse_box_type='box', det_pse_min_area=16, det_pse_scale=1, det_pse_thresh=0, det_sast_nms_thresh=0.2, det_sast_polygon=False, det_sast_score_thresh=0.5, draw_img_save_dir='./inference_results', drop_score=0.5, e2e_algorithm='PGNet', e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_limit_side_len=768, e2e_limit_type='max', e2e_model_dir=None, e2e_pgnet_mode='fast', e2e_pgnet_score_thresh=0.5, e2e_pgnet_valid_set='totaltext', enable_mkldnn=False, gpu_mem=500, help='==SUPPRESS==', image_dir=None, ir_optim=True, label_list=['0', '180'], label_map_path='./vqa/labels/labels_ser.txt', lang='ch', layout_path_model='lp://PubLayNet/ppyolov2_r50vd_dcn_365e_publaynet/config', max_batch_size=10, max_seq_length=512, max_text_length=25, min_subgraph_size=15, mode='structure', model_name_or_path=None, ocr_version='PP-OCRv2', output='./output', precision='fp32', process_id=0, rec=True, rec_algorithm='CRNN', rec_batch_num=6, rec_char_dict_path='/home/aistudio/PaddleOCR/ppocr/utils/ppocr_keys_v1.txt', rec_image_shape='3, 32, 320', rec_model_dir='/home/aistudio/.paddleocr/2.3.0.2/ocr/rec/ch/ch_PP-OCRv2_rec_infer', save_crop_res=False, save_log_path='./log_output/', show_log=True, structure_version='STRUCTURE', table_char_dict_path=None, table_char_type='en', table_max_len=488, table_model_dir=None, total_process_num=1, type='ocr', use_angle_cls=True, use_dilation=False, use_gpu=False, use_mp=False, use_onnx=False, use_pdserving=False, use_space_char=True, use_tensorrt=False, vis_font_path='./doc/fonts/simfang.ttf', warmup=False)\n",
      "[2021/12/25 16:08:00] root DEBUG: dt_boxes num : 16, elapse : 1.1848454475402832\n",
      "[2021/12/25 16:08:01] root DEBUG: cls num  : 16, elapse : 0.7081763744354248\n",
      "[2021/12/25 16:08:08] root DEBUG: rec_res num  : 16, elapse : 7.406725883483887\n",
      "[[[28.0, 37.0], [302.0, 39.0], [302.0, 72.0], [28.0, 70.0]], ('纯臻营养护发素', 0.9924613)]\n",
      "[[[27.0, 82.0], [172.0, 82.0], [172.0, 103.0], [27.0, 103.0]], ('产品信息/参数', 0.992365)]\n",
      "[[[28.0, 113.0], [330.0, 113.0], [330.0, 133.0], [28.0, 133.0]], ('（45元/每公斤，100公斤起订）', 0.90023524)]\n",
      "[[[26.0, 143.0], [281.0, 144.0], [281.0, 164.0], [26.0, 163.0]], ('每瓶22元，1000瓶起订）', 0.9793598)]\n",
      "[[[25.0, 177.0], [300.0, 177.0], [300.0, 196.0], [25.0, 196.0]], ('【品牌】：代加工方式/OEMODM', 0.98173994)]\n",
      "[[[25.0, 208.0], [234.0, 208.0], [234.0, 228.0], [25.0, 228.0]], ('【品名】：纯臻营养护发素', 0.9742198)]\n",
      "[[[24.0, 239.0], [241.0, 238.0], [241.0, 258.0], [24.0, 259.0]], ('【产品编号】：YM-X-3011', 0.9428464)]\n",
      "[[[413.0, 233.0], [430.0, 233.0], [430.0, 303.0], [413.0, 303.0]], ('ODM OEM', 0.91271496)]\n",
      "[[[24.0, 271.0], [180.0, 269.0], [180.0, 289.0], [24.0, 290.0]], ('【净含量】：220ml', 0.97186875)]\n",
      "[[[26.0, 304.0], [251.0, 304.0], [251.0, 320.0], [26.0, 320.0]], ('【适用人群】：适合所有肤质', 0.94716364)]\n",
      "[[[27.0, 335.0], [343.0, 335.0], [343.0, 352.0], [27.0, 352.0]], ('（主要成分】：鲸蜡硬脂醇、燕麦B-葡聚', 0.89800084)]\n",
      "[[[28.0, 367.0], [279.0, 367.0], [279.0, 381.0], [28.0, 381.0]], ('糖、椰油酰胺丙基甜菜碱泛醒', 0.8583736)]\n",
      "[[[369.0, 368.0], [475.0, 368.0], [475.0, 389.0], [369.0, 389.0]], ('（成品包材）', 0.971282)]\n",
      "[[[26.0, 397.0], [361.0, 397.0], [361.0, 414.0], [26.0, 414.0]], ('主要功能】：可紧致头发磷层，从而达到', 0.97077614)]\n",
      "[[[29.0, 431.0], [369.0, 431.0], [369.0, 444.0], [29.0, 444.0]], ('即时持久改善头发光泽的效果，给干燥的头', 0.9430727)]\n",
      "[[[27.0, 459.0], [135.0, 459.0], [135.0, 479.0], [27.0, 479.0]], ('发足够的滋养', 0.97956467)]\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 289932 characters]",
      "text/plain": [
       "<Figure size 1080x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from paddleocr import PaddleOCR, draw_ocr\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# PaddleOCRCurrently supports Chinese and English, English, French, German, Korean, Japanese, and can be switched by modifying the lang parameter\n",
    "# The parameters are `ch`, `en`, `french`, `german`, `korean`, `japan`.\n",
    "ocr = PaddleOCR(use_angle_cls=True, lang=\"ch\", use_gpu=False)  # need to run only once to download and load model into memory\n",
    "img_path = '/home/aistudio/PaddleOCR/doc/imgs/11.jpg'\n",
    "result = ocr.ocr(img_path, cls=True)\n",
    "for line in result:\n",
    "    print(line)\n",
    "\n",
    "# show result\n",
    "from PIL import Image\n",
    "\n",
    "image = Image.open(img_path).convert('RGB')\n",
    "boxes = [line[0] for line in result]\n",
    "txts = [line[1][0] for line in result]\n",
    "scores = [line[1][1] for line in result]\n",
    "im_show = draw_ocr(image, boxes, txts, scores, font_path='/home/aistudio/PaddleOCR/doc/fonts/simfang.ttf')\n",
    "plt.figure(figsize=(15, 8))\n",
    "plt.imshow(im_show)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result is a list, each item contains a text box, text and recognition confidence."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3 C++ Inference Based on Paddle Inference\n",
    "\n",
    "In the inference deployment process, the performance of C++ is generally better than that of Python. Therefore, in many actual inference scenarios, C++ will be considered as a development language for inference.\n",
    "\n",
    "The Paddle Inference introduced to you in the previous section also supports the C++ inference process. This section mainly introduces the C++ PP-OCRv2 inference process.\n",
    "\n",
    "When using C++ inference on the PP-OCRv2 system based on Paddle Inference, there are the following steps.\n",
    "\n",
    "(1) Prepare the model\n",
    "\n",
    "(2) Compile the opencv library\n",
    "\n",
    "(3) Get Paddle Inference prediction library\n",
    "\n",
    "(4) Compile PaddleOCR C++ inference code\n",
    "\n",
    "(5) Run PP-OCRv2 system\n",
    "\n",
    "Due to the limitation of the version on AiStudio, the specific demonstration process will not be done here, only the specific process will be introduced to you. It is recommended that you experience the C++ inference process of PP-OCRv2 locally.\n",
    "\n",
    "For more detailed content in this section, please refer to: [PP-OCRv2 C++ Inference Tutorial](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.4/deploy/cpp_infer/readme.md).\n",
    "\n",
    "## 3.1 Preparing the Model\n",
    "\n",
    "Use the following command to prepare the inference model of PP-OCRv2.\n",
    "\n",
    "```bash\n",
    "cd deploy/cpp_infer\n",
    "wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar -O ch_PP-OCRv2_det_infer.tar && tar -xf ch_PP-OCRv2_det_infer.tar\n",
    "wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar -O ch_PP-OCRv2_rec_infer.tar &&  tar -xf ch_PP-OCRv2_rec_infer.tar\n",
    "wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar -O ch_ppocr_mobile_v2.0_cls_infer.tar && tar -xf ch_ppocr_mobile_v2.0_cls_infer.tar\n",
    "```\n",
    "\n",
    "## 3.2 Compile OpenCV Library\n",
    "\n",
    "* First, you need to download the package compiled in the Linux environment from the opencv official website. Take opencv 3.4.7 as an example. The download command is as follows.\n",
    "\n",
    "```bash\n",
    "wget https://paddleocr.bj.bcebos.com/libs/opencv/opencv-3.4.7.tar.gz\n",
    "tar -xf opencv-3.4.7.tar.gz\n",
    "```\n",
    "\n",
    "Finally, you can see the folder `opencv-3.4.7/` in the current directory.\n",
    "\n",
    "* Compile OpenCV, set OpenCV source path (`root_path`) and installation path (`install_path`). Enter the opencv source code path and compile in the following way.\n",
    "\n",
    "```shell\n",
    "root_path=\"your_opencv_root_path\"\n",
    "install_path=${root_path}/opencv3\n",
    "build_dir=${root_path}/build\n",
    "\n",
    "rm -rf ${build_dir}\n",
    "mkdir ${build_dir}\n",
    "cd ${build_dir}\n",
    "\n",
    "cmake .. \\\n",
    "    -DCMAKE_INSTALL_PREFIX=${install_path} \\\n",
    "    -DCMAKE_BUILD_TYPE=Release \\\n",
    "    -DBUILD_SHARED_LIBS=OFF \\\n",
    "    -DWITH_IPP=OFF \\\n",
    "    -DBUILD_IPP_IW=OFF \\\n",
    "    -DWITH_LAPACK=OFF \\\n",
    "    -DWITH_EIGEN=OFF \\\n",
    "    -DCMAKE_INSTALL_LIBDIR=lib64 \\\n",
    "    -DWITH_ZLIB=ON \\\n",
    "    -DBUILD_ZLIB=ON \\\n",
    "    -DWITH_JPEG=ON \\\n",
    "    -DBUILD_JPEG=ON \\\n",
    "    -DWITH_PNG=ON \\\n",
    "    -DBUILD_PNG=ON \\\n",
    "    -DWITH_TIFF=ON \\\n",
    "    -DBUILD_TIFF=ON\n",
    "\n",
    "make -j\n",
    "make install\n",
    "```\n",
    "\n",
    "You can also directly modify the content of `tools/build_opencv.sh`, and then directly run the following command to compile.\n",
    "```shell\n",
    "sh tools/build_opencv.sh\n",
    "```\n",
    "\n",
    "Among them, `root_path` is the downloaded opencv source code path, `install_path` is the opencv installation path, after the completion of `make install`, opencv header files and library files will be generated in this folder for OCR code compilation later.\n",
    "\n",
    "The final file structure under the installation path is shown below.\n",
    "```\n",
    "opencv3/\n",
    "|-- bin\n",
    "|-- include\n",
    "|-- lib\n",
    "|-- lib64\n",
    "|-- share\n",
    "```\n",
    "\n",
    "## 3.3 Get **Paddle Inference** Prediction Library\n",
    "\n",
    "* The Linux prediction library of different cuda versions is provided on [Paddle Inference library official website](https://paddle-inference.readthedocs.io/en/latest/user_guides/download_lib.html). You can choose the appropriate prediction library version according to your own environment on the official website.\n",
    "\n",
    "* After downloading, use the following method to decompress.\n",
    "\n",
    "```bash\n",
    "wget https://paddle-inference-lib.bj.bcebos.com/2.2.1/cxx_c/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.2_cudnn8.1.1_trt7.2.3.4/paddle_inference.tgz -O paddle_inference.tgz\n",
    "tar -xf paddle_inference.tgz\n",
    "```\n",
    "\n",
    "Finally, a subfolder of `paddle_inference/` will be generated in the current folder.\n",
    "\n",
    "## 3.4 Compile the Inference Code of PaddleOCR\n",
    "\n",
    "The compilation command is as follows, among which the addresses of other dependent libraries such as Paddle C++ prediction library and opencv need to be replaced with the actual addresses on your own machine.\n",
    "```shell\n",
    "sh tools/build.sh\n",
    "```\n",
    "* Specifically, you need to modify the environment path in `tools/build.sh`, the relevant content is as follows:\n",
    "\n",
    "```shell\n",
    "OPENCV_DIR=your_opencv_dir\n",
    "LIB_DIR=your_paddle_inference_dir\n",
    "CUDA_LIB_DIR=your_cuda_lib_dir\n",
    "CUDNN_LIB_DIR=/your_cudnn_lib_dir\n",
    "```\n",
    "\n",
    "Among them, `OPENCV_DIR` is the address where opencv is compiled and installed; `LIB_DIR` is the download (`paddle_inference` folder) or compiled Paddle prediction library address (`build/paddle_inference_install_dir` folder); `CUDA_LIB_DIR` is the cuda library file address , In docker it is `/usr/local/cuda/lib64`; `CUDNN_LIB_DIR` is the cudnn library file address, in docker it is `/usr/lib/x86_64-linux-gnu/`. **Note: The above paths are written as absolute paths, do not write relative paths.**\n",
    "\n",
    "* After the compilation is complete, an executable file named `ppocr` will be generated in the `build` folder.\n",
    "\n",
    "## 3.5 Run PP-OCRv2 System\n",
    "\n",
    "Operation mode：  \n",
    "```shell\n",
    "./build/ppocr <mode> [--param1] [--param2] [...]\n",
    "```\n",
    "Among them, `mode` is a required parameter, which means the selected function, and the value range is ['det','rec','system'], which respectively means calling detection, recognition, and detection and recognition series (including direction classifier). The specific commands are as follows:\n",
    "\n",
    "*Run text detection model only\n",
    "\n",
    "```shell\n",
    "./build/ppocr det \\\n",
    "    --det_model_dir=./ch_PP-OCRv2_det_infer/ \\\n",
    "    --image_dir=../../doc/imgs/12.jpg\n",
    "```\n",
    "* Run text detection model only\n",
    "\n",
    "```shell\n",
    "./build/ppocr rec \\\n",
    "    --rec_model_dir=./ch_PP-OCRv2_rec_infer/ \\\n",
    "    --image_dir=../../doc/imgs_words/ch/\n",
    "```\n",
    "* Run PP-OCRv2 system\n",
    "\n",
    "```shell\n",
    "# Do not use direction classifier\n",
    "./build/ppocr system \\\n",
    "    --det_model_dir=./ch_PP-OCRv2_det_infer/ \\\n",
    "    --rec_model_dir=./ch_PP-OCRv2_rec_infer/ \\\n",
    "    --image_dir=../../doc/imgs/12.jpg\n",
    "\n",
    "#  use direction classifier\n",
    "./build/ppocr system \\\n",
    "    --det_model_dir=./ch_PP-OCRv2_det_infer/ \\\n",
    "    --rec_model_dir=./ch_PP-OCRv2_rec_infer/ \\\n",
    "    --use_angle_cls=true \\\n",
    "    --cls_model_dir=./ch_ppocr_mobile_v2.0_cls_infer \\\n",
    "    --image_dir=../../doc/imgs/12.jpg\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4 **Paddle Serving** Service Deployment Practice\n",
    "\n",
    "In Sections 2 and 3, we introduced in detail the PP-OCRv2 system inference based on Paddle Inference, which belongs to offline inference, that is, the code deployed on a specific machine can only be used on this machine and cannot be passed through other machines. Make a visit. Therefore, the demand for model service deployment is derived.\n",
    "\n",
    "Service-oriented deployment refers to deploying the model as a service, and other devices can access the service by sending a request to obtain the inference result of the model service. The service deployment diagram is shown below.\n",
    "\n",
    "<div align=\"center\">\n",
    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/831abb70838e456fb2cec236817b9c5b1a0dff15fff7479e8f8430493c4506ea\"  width = \"400\" />\n",
    "</div>\n",
    "<center>Figure 4: Schematic diagram of service deployment</center>\n",
    "\n",
    "After the model is successfully deployed, different users can obtain inference services by sending network requests through the client.\n",
    "\n",
    "Paddle Serving is a tool created by Flying Paddle to facilitate developers' service deployment. This section mainly introduces the service deployment process of PP-OCRv2 system based on Paddle Serving.\n",
    "\n",
    "## 4.1 Paddle Serving Introduction\n",
    "\n",
    "Paddle Serving is an open source service deployment framework for PaddlePaddle. Its long-term goal is to provide more and more professional, reliable, and easy-to-use services around the last mile of artificial intelligence landing. Paddle Serving currently provides two frameworks, C++ Serving and Python Pipeline. The Python Pipeline framework tends to facilitate the secondary development, and the C++ Serving framework tends to pursue the ultimate performance.\n",
    "\n",
    "When the PP-OCRv2 model is deployed as a service based on Paddle Serving, the process is as follows.\n",
    "\n",
    "<div align=\"center\">\n",
    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/348c446f79a04bca8801b5fc6dd3c5d5207711179d4d4d9db15e8cb4b1bd8810\"  width = \"800\" />\n",
    "</div>\n",
    "<center>Figure 5: Flow chart of PP-OCRv2 system deployment based on Paddle Serving</center>\n",
    "\n",
    "\n",
    "## 4.2 Prepare Inference Data and Deployment Environment\n",
    "\n",
    "The data is consistent with the data used for model inference.\n",
    "\n",
    "To run Paddle Serving, three installation packages of Paddle Serving need to be installed: paddle-serving-server, paddle-serving-client and paddle-serving-app. The commands are as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-12-25 16:25:12--  https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl\n",
      "Resolving paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)... 100.67.200.6\n",
      "Connecting to paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)|100.67.200.6|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 7146257 (6.8M) [application/octet-stream]\n",
      "Saving to: ‘paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl’\n",
      "\n",
      "paddle_serving_serv 100%[===================>]   6.81M  --.-KB/s    in 0.1s    \n",
      "\n",
      "2021-12-25 16:25:12 (57.6 MB/s) - ‘paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl’ saved [7146257/7146257]\n",
      "\n",
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Processing ./paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl\n",
      "Collecting Jinja2==2.11.3\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7e/c2/1eece8c95ddbc9b1aeb64f5783a9e07a286de42191b7204d67b7496ddf35/Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)\n",
      "     |████████████████████████████████| 125 kB 14.9 MB/s            \n",
      "\u001b[?25hCollecting func-timeout\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/b3/0d/bf0567477f7281d9a3926c582bfef21bff7498fc0ffd3e9de21811896a0b/func_timeout-4.3.5.tar.gz (44 kB)\n",
      "     |████████████████████████████████| 44 kB 6.3 MB/s             \n",
      "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25hRequirement already satisfied: six>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (1.15.0)\n",
      "Requirement already satisfied: itsdangerous==1.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (1.1.0)\n",
      "Requirement already satisfied: MarkupSafe==1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (1.1.1)\n",
      "Collecting Werkzeug==1.0.1\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/cc/94/5f7079a0e00bd6863ef8f1da638721e9da21e5bacee597595b318f71d62e/Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)\n",
      "     |████████████████████████████████| 298 kB 20.6 MB/s            \n",
      "\u001b[?25hRequirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (3.14.0)\n",
      "Collecting grpcio<=1.33.2\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/7a/46/d08d8a5d0e0449f541fe9e7a226854019a41a4fa41fd14332e55b0e4394f/grpcio-1.33.2-cp37-cp37m-manylinux2014_x86_64.whl (3.8 MB)\n",
      "     |████████████████████████████████| 3.8 MB 82.7 MB/s            \n",
      "\u001b[?25hCollecting grpcio-tools<=1.33.2\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/77/1e/91eaee901589ebee04c21df2f551502e7ba946bab99338f77a1f8a4237e1/grpcio_tools-1.33.2-cp37-cp37m-manylinux2014_x86_64.whl (2.5 MB)\n",
      "     |████████████████████████████████| 2.5 MB 2.7 MB/s            \n",
      "\u001b[?25hCollecting click==7.1.2\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl (82 kB)\n",
      "     |████████████████████████████████| 82 kB 2.6 MB/s             \n",
      "\u001b[?25hRequirement already satisfied: flask<2.0.0,>=1.1.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (1.1.1)\n",
      "Requirement already satisfied: pyyaml in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-server-gpu==0.7.0.post102) (5.1.2)\n",
      "Building wheels for collected packages: func-timeout\n",
      "  Building wheel for func-timeout (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for func-timeout: filename=func_timeout-4.3.5-py3-none-any.whl size=15077 sha256=787d26c33bea8918d151100cf0cd6c5953b1b065c427075198f1610f8788274d\n",
      "  Stored in directory: /home/aistudio/.cache/pip/wheels/ea/6b/56/b7bcacbd1bd8cd29883e7f7a29cbd98b87b2d789b25bae5563\n",
      "Successfully built func-timeout\n",
      "Installing collected packages: Werkzeug, Jinja2, grpcio, click, grpcio-tools, func-timeout, paddle-serving-server-gpu\n",
      "  Attempting uninstall: Werkzeug\n",
      "    Found existing installation: Werkzeug 0.16.0\n",
      "    Uninstalling Werkzeug-0.16.0:\n",
      "      Successfully uninstalled Werkzeug-0.16.0\n",
      "  Attempting uninstall: Jinja2\n",
      "    Found existing installation: Jinja2 2.11.0\n",
      "    Uninstalling Jinja2-2.11.0:\n",
      "      Successfully uninstalled Jinja2-2.11.0\n",
      "  Attempting uninstall: grpcio\n",
      "    Found existing installation: grpcio 1.35.0\n",
      "    Uninstalling grpcio-1.35.0:\n",
      "      Successfully uninstalled grpcio-1.35.0\n",
      "  Attempting uninstall: click\n",
      "    Found existing installation: Click 7.0\n",
      "    Uninstalling Click-7.0:\n",
      "      Successfully uninstalled Click-7.0\n",
      "ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "blackhole 1.0.1 requires numpy<=1.19.5, but you have numpy 1.20.3 which is incompatible.\n",
      "Successfully installed Jinja2-2.11.3 Werkzeug-1.0.1 click-7.1.2 func-timeout-4.3.5 grpcio-1.33.2 grpcio-tools-1.33.2 paddle-serving-server-gpu-0.7.0.post102\n",
      "--2021-12-25 16:25:19--  https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_client-0.7.0-cp37-none-any.whl\n",
      "Resolving paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)... 100.67.200.6\n",
      "Connecting to paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)|100.67.200.6|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 44311645 (42M) [application/octet-stream]\n",
      "Saving to: ‘paddle_serving_client-0.7.0-cp37-none-any.whl’\n",
      "\n",
      "paddle_serving_clie 100%[===================>]  42.26M  66.9MB/s    in 0.6s    \n",
      "\n",
      "2021-12-25 16:25:20 (66.9 MB/s) - ‘paddle_serving_client-0.7.0-cp37-none-any.whl’ saved [44311645/44311645]\n",
      "\n",
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Processing ./paddle_serving_client-0.7.0-cp37-none-any.whl\n",
      "Requirement already satisfied: six>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (1.15.0)\n",
      "Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (2.22.0)\n",
      "Requirement already satisfied: grpcio-tools<=1.33.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (1.33.2)\n",
      "Requirement already satisfied: grpcio<=1.33.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (1.33.2)\n",
      "Requirement already satisfied: protobuf>=3.11.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (3.14.0)\n",
      "Requirement already satisfied: numpy>=1.12 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-client==0.7.0) (1.20.3)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddle-serving-client==0.7.0) (1.25.6)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddle-serving-client==0.7.0) (2019.9.11)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddle-serving-client==0.7.0) (3.0.4)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->paddle-serving-client==0.7.0) (2.8)\n",
      "Installing collected packages: paddle-serving-client\n",
      "Successfully installed paddle-serving-client-0.7.0\n",
      "--2021-12-25 16:25:23--  https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_app-0.7.0-py3-none-any.whl\n",
      "Resolving paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)... 100.67.200.6\n",
      "Connecting to paddle-serving.bj.bcebos.com (paddle-serving.bj.bcebos.com)|100.67.200.6|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 54130 (53K) [application/octet-stream]\n",
      "Saving to: ‘paddle_serving_app-0.7.0-py3-none-any.whl’\n",
      "\n",
      "paddle_serving_app- 100%[===================>]  52.86K  --.-KB/s    in 0.002s  \n",
      "\n",
      "2021-12-25 16:25:23 (27.6 MB/s) - ‘paddle_serving_app-0.7.0-py3-none-any.whl’ saved [54130/54130]\n",
      "\n",
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
      "Processing ./paddle_serving_app-0.7.0-py3-none-any.whl\n",
      "Collecting sentencepiece<=0.1.92\n",
      "  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/1e/05/8d8d8fc7ef89d36bbdc9b4ff808def3db0dd52658e67f16779eb05ddb558/sentencepiece-0.1.92-cp37-cp37m-manylinux1_x86_64.whl (1.2 MB)\n",
      "     |████████████████████████████████| 1.2 MB 25.9 MB/s            \n",
      "\u001b[?25hRequirement already satisfied: opencv-python<=4.2.0.32 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (4.1.1.26)\n",
      "Requirement already satisfied: six>=1.10.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (1.15.0)\n",
      "Requirement already satisfied: pyclipper in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (1.3.0.post2)\n",
      "Requirement already satisfied: pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (7.1.2)\n",
      "Requirement already satisfied: shapely in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddle-serving-app==0.7.0) (1.8.0)\n",
      "Requirement already satisfied: numpy>=1.14.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from opencv-python<=4.2.0.32->paddle-serving-app==0.7.0) (1.20.3)\n",
      "Installing collected packages: sentencepiece, paddle-serving-app\n",
      "  Attempting uninstall: sentencepiece\n",
      "    Found existing installation: sentencepiece 0.1.96\n",
      "    Uninstalling sentencepiece-0.1.96:\n",
      "      Successfully uninstalled sentencepiece-0.1.96\n",
      "Successfully installed paddle-serving-app-0.7.0 sentencepiece-0.1.92\n"
     ]
    }
   ],
   "source": [
    "!wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl\n",
    "!pip install paddle_serving_server_gpu-0.7.0.post102-py3-none-any.whl\n",
    "\n",
    "!wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_client-0.7.0-cp37-none-any.whl\n",
    "!pip install paddle_serving_client-0.7.0-cp37-none-any.whl\n",
    "\n",
    "!wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_app-0.7.0-py3-none-any.whl\n",
    "!pip install paddle_serving_app-0.7.0-py3-none-any.whl\n",
    "\n",
    "!rm ./*.whl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3 Preparing to deploy the model\n",
    "\n",
    "When deploying a model as a service, the inference model needs to be transformed into a model for user service deployment.\n",
    "\n",
    "First run the following command to download the inference model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-12-25 16:25:32--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar\n",
      "Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6\n",
      "Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 3190272 (3.0M) [application/x-tar]\n",
      "Saving to: ‘ch_PP-OCRv2_det_infer.tar’\n",
      "\n",
      "ch_PP-OCRv2_det_inf 100%[===================>]   3.04M  --.-KB/s    in 0.09s   \n",
      "\n",
      "2021-12-25 16:25:32 (35.0 MB/s) - ‘ch_PP-OCRv2_det_infer.tar’ saved [3190272/3190272]\n",
      "\n",
      "--2021-12-25 16:25:33--  https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar\n",
      "Resolving paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)... 100.67.200.6\n",
      "Connecting to paddleocr.bj.bcebos.com (paddleocr.bj.bcebos.com)|100.67.200.6|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 8875520 (8.5M) [application/x-tar]\n",
      "Saving to: ‘ch_PP-OCRv2_rec_infer.tar’\n",
      "\n",
      "ch_PP-OCRv2_rec_inf 100%[===================>]   8.46M  --.-KB/s    in 0.1s    \n",
      "\n",
      "2021-12-25 16:25:33 (69.0 MB/s) - ‘ch_PP-OCRv2_rec_infer.tar’ saved [8875520/8875520]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "os.chdir(\"/home/aistudio/PaddleOCR/deploy/pdserving/\")\n",
    "\n",
    "# Download and unzip the OCR text detection model\n",
    "!wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_det_infer.tar -O ch_PP-OCRv2_det_infer.tar && tar -xf ch_PP-OCRv2_det_infer.tar && rm ch_PP-OCRv2_det_infer.tar\n",
    "# Download and unzip the OCR text detection model\n",
    "!wget https://paddleocr.bj.bcebos.com/PP-OCRv2/chinese/ch_PP-OCRv2_rec_infer.tar -O ch_PP-OCRv2_rec_infer.tar &&  tar -xf ch_PP-OCRv2_rec_infer.tar && rm ch_PP-OCRv2_rec_infer.tar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the following command to convert the model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ppocrv2_det_client\n",
      "├── [ 296]  serving_client_conf.prototxt\n",
      "└── [  98]  serving_client_conf.stream.prototxt\n",
      "ppocrv2_rec_client\n",
      "├── [ 284]  serving_client_conf.prototxt\n",
      "└── [  93]  serving_client_conf.stream.prototxt\n",
      "ppocrv2_det_serving\n",
      "├── [2.2M]  inference.pdiparams\n",
      "├── [842K]  inference.pdmodel\n",
      "├── [ 296]  serving_server_conf.prototxt\n",
      "└── [  98]  serving_server_conf.stream.prototxt\n",
      "ppocrv2_rec_serving\n",
      "├── [7.9M]  inference.pdiparams\n",
      "├── [527K]  inference.pdmodel\n",
      "├── [ 284]  serving_server_conf.prototxt\n",
      "└── [  93]  serving_server_conf.stream.prototxt\n",
      "\n",
      "0 directories, 12 files\n"
     ]
    }
   ],
   "source": [
    "# Conversion detection model\n",
    "!python -m paddle_serving_client.convert --dirname ./ch_PP-OCRv2_det_infer/ \\\n",
    "                                         --model_filename inference.pdmodel          \\\n",
    "                                         --params_filename inference.pdiparams       \\\n",
    "                                         --serving_server ./ppocrv2_det_serving/ \\\n",
    "                                         --serving_client ./ppocrv2_det_client/\n",
    "\n",
    "# Conversion recognition model\n",
    "!python -m paddle_serving_client.convert --dirname ./ch_PP-OCRv2_rec_infer/ \\\n",
    "                                         --model_filename inference.pdmodel          \\\n",
    "                                         --params_filename inference.pdiparams       \\\n",
    "                                         --serving_server ./ppocrv2_rec_serving/  \\\n",
    "                                         --serving_client ./ppocrv2_rec_client/\n",
    "\n",
    "# View folder\n",
    "!tree -h *_client *_serving"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the detection model is converted, there will be additional folders of `ppocrv2_det_mobile_serving` and `ppocrv2_det_mobile_client` in the current folder, with the following format:\n",
    "\n",
    "```\n",
    "|- ppocrv2_det_mobile_serving/\n",
    "  |- __model__  \n",
    "  |- __params__\n",
    "  |- serving_server_conf.prototxt  \n",
    "  |- serving_server_conf.stream.prototxt\n",
    "\n",
    "|- ppocrv2_det_mobile_client\n",
    "  |- serving_client_conf.prototxt  \n",
    "  |- serving_client_conf.stream.prototxt\n",
    "\n",
    "```\n",
    "\n",
    "The recognition model is the same.\n",
    "## 4.4 Paddle Serving pipeline deployment\n",
    "\n",
    "**Note**: Modify the two `model_config` fields in the PaddleOCR/deploy/pdserving/config.yml file to ppocrv2_det_mobile_serving and ppocrv2_rec_mobile_serving, corresponding to the folder for model conversion.\n",
    "\n",
    "     The pdserving directory contains the code to start the pipeline service and send prediction requests, including:\n",
    "    ```\n",
    "    __init__.py\n",
    "    config.yml            # Start the service configuration file\n",
    "    ocr_reader.py         # OCR model pre-processing and post-processing code implementation\n",
    "    pipeline_http_client.py   # Script to send prediction request\n",
    "    web_service.py        # Start the server-side script\n",
    "    ```\n",
    "\n",
    "### 4.4.1 Start the service\n",
    "\n",
    "Run the following command to start the service:\n",
    "\n",
    "**Open a new terminal and run the following command to start the service**\n",
    "\n",
    "    ```\n",
    "    # Start the service and save the running log inweb_serving_log.txt\n",
    "    cd PaddleOCR/deploy/pdserving/\n",
    "    nohup python web_service.py &>web_serving_log.txt &\n",
    "    ```\n",
    "After the service is successfully started, a log similar to the following will be printed in web_serving_log.txt\n",
    "<div align=\"center\">\n",
    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/58b32c0158f34a739f9d4b5246b68575519543c786044c19849ab43246f1b482\"  width = \"800\" />\n",
    "</div>\n",
    "<center>Figure 6: Server log</center>\n",
    "\n",
    "\n",
    "### 4.4.2 Send service request:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['土地整治与土壤修复研究中心', '华南农业大学|东图']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['纯臻营养护发素', '产品信息/参数', '（45元/每公斤，100公斤起订', '每瓶22元，1000瓶起订）', '【品牌】：代加工方式/OEMODM', '【品名】：纯臻营养护发素', '【产品编号】：YM-X-3011', 'ODMOEM', '（净含量】：220ml', '【适用人群）：适合所有肤质', '［主要成分】：鲸蜡硬脂醇、燕麦B-葡聚', '糖、椰油酰胺内基甜菜碱、泛醒', '（成品包材）', '主要功能】：可紧致头发磷层，从而达到', '即时持久改善头发光泽的效果，给干燥的头', '发足够的滋养']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['Mairiedu|er|农', '*|Palais du LOUVRE', '公 LES ARTS DECORATIFS', '＊ Musée du LOUVRE', 'Theatre', 'du PALAIS-ROYAL L&']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['2', '证#', '', '##', '孔叫叫E2号动']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['合格证', '品名：纯棉单面背心套', '执行标准：FZ/T73025-2006', '安全类别：GB18401-2010', 'A类婴幼儿用品', '货号：B2390', 'DREA', '等级：一等品检验员：08', 'WOTHERAL', '成份：面料：100%棉', '小晶晶母婴', '品名：纯棉单面背心套', '上装：66/48', '零售价：47.50元', '下装：66/47']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['中国农业银行', 'AGRICULTURAL BANK OF CHINA', '美太网点办', '理务贵全融华务']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['Halten', 'Sie', 'Sie', 'was', 'versprechen', 'Und', 'geben', 'Sie noch', 'was dazu']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['上海斯格威铂尔受大酒店', '一打浦路15号', '绿洲仕格维花园公寓', '打浦路252935号']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['小学数学第八册第三单元测试题', '（小数乘法）', '姓名：', '班别：', '分数：', '计算题.(共38分）', '1.直接写出得数。（10分）', '0.8X25=', '0X4.23=', '0.57+0.43=', '2.4X0.2=', '0.02', 'X500=', '2.8X10=', '8.29X100=', '2.5X0.4=', '4.87-1.7=', '8.07', 'X1000=', '2.用坚式计算。（12分）', '2.97X3.8=', '7.05X0.84=']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['Paddle.jit.saveO', ': best accuracy.pdopt', ': inference.pdmodel', 'ich det mv3', '模型训练', 'i best accuracy.pdparams', '模型转换', 'opt优化模型', 'i inference.pdiparams', 'idb opt.nb', ': best accuracy.states']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['CreditCord', '中国工商银行', 'INDUSTRIL AD COMMERCIA BAHKSLUHIN', '持卡人存根', '商户名称&编号(MERCHANT NAME&CODE)', '工行浣纱支行积分兑换', '120214020408', '检索参考号(SYSTEMINOJ）', '终端号(TERMINAL)', '60128018', '001', '卡号（CARD NUMBER):', '622210512', 'S', '卡别（CARDTYPE)', '有效期(EXPIRY', '牡丹准货记卡', '交易日期DATE)', '交易时间(TIMEE)', '2012-10-23', '12:43:15', '批次号(BATCHNO.）', '流水号（TRACEHOJ', '000016', '积分兑奖', '信用卡积分：50000', '持卡人姓名（CARDHOLDERNAME）', 'MR.1', '持卡人签名', '(CARDHOLDER S', 'SIGNATURE:']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['1', 'EPPING', 'Twinned with', 'Eppingen，']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['订007778', 'PASS', '登机牌', 'BOARDING', '座位号', '序号', 'SERIALNO.', '舱位 CLASS', 'SEAT NO.', '航班 FLIGHT', '日期 DATE', 'W', '03DEC', '035', 'MU2379', 'GATE', '登机口', '始发地', 'FROM', '登机时间BDT', '自的地 TO', '福州', 'TAIYUAN', 'G11', 'FUZHOU', '身份识别ID NO.', '姓名NAME', 'ZHANGQIWET', '票号TKT NO', '张祺伟', '票价 FARE', 'ETKT7813699238489/1', '登机口于起飞前10分钟关闭 GATES CLOSE 10 MINUTES BEFORE DEPARTURE TIME']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['右', '天然の', '#', '味の占', '濃厚衣', '梦少久', '本勇5办', '味\\\\\\\\深', '燒', '深办の苏石', '苏扣子布', '5味の', '3-9', '警', '鲍の己品', '匕辛', '节办', '永书', '香', '梦之', '二の两石', '脂の秉龙', '力力', 'N', '乙龙の揚', '', '出来龙了', ':', '新鲜', '少手', '炊吉大了', '風味豊', '熟成L力', '二人办', '美味', '芳醇', '口の.', 'C', '二の石', '史方办套', '旬', '産地直送', '久七', '食人二', '秘の', '<<', '淹机个一卜', '>1一>一', '本場の', '朝探', '俊味吉', '七3寸子']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['7788.com', 'Z57A001950', '杭州东售', '2013年07月07日13:39开', '06车12B号', '二等座', '杭州东', 'G7512次', '上海虹桥', 'HangZhouDong', 'Shang HaiHongQiao', '￥73.00元', '限乘当日当次车', '余友红', '检票口16', '3623301993****0941', '9004-1300-5707-08A0-0195-0', '和谐号', 'C0400']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['Meters/bonwe', '洗涤说明', '淘X口Ne', '最高洗涤温度30°C', '常规程序', '不可漂白', '悬挂晾干', '熨斗底板最高温度110°C', '常规干洗', '注童事项：']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['3504811', '属瑞金医院', '标本号：OA', '支吉单（临床内分泌', '标本种类：血清', '标志提示：', '门诊号：E05535540', '门诊科别：门诊内分泌', '条码号：10080603139 临床诊断：', '申请医生：孙首悦', '检测日期：2010', '产', '龄：31', '检验备注：', '结果', '参考范围', '项目', '3.12', 'FT3', '2.62--6.49pmol/L', '15.40', 'FT4', '9.01--19.04pmol/L', '1.31', '0.35--4.94uIU/ml', 'STSH', '0.15', '<4.1110/ml', 'TGAB', '0.1', '25.0U/L', 'TRAb', '0.17', '<5.611U/ml', 'TPOAb', '0.1', '0.1--10.0pg/ml', '降钙素', '报告时间：10-08-0711:28', '打印时间：1', '采样时间：10-08-0611:23', '收到时间：10-08-0611:44', '核', '检验：李纪平', '对：保', '以上7项检验结果仅供临床医师参考']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['栏L。未来味元氨']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['代号', '项目', '结果', '参考值', '单位', '谷丙转氨酶', '25.6', '0--40', 'ALT', 'U/L', 'TBIL', '总胆红素', '11.2', '<20', 'UMO1/L', '直接胆红素', '3.3', '0--7', 'DBIL', 'UMOL/L', '间接胆红素', '7.9', 'IBIL', '1.5--15', 'UMOL/L', '总蛋白', '58.9V', '60--80', 'TP', 'S/L', '白蛋白', '35.1', '33--55', 'ALB', ':/L', '球蛋白', '23.8', '20--30', 'GLO', '8/L', '白球比', 'A/G', '1.5', '1.5--2.5', 'ALP', '可性磷酉酯', '93', '15--112', 'IU/L', '谷氨酰转肽酸', '<50', '14.3', 'GGT', 'U/L', '谷草转氨酶', 'AST', '16.3', '8--40', 'U/L', '乳酸脱氢酶', 'LDH', '167', '114--240', 'U/L', '腺甘脱氨酶', '12.6', '4--24', 'ADA', 'U/L']\"], 'tensors': []}\n",
      "{'err_no': 0, 'err_msg': '', 'key': ['res'], 'value': [\"['股股东的持股情况', '单位：股', '股东总数（户）', '75.103', '前十名股东持股情况', '持有有限售条', '质押或冻结情况', '股东名称', '期末持股数量', '比例', '件股份数量', '股东性质', '股份状态', '数量', '国有法人', 'HONG Leong Bank', '境外法人', '浩海产业投资基金管理', '240000000', '境内非国有法人', '0', '国有法人', '公司', '16000000', '16000000', '公司', '国有法人', '成都欣天颐投资有限责', '0', '国有法人', '上海东昌投资发展有限', '3.3220%', '冻结', '公司', '50.000.00境内非国有法人', '新华文轩出版传媒股份', '0', '80000.000', '国有法人', '四川新华发行集团有限', '71243800', '1.9723%', '公司', '国有法人', '71.154.900', '限责任公司', '国有法人']\"], 'tensors': []}\n",
      "==> total number of test imgs:  20\n"
     ]
    }
   ],
   "source": [
    "!python pipeline_http_client.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can further adjust the number of concurrency in config.yml, here only the run-through effect is demonstrated, and it is set to 1 by default here\n",
    "\n",
    "```\n",
    "    det:\n",
    "        #Concurrency, when is_thread_op=True, it is thread concurrency; otherwise, it is process concurrency\n",
    "        concurrency: 1\n",
    "        ...\n",
    "    rec:\n",
    "        #Concurrency, when is_thread_op=True, it is thread concurrency; otherwise, it is process concurrency\n",
    "        concurrency: 1\n",
    "        ...\n",
    "```\n",
    "\n",
    "The predicted performance data will be automatically written into the `PipelineServingLogs/pipeline.tracer` file.\n",
    "\n",
    "\n",
    "\n",
    "## 4.5 FAQ\n",
    "\n",
    "**Q1**: After sending the request, there is no result returned or the output decoding error is prompted\n",
    "\n",
    "**A1**: Do not set the proxy when starting the service and sending the request. You can close the proxy before starting the service and before sending the request. The command to close the proxy is:\n",
    "```\n",
    "unset https_proxy\n",
    "unset http_proxy\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Practical end-to-side inference based on Paddle Lite\n",
    "\n",
    "With the popularity of the mobile Internet at any time, more and more mobile phones and embedded devices are used. At the same time, considering data security and model operating costs, more and more models are directly run in end-side devices.\n",
    "\n",
    "Paddle Lite is a lightweight inference engine for flying paddles. It provides efficient inference capabilities for mobile phones and IOT terminals, and extensively integrates cross-platform hardware to provide lightweight deployment solutions for end-side deployment and application landing issues.\n",
    "\n",
    "The introduction in this chapter is based on [Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite) Steps to deploy PaddleOCR ultra-lightweight Chinese detection and recognition model on the mobile terminal.\n",
    "\n",
    "\n",
    "Let me show you the demo of the PP-OCRv2 series model running on Android.\n",
    "\n",
    "[Android demo demo link](https://paddleocr.bj.bcebos.com/presentation_demo/android_lite%20demo.mp4)\n",
    "\n",
    "Since it is not possible to demonstrate here, here is how to develop the PP-OCRv2 system running program based on Paddle Lite.\n",
    "\n",
    "If you want practical operation, you can refer to [PaddleOCR deployment document based on Paddle Lite](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/deploy/lite/readme.md).\n",
    "\n",
    "## 5.1 Prepare the environment\n",
    "\n",
    "Here you need to prepare the cross-compilation environment and the Paddle Lite prediction library at the same time. The cross-compilation environment is used to generate executable files that can be used on end-side devices. It is recommended to use docker as a cross-compilation environment.\n",
    "\n",
    "## 5.2 Preparing the model\n",
    "\n",
    "When using Paddle Lite for model prediction, it is necessary to first convert the inference model into an optimized model for Paddle Lite prediction (the suffix is generally `nb`). It adopts a variety of strategies to automatically optimize the original model, including quantification, Methods such as subgraph fusion, hybrid scheduling, and Kernel optimization. The optimized model is lighter and faster.\n",
    "\n",
    "## 5.3 Compile\n",
    "\n",
    "Run `make -j` to execute the compilation and get the executable file. The first execution of this command will download dependent libraries such as opencv. After the download is complete, run `make -j` again.\n",
    "\n",
    "## 5.4 Upload to mobile terminals such as mobile phones\n",
    "\n",
    "Use tools such as adb to transfer executable files, model files, configuration files, etc. to mobile terminals such as mobile phones.\n",
    "\n",
    "## 5.5 Run\n",
    "\n",
    "Run the executable file on the mobile terminal to get the result, and the output example is as follows.\n",
    "\n",
    "<div align=\"center\">\n",
    "<img src=\"https://ai-studio-static-online.cdn.bcebos.com/c05a80380a6d483b81dd1be87f78beb4ec67659eabc7454686ea9588ea46c3ab\"  width = \"600\" />\n",
    "</div>\n",
    "<center>Figure 7: Mobile terminal output result</center>\n",
    "\n",
    "## 5.6 FAQ\n",
    "\n",
    "Q1: What if I want to change the model, do I need to go through the process again?\n",
    "\n",
    "A1: If you have gone through the above steps, you only need to replace the .nb model file to replace the model. At the same time, pay attention to updating the dictionary\n",
    "\n",
    "Q2: How to test with another picture?\n",
    "\n",
    "A2: Replace the .jpg test image under debug with the image you want to test, just push adb to the phone\n",
    "\n",
    "Q3: How to package it into the mobile APP?\n",
    "\n",
    "A3: This demo aims to provide the core algorithm part that can run OCR on mobile phones. PaddleOCR/deploy/android_demo is an example of encapsulating this demo into a mobile app for reference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Homework\n",
    "\n",
    "Please refer to [Predictive Deployment Objective Questions](https://aistudio.baidu.com/aistudio/education/objective/28710) and [Predictive Deployment Practice Questions](https://aistudio.baidu.com/aistudio/projectdetail/3297083) part."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
