Merge branch 'dygraph' of https://github.com/PaddlePaddle/PaddleOCR into dygraph

update

notebook/notebook_ch/如何使用本书.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": false
+   },
+   "source": [
+    "# 1. 课程预备知识\n",
+    "\n",
+    "本课所涉及的OCR模型建立在深度学习的基础之上，因此与其相关的基础知识、环境配置、项目工程与其他资料将在本节介绍，尤其对深度学习不熟悉的读者可以查看和学习相应内容。\n",
+    "\n",
+    "### 1.1 预备知识\n",
+    "\n",
+    "深度学习的“学习”由机器学习中的神经元、感知机、多层神经网络等内容一路发展而来，因此了解基础的机器学习算法对于深度学习的理解和应用有很大帮助。而深度学习的“深”则体现在对大量信息处理过程中使用的卷积、池化等一系列以向量为基础的数学运算。如果缺乏这两者的理论基础，可以学习李宏毅老师的[线性代数](https://aistudio.baidu.com/aistudio/course/introduce/2063)和[机器学习](https://aistudio.baidu.com/aistudio/course/introduce/1978)课程。\n",
+    "\n",
+    "对于深度学习本身的理解，可以参考百度杰出架构师毕然老师的零基础课程：[百度架构师手把手带你零基础实践深度学习](https://aistudio.baidu.com/aistudio/course/introduce/1297)，其中覆盖了深度学习的发展历史，通过一个经典案例介绍深度学习的完整组成部分，是一套以实践为导向的深度学习课程。\n",
+    "\n",
+    "对于理论知识的实践，[Python基础知识](https://aistudio.baidu.com/aistudio/course/introduce/1224)必不可少，同时为了快速复现深度学习模型，本课程使用的深度学习框架为：飞桨PaddlePaddle。如果你已经使用过其他框架，通过[快速上手文档](https://www.paddlepaddle.org.cn/documentation/docs/zh/practices/quick_start/hello_paddle.html)可以迅速了解飞桨的使用方法。\n",
+    "\n",
+    "### 1.2 基础环境准备\n",
+    "\n",
+    "如果你想在本地环境运行本课程的代码且之前未搭建过Python环境，可以根据[零基础运行环境准备](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.3/doc/doc_ch/environment.md)，根据自己的操作系统安装Anaconda或docker环境。\n",
+    "\n",
+    "如果你没有本地资源，可以通过AI Studio实训平台完成代码运行，其中的每个项目都通过Notebook的方式呈现，方便开发者学习。若对Notebook的相关操作不熟悉，可以参考[AI Studio项目说明](https://ai.baidu.com/ai-doc/AISTUDIO/0k3e2tfzm)。\n",
+    "\n",
+    "### 1.3 获取和运行代码\n",
+    "\n",
+    "本课程依托PaddleOCR的代码库形成，首先，克隆PaddleOCR的完整项目：\n",
+    "\n",
+    "```bash\n",
+    "#【推荐】\n",
+    "git clone https://github.com/PaddlePaddle/PaddleOCR\n",
+    "\n",
+    "# 如果因为网络问题无法pull成功，也可选择使用码云上的托管：\n",
+    "git clone https://gitee.com/paddlepaddle/PaddleOCR\n",
+    "```\n",
+    "\n",
+    "> 注：码云托管代码可能无法实时同步本github项目更新，存在3~5天延时，请优先使用推荐方式。\n",
+    ">\n",
+    "> ​\t\t如果你不熟悉git操作，可以直接在PaddleOCR的首页的 `Code` 中下载压缩包\n",
+    "\n",
+    "然后安装第三方库：\n",
+    "\n",
+    "```\n",
+    "cd PaddleOCR\n",
+    "pip3 install -r requirements.txt\n",
+    "```\n",
+    "\n",
+    "\n",
+    "\n",
+    "### 1.4 查阅资料\n",
+    "\n",
+    "[PaddleOCR使用文档](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.3/README_ch.md#%E6%96%87%E6%A1%A3%E6%95%99%E7%A8%8B) (中文) 中详细介绍了如何使用PaddleOCR完成模型应用、训练和部署。文档内容丰富，大多数用户的问题都在文档或FAQ中有所描述，尤其在[FAQ(中文)](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.3/doc/doc_ch/FAQ.md)中，按照深度学习的应用过程沉淀了用户的常见问题，建议大家仔细阅读。\n",
+    "\n",
+    "### 1.5 寻求帮助\n",
+    "\n",
+    "如果你在使用PaddleOCR的过程中遇到BUG、易用性或者文档相关的问题，可通过[Github issue](https://github.com/PaddlePaddle/PaddleOCR/issues)与官方联系，请按照issue模板尽可能多的提供信息，以便官方人员迅速定位问题。同时，微信群是广大PaddleOCR用户的日常交流阵地，更适合提问一些咨询类问题，除了有PaddleOCR团队成员以外，还会有热心开发者回答大家的问题。"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "py35-paddle1.2.0"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.4"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}

ppocr/data/imaug/label_ops.py

@@ -344,7 +344,7 @@ class KieLabelEncode(object):
         max_num = 300
         temp_bboxes = np.zeros([max_num, 4])
         h, _ = bboxes.shape
-        temp_bboxes[:h, :h] = bboxes
+        temp_bboxes[:h, :] = bboxes
 
         temp_relations = np.zeros([max_num, max_num, 5])
         temp_relations[:h, :h, :] = relations

ppstructure/vqa/README.md

@@ -96,10 +96,7 @@ git clone https://gitee.com/paddlepaddle/PaddleOCR
 - **（3）安装PaddleNLP**
 
 ```bash
-# 需要使用PaddleNLP最新的代码版本进行安装
-git clone https://github.com/PaddlePaddle/PaddleNLP -b develop
-cd PaddleNLP
-pip3 install -e .
+pip3 install "paddlenlp>=2.2.1"
 ```
 
 
@@ -234,7 +231,7 @@ python3 train_re.py \
     --train_label_path "XFUND/zh_train/xfun_normalize_train.json" \
     --eval_data_dir "XFUND/zh_val/image" \
     --eval_label_path "XFUND/zh_val/xfun_normalize_val.json" \
-    --label_map_path 'labels/labels_ser.txt' \
+    --label_map_path "labels/labels_ser.txt" \
     --num_train_epochs 200 \
     --eval_steps 10 \
     --output_dir "output/re/"  \
@@ -258,7 +255,7 @@ python3 train_re.py \
     --train_label_path "XFUND/zh_train/xfun_normalize_train.json" \
     --eval_data_dir "XFUND/zh_val/image" \
     --eval_label_path "XFUND/zh_val/xfun_normalize_val.json" \
-    --label_map_path 'labels/labels_ser.txt' \
+    --label_map_path "labels/labels_ser.txt" \
     --num_train_epochs 2 \
     --eval_steps 10 \
     --output_dir "output/re/"  \
@@ -283,7 +280,7 @@ python3 eval_re.py \
     --max_seq_length 512 \
     --eval_data_dir "XFUND/zh_val/image" \
     --eval_label_path "XFUND/zh_val/xfun_normalize_val.json" \
-    --label_map_path 'labels/labels_ser.txt' \
+    --label_map_path "labels/labels_ser.txt" \
     --output_dir "output/re/"  \
     --per_gpu_eval_batch_size 8 \
     --num_workers 8 \
@@ -301,7 +298,7 @@ python3 infer_re.py \
     --max_seq_length 512 \
     --eval_data_dir "XFUND/zh_val/image" \
     --eval_label_path "XFUND/zh_val/xfun_normalize_val.json" \
-    --label_map_path 'labels/labels_ser.txt' \
+    --label_map_path "labels/labels_ser.txt" \
     --output_dir "output/re/"  \
     --per_gpu_eval_batch_size 1 \
     --seed 2048

ppstructure/vqa/eval_re.py

@@ -24,7 +24,7 @@ import paddle
 from paddlenlp.transformers import LayoutXLMTokenizer, LayoutXLMModel, LayoutXLMForRelationExtraction
 
 from xfun import XFUNDataset
-from utils import parse_args, get_bio_label_maps, print_arguments
+from vqa_utils import parse_args, get_bio_label_maps, print_arguments
 from data_collator import DataCollator
 from metric import re_score
 

ppstructure/vqa/eval_ser.py

@@ -33,7 +33,7 @@ from paddlenlp.transformers import LayoutLMModel, LayoutLMTokenizer, LayoutLMFor
 
 from xfun import XFUNDataset
 from losses import SERLoss
-from utils import parse_args, get_bio_label_maps, print_arguments
+from vqa_utils import parse_args, get_bio_label_maps, print_arguments
 
 from ppocr.utils.logging import get_logger
 

ppstructure/vqa/infer_re.py

@@ -15,7 +15,7 @@ import paddle
 from paddlenlp.transformers import LayoutXLMTokenizer, LayoutXLMModel, LayoutXLMForRelationExtraction
 
 from xfun import XFUNDataset
-from utils import parse_args, get_bio_label_maps, draw_re_results
+from vqa_utils import parse_args, get_bio_label_maps, draw_re_results
 from data_collator import DataCollator
 
 from ppocr.utils.logging import get_logger

ppstructure/vqa/infer_ser.py

@@ -14,6 +14,10 @@
 
 import os
 import sys
+
+__dir__ = os.path.dirname(os.path.abspath(__file__))
+sys.path.append(__dir__)
+
 import json
 import cv2
 import numpy as np
@@ -22,7 +26,7 @@ from copy import deepcopy
 import paddle
 
 # relative reference
-from utils import parse_args, get_image_file_list, draw_ser_results, get_bio_label_maps
+from vqa_utils import parse_args, get_image_file_list, draw_ser_results, get_bio_label_maps
 from paddlenlp.transformers import LayoutXLMModel, LayoutXLMTokenizer, LayoutXLMForTokenClassification
 from paddlenlp.transformers import LayoutLMModel, LayoutLMTokenizer, LayoutLMForTokenClassification
 

ppstructure/vqa/infer_ser_e2e.py

@@ -14,6 +14,10 @@
 
 import os
 import sys
+
+__dir__ = os.path.dirname(os.path.abspath(__file__))
+sys.path.append(__dir__)
+
 import json
 import cv2
 import numpy as np
@@ -25,9 +29,9 @@ from paddlenlp.transformers import LayoutXLMModel, LayoutXLMTokenizer, LayoutXLM
 from paddlenlp.transformers import LayoutLMModel, LayoutLMTokenizer, LayoutLMForTokenClassification
 
 # relative reference
-from utils import parse_args, get_image_file_list, draw_ser_results, get_bio_label_maps
+from vqa_utils import parse_args, get_image_file_list, draw_ser_results, get_bio_label_maps
 
-from utils import pad_sentences, split_page, preprocess, postprocess, merge_preds_list_with_ocr_info
+from vqa_utils import pad_sentences, split_page, preprocess, postprocess, merge_preds_list_with_ocr_info
 
 MODELS = {
     'LayoutXLM':

ppstructure/vqa/infer_ser_re_e2e.py

@@ -24,7 +24,7 @@ import paddle
 from paddlenlp.transformers import LayoutXLMModel, LayoutXLMTokenizer, LayoutXLMForRelationExtraction
 
 # relative reference
-from utils import parse_args, get_image_file_list, draw_re_results
+from vqa_utils import parse_args, get_image_file_list, draw_re_results
 from infer_ser_e2e import SerPredictor
 
 

ppstructure/vqa/train_re.py

@@ -27,7 +27,7 @@ import paddle
 from paddlenlp.transformers import LayoutXLMTokenizer, LayoutXLMModel, LayoutXLMForRelationExtraction
 
 from xfun import XFUNDataset
-from utils import parse_args, get_bio_label_maps, print_arguments, set_seed
+from vqa_utils import parse_args, get_bio_label_maps, print_arguments, set_seed
 from data_collator import DataCollator
 from eval_re import evaluate
 

ppstructure/vqa/train_ser.py

@@ -32,7 +32,7 @@ from paddlenlp.transformers import LayoutXLMModel, LayoutXLMTokenizer, LayoutXLM
 from paddlenlp.transformers import LayoutLMModel, LayoutLMTokenizer, LayoutLMForTokenClassification
 
 from xfun import XFUNDataset
-from utils import parse_args, get_bio_label_maps, print_arguments, set_seed
+from vqa_utils import parse_args, get_bio_label_maps, print_arguments, set_seed
 from eval_ser import evaluate
 from losses import SERLoss
 from ppocr.utils.logging import get_logger

requirements.txt

@@ -13,4 +13,4 @@ lxml
 premailer
 openpyxl
 fasttext==0.9.1
-
+paddlenlp>=2.2.1

test_tipc/configs/ch_ppocr_mobile_V2.0_det_FPGM/train_infer_python.txt

@@ -26,7 +26,7 @@ null:null
 ##
 ===========================infer_params===========================
 Global.save_inference_dir:./output/
-Global.pretrained_model:
+Global.checkpoints:
 norm_export:null
 quant_export:null
 fpgm_export:deploy/slim/prune/export_prune_model.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o 

test_tipc/configs/ch_ppocr_mobile_v2.0_det/train_infer_python.txt

@@ -26,7 +26,7 @@ null:null
 ##
 ===========================infer_params===========================
 Global.save_inference_dir:./output/
-Global.pretrained_model:
+Global.checkpoints:
 norm_export:tools/export_model.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o 
 quant_export:null
 fpgm_export:null

test_tipc/configs/ch_ppocr_mobile_v2.0_rec_FPGM/train_infer_python.txt

@@ -26,7 +26,7 @@ null:null
 ##
 ===========================infer_params===========================
 Global.save_inference_dir:./output/
-Global.pretrained_model:
+Global.checkpoints:
 norm_export:null
 quant_export:null
 fpgm_export:deploy/slim/prune/export_prune_model.py -c test_tipc/configs/ch_ppocr_mobile_v2.0_rec_FPGM/rec_chinese_lite_train_v2.0.yml -o 

test_tipc/configs/ch_ppocr_mobile_v2.0_rec_KL/model_linux_gpu_normal_normal_infer_python_linux_gpu_cpu.txt

@@ -13,7 +13,7 @@ inference:tools/infer/predict_rec.py
 --rec_batch_num:1
 --use_tensorrt:False|True
 --precision:int8
---det_model_dir:
+--rec_model_dir:
 --image_dir:./inference/rec_inference
 null:null
 --benchmark:True

test_tipc/configs/ch_ppocr_server_v2.0_det/train_infer_python.txt

@@ -26,7 +26,7 @@ null:null
 ##
 ===========================infer_params===========================
 Global.save_inference_dir:./output/
-Global.pretrained_model:
+Global.checkpoints:
 norm_export:tools/export_model.py -c test_tipc/configs/ch_ppocr_server_v2.0_det/det_r50_vd_db.yml -o 
 quant_export:null 
 fpgm_export:null