Merge branch 'dygraph' into dygraph

doc/doc_ch/algorithm_overview.md

@@ -68,4 +68,5 @@ PaddleOCR基于动态图开源的文本识别算法列表：
 |NRTR|NRTR_MTB| 84.3% | rec_mtb_nrtr | [下载链接](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_mtb_nrtr_train.tar) |
 |SAR|Resnet31| 87.2% | rec_r31_sar | [下载链接](https://paddleocr.bj.bcebos.com/dygraph_v2.1/rec/rec_r31_sar_train.tar) |
 |SEED| Aster_Resnet | 85.2% | rec_resnet_stn_bilstm_att | [下载链接](https://paddleocr.bj.bcebos.com/dygraph_v2.1/rec/rec_resnet_stn_bilstm_att.tar)|
+
 PaddleOCR文本识别算法的训练和使用请参考文档教程中[模型训练/评估中的文本识别部分](./recognition.md)。

doc/doc_ch/detection.md

@@ -96,15 +96,28 @@ python3 tools/train.py -c configs/det/det_mv3_db.yml \
 # 单机多卡训练，通过 --gpus 参数设置使用的GPU ID
 python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/det_mv3_db.yml \
      -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained
+ 
+# 多机多卡训练，通过 --ips 参数设置使用的机器IP地址，通过 --gpus 参数设置使用的GPU ID
+python3 -m paddle.distributed.launch --ips="10.21.226.181,10.21.226.133" --gpus '0,1,2,3' tools/train.py -c configs/det/det_mv3_db.yml \
+     -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained
 ```
 
 上述指令中，通过-c 选择训练使用configs/det/det_db_mv3.yml配置文件。
 有关配置文件的详细解释，请参考[链接](./config.md)。
-
+ 
 您也可以通过-o参数在不需要修改yml文件的情况下，改变训练的参数，比如，调整训练的学习率为0.0001
 ```shell
 python3 tools/train.py -c configs/det/det_mv3_db.yml -o Optimizer.base_lr=0.0001
 ```
+ 
+**注意:** 采用多机多卡训练时，需要替换上面命令中的ips值为您机器的地址，机器之间需要能够相互ping通。查看机器ip地址的命令为`ifconfig`。
+ 
+如果您想进一步加快训练速度，可以使用[自动混合精度训练](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/01_paddle2.0_introduction/basic_concept/amp_cn.html)， 以单机单卡为例，命令如下：
+```shell
+python3 tools/train.py -c configs/det/det_mv3_db.yml \
+     -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained \
+     Global.use_amp=True Global.scale_loss=1024.0 Global.use_dynamic_loss_scaling=True
+ ```
 
 <a name="14-----"></a>
 ## 1.4 断点训练

doc/doc_ch/enhanced_ctc_loss.md

@@ -64,7 +64,7 @@ C-CTC Loss是CTC Loss + Center Loss的简称。 其中Center Loss出自论文 <
 
 以配置文件`configs/rec/ch_PP-OCRv2/ch_PP-OCRv2_rec.yml`为例， center提取命令如下所示:
 ```
-python tools/export_center.py -c configs/rec/ch_PP-OCRv2/ch_PP-OCRv2_rec.yml -o  Global.pretrained_model: "./output/rec_mobile_pp-OCRv2/best_accuracy"
+python tools/export_center.py -c configs/rec/ch_PP-OCRv2/ch_PP-OCRv2_rec.yml -o Global.pretrained_model="./output/rec_mobile_pp-OCRv2/best_accuracy"
 ```
 运行完后，会在PaddleOCR主目录下生成`train_center.pkl`.
 

doc/doc_ch/training.md

@@ -4,15 +4,16 @@
 
 同时会简单介绍PaddleOCR模型训练数据的组成部分，以及如何在垂类场景中准备数据finetune模型。
 
-- [1. 基本概念](#基本概念)
-  * [1.1 学习率](#学习率)
-  * [1.2 正则化](#正则化)
-  * [1.3 评估指标](#评估指标)
-- [2. 数据与垂类场景](#数据与垂类场景)
-  * [2.1 训练数据](#训练数据)
-  * [2.2 垂类场景](#垂类场景)
-  * [2.3 自己构建数据集](#自己构建数据集)
-* [3. 常见问题](#常见问题)
+- [1.配置文件说明](#配置文件)
+- [2. 基本概念](#基本概念)
+  * [2.1 学习率](#学习率)
+  * [2.2 正则化](#正则化)
+  * [2.3 评估指标](#评估指标)
+- [3. 数据与垂类场景](#数据与垂类场景)
+  * [3.1 训练数据](#训练数据)
+  * [3.2 垂类场景](#垂类场景)
+  * [3.3 自己构建数据集](#自己构建数据集)
+* [4. 常见问题](#常见问题)
 
 <a name="基本概念"></a>
 ## 1. 基本概念
@@ -23,7 +24,7 @@ OCR(Optical Character Recognition,光学字符识别)是指对图像进行分析
 模型调优时需要关注以下参数：
 
 <a name="学习率"></a>
-### 1.1 学习率
+### 2.1 学习率
 
 学习率是训练神经网络的重要超参数之一，它代表在每一次迭代中梯度向损失函数最优解移动的步长。
 在PaddleOCR中提供了多种学习率更新策略,可以通过配置文件修改，例如：
@@ -42,7 +43,7 @@ Piecewise 代表分段常数衰减，在不同的学习阶段指定不同的学
 warmup_epoch 代表在前5个epoch中，学习率将逐渐从0增加到base_lr。全部策略可以参考代码[learning_rate.py](../../ppocr/optimizer/learning_rate.py) 。
 
 <a name="正则化"></a>
-### 1.2 正则化
+### 2.2 正则化
 
 正则化可以有效的避免算法过拟合，PaddleOCR中提供了L1、L2正则方法，L1 和 L2 正则化是最常用的正则化方法。L1 正则化向目标函数添加正则化项，以减少参数的绝对值总和；而 L2 正则化中，添加正则化项的目的在于减少参数平方的总和。配置方法如下：
 
@@ -55,7 +56,7 @@ Optimizer:
 ```
 
 <a name="评估指标"></a>
-### 1.3 评估指标
+### 2.3 评估指标
 
 （1）检测阶段：先按照检测框和标注框的IOU评估，IOU大于某个阈值判断为检测准确。这里检测框和标注框不同于一般的通用目标检测框，是采用多边形进行表示。检测准确率：正确的检测框个数在全部检测框的占比，主要是判断检测指标。检测召回率：正确的检测框个数在全部标注框的占比，主要是判断漏检的指标。
 
@@ -65,10 +66,10 @@ Optimizer:
 
 <a name="数据与垂类场景"></a>
 
-## 2. 数据与垂类场景
+## 3. 数据与垂类场景
 
 <a name="训练数据"></a>
-### 2.1 训练数据
+### 3.1 训练数据
 目前开源的模型，数据集和量级如下：
 
     - 检测：  
@@ -83,13 +84,14 @@ Optimizer:
 其中，公开数据集都是开源的，用户可自行搜索下载，也可参考[中文数据集](./datasets.md)，合成数据暂不开源，用户可使用开源合成工具自行合成，可参考的合成工具包括[text_renderer](https://github.com/Sanster/text_renderer) 、[SynthText](https://github.com/ankush-me/SynthText) 、[TextRecognitionDataGenerator](https://github.com/Belval/TextRecognitionDataGenerator) 等。
 
 <a name="垂类场景"></a>
-### 2.2 垂类场景
+### 3.2 垂类场景
 
 PaddleOCR主要聚焦通用OCR，如果有垂类需求，您可以用PaddleOCR+垂类数据自己训练；
 如果缺少带标注的数据，或者不想投入研发成本，建议直接调用开放的API，开放的API覆盖了目前比较常见的一些垂类。
 
 <a name="自己构建数据集"></a>
-### 2.3 自己构建数据集
+
+### 3.3 自己构建数据集
 
 在构建数据集时有几个经验可供参考：
 
@@ -107,7 +109,7 @@ PaddleOCR主要聚焦通用OCR，如果有垂类需求，您可以用PaddleOCR+
 
 <a name="常见问题"></a>
 
-## 3. 常见问题
+## 4. 常见问题
 
 **Q**：训练CRNN识别时，如何选择合适的网络输入shape？
 

doc/doc_en/detection_en.md

@@ -98,7 +98,19 @@ python3 tools/train.py -c configs/det/det_mv3_db.yml -o   \
 # multi-GPU training
 # Set the GPU ID used by the '--gpus' parameter.
 python3 -m paddle.distributed.launch --gpus '0,1,2,3'  tools/train.py -c configs/det/det_mv3_db.yml -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained
-
+ 
+# multi-Node, multi-GPU training
+# Set the IPs of your nodes used by the '--ips' parameter. Set the GPU ID used by the '--gpus' parameter.
+python3 -m paddle.distributed.launch --ips="10.21.226.181,10.21.226.133" --gpus '0,1,2,3' tools/train.py -c configs/det/det_mv3_db.yml \
+     -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained
+```
+**Note:** For multi-Node multi-GPU training, you need to replace the `ips` value in the preceding command with the address of your machine, and the machines must be able to ping each other. The command for viewing the IP address of the machine is `ifconfig`.
+ 
+If you want to further speed up the training, you can use [automatic mixed precision training](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/01_paddle2.0_introduction/basic_concept/amp_en.html). for single card training, the command is as follows:
+```
+python3 tools/train.py -c configs/det/det_mv3_db.yml \
+     -o Global.pretrained_model=./pretrain_models/MobileNetV3_large_x0_5_pretrained \
+     Global.use_amp=True Global.scale_loss=1024.0 Global.use_dynamic_loss_scaling=True
 ```
 
 ### 2.2 Load Trained Model and Continue Training

ppocr/data/imaug/east_process.py

@@ -11,7 +11,10 @@
 #WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 #See the License for the specific language governing permissions and
 #limitations under the License.
-
+"""
+This code is refered from: 
+https://github.com/songdejia/EAST/blob/master/data_utils.py
+"""
 import math
 import cv2
 import numpy as np
@@ -24,10 +27,10 @@ __all__ = ['EASTProcessTrain']
 
 class EASTProcessTrain(object):
     def __init__(self,
-                 image_shape = [512, 512],
-                 background_ratio = 0.125,
-                 min_crop_side_ratio = 0.1,
-                 min_text_size = 10,
+                 image_shape=[512, 512],
+                 background_ratio=0.125,
+                 min_crop_side_ratio=0.1,
+                 min_text_size=10,
                  **kwargs):
         self.input_size = image_shape[1]
         self.random_scale = np.array([0.5, 1, 2.0, 3.0])
@@ -282,12 +285,7 @@ class EASTProcessTrain(object):
                 1.0 / max(min(poly_h, poly_w), 1.0)
         return score_map, geo_map, training_mask
 
-    def crop_area(self,
-                  im,
-                  polys,
-                  tags,
-                  crop_background=False,
-                  max_tries=50):
+    def crop_area(self, im, polys, tags, crop_background=False, max_tries=50):
         """
         make random crop from the input image
         :param im:
@@ -435,5 +433,4 @@ class EASTProcessTrain(object):
         data['score_map'] = score_map
         data['geo_map'] = geo_map
         data['training_mask'] = training_mask
-        # print(im.shape, score_map.shape, geo_map.shape, training_mask.shape)
-        return data
\ No newline at end of file
+        return data

ppocr/data/imaug/iaa_augment.py

@@ -11,6 +11,10 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+"""
+This code is refer from:
+https://github.com/WenmuZhou/DBNet.pytorch/blob/master/data_loader/modules/iaa_augment.py
+"""
 from __future__ import absolute_import
 from __future__ import division
 from __future__ import print_function

ppocr/data/imaug/make_border_map.py

-# -*- coding:utf-8 -*- 
+# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+This code is refer from:
+https://github.com/WenmuZhou/DBNet.pytorch/blob/master/data_loader/modules/make_border_map.py
+"""
 
 from __future__ import absolute_import
 from __future__ import division

ppocr/data/imaug/make_pse_gt.py

-# -*- coding:utf-8 -*- 
+# copyright (c) 2021 PaddlePaddle Authors. All Rights Reserve.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
 
 from __future__ import absolute_import
 from __future__ import division
@@ -12,12 +24,8 @@ from shapely.geometry import Polygon
 
 __all__ = ['MakePseGt']
 
-class MakePseGt(object):
-    r'''
-    Making binary mask from detection data with ICDAR format.
-    Typically following the process of class `MakeICDARData`.
-    '''
 
+class MakePseGt(object):
     def __init__(self, kernel_num=7, size=640, min_shrink_ratio=0.4, **kwargs):
         self.kernel_num = kernel_num
         self.min_shrink_ratio = min_shrink_ratio
@@ -38,16 +46,20 @@ class MakePseGt(object):
             text_polys *= scale
 
         gt_kernels = []
-        for i in range(1,self.kernel_num+1):
+        for i in range(1, self.kernel_num + 1):
             # s1->sn, from big to small
-            rate = 1.0 - (1.0 - self.min_shrink_ratio) / (self.kernel_num - 1) * i
-            text_kernel, ignore_tags = self.generate_kernel(image.shape[0:2], rate, text_polys, ignore_tags)
+            rate = 1.0 - (1.0 - self.min_shrink_ratio) / (self.kernel_num - 1
+                                                          ) * i
+            text_kernel, ignore_tags = self.generate_kernel(
+                image.shape[0:2], rate, text_polys, ignore_tags)
             gt_kernels.append(text_kernel)
 
         training_mask = np.ones(image.shape[0:2], dtype='uint8')
         for i in range(text_polys.shape[0]):
             if ignore_tags[i]:
-                cv2.fillPoly(training_mask, text_polys[i].astype(np.int32)[np.newaxis, :, :], 0)
+                cv2.fillPoly(training_mask,
+                             text_polys[i].astype(np.int32)[np.newaxis, :, :],
+                             0)
 
         gt_kernels = np.array(gt_kernels)
         gt_kernels[gt_kernels > 0] = 1
@@ -59,16 +71,25 @@ class MakePseGt(object):
         data['mask'] = training_mask.astype('float32')
         return data
 
-    def generate_kernel(self, img_size, shrink_ratio, text_polys, ignore_tags=None):
+    def generate_kernel(self,
+                        img_size,
+                        shrink_ratio,
+                        text_polys,
+                        ignore_tags=None):
+        """
+        Refer to part of the code:
+        https://github.com/open-mmlab/mmocr/blob/main/mmocr/datasets/pipelines/textdet_targets/base_textdet_targets.py
+        """
+
         h, w = img_size
         text_kernel = np.zeros((h, w), dtype=np.float32)
         for i, poly in enumerate(text_polys):
             polygon = Polygon(poly)
-            distance = polygon.area * (1 - shrink_ratio * shrink_ratio) / (polygon.length + 1e-6)
+            distance = polygon.area * (1 - shrink_ratio * shrink_ratio) / (
+                polygon.length + 1e-6)
             subject = [tuple(l) for l in poly]
             pco = pyclipper.PyclipperOffset()
-            pco.AddPath(subject, pyclipper.JT_ROUND,
-                        pyclipper.ET_CLOSEDPOLYGON)
+            pco.AddPath(subject, pyclipper.JT_ROUND, pyclipper.ET_CLOSEDPOLYGON)
             shrinked = np.array(pco.Execute(-distance))
 
             if len(shrinked) == 0 or shrinked.size == 0:

ppocr/data/imaug/make_shrink_map.py

-# -*- coding:utf-8 -*- 
+# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+This code is refer from:
+https://github.com/WenmuZhou/DBNet.pytorch/blob/master/data_loader/modules/make_shrink_map.py
+"""
 
 from __future__ import absolute_import
 from __future__ import division

ppocr/data/imaug/random_crop_data.py

-# -*- coding:utf-8 -*- 
+# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+This code is refer from:
+https://github.com/WenmuZhou/DBNet.pytorch/blob/master/data_loader/modules/random_crop_data.py
+"""
 
 from __future__ import absolute_import
 from __future__ import division

ppocr/data/imaug/sast_process.py

@@ -11,7 +11,10 @@
 #WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 #See the License for the specific language governing permissions and
 #limitations under the License.
-
+"""
+This part code is refered from: 
+https://github.com/songdejia/EAST/blob/master/data_utils.py
+"""
 import math
 import cv2
 import numpy as np

ppocr/data/imaug/text_image_aug/augment.py

@@ -11,6 +11,10 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+"""
+This code is refer from:
+https://github.com/RubanSeven/Text-Image-Augmentation-python/blob/master/augment.py
+"""
 
 import numpy as np
 from .warp_mls import WarpMLS

ppocr/data/imaug/text_image_aug/warp_mls.py

@@ -11,6 +11,10 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+"""
+This code is refer from:
+https://github.com/RubanSeven/Text-Image-Augmentation-python/blob/master/warp_mls.py
+"""
 
 import numpy as np
 
@@ -161,4 +165,4 @@ class WarpMLS:
         dst = np.clip(dst, 0, 255)
         dst = np.array(dst, dtype=np.uint8)
 
-        return dst
\ No newline at end of file
+        return dst

ppocr/losses/ace_loss.py

@@ -11,6 +11,9 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+
+# This code is refer from: https://github.com/viig99/LS-ACELoss
+
 from __future__ import absolute_import
 from __future__ import division
 from __future__ import print_function
@@ -32,7 +35,7 @@ class ACELoss(nn.Layer):
     def __call__(self, predicts, batch):
         if isinstance(predicts, (list, tuple)):
             predicts = predicts[-1]
-            
+
         B, N = predicts.shape[:2]
         div = paddle.to_tensor([N]).astype('float32')
 

ppocr/losses/center_loss.py

@@ -12,6 +12,8 @@
 #See the License for the specific language governing permissions and
 #limitations under the License.
 
+# This code is refer from: https://github.com/KaiyangZhou/pytorch-center-loss
+
 from __future__ import absolute_import
 from __future__ import division
 from __future__ import print_function
@@ -27,6 +29,7 @@ class CenterLoss(nn.Layer):
     """
     Reference: Wen et al. A Discriminative Feature Learning Approach for Deep Face Recognition. ECCV 2016.
     """
+
     def __init__(self,
                  num_classes=6625,
                  feat_dim=96,

ppocr/losses/det_basic_loss.py

@@ -11,7 +11,10 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-
+"""
+This code is refer from:
+https://github.com/WenmuZhou/DBNet.pytorch/blob/master/models/losses/basic_loss.py
+"""
 from __future__ import absolute_import
 from __future__ import division
 from __future__ import print_function
@@ -147,4 +150,4 @@ class BCELoss(nn.Layer):
 
     def forward(self, input, label, mask=None, weight=None, name=None):
         loss = F.binary_cross_entropy(input, label, reduction=self.reduction)
-        return loss
\ No newline at end of file
+        return loss

ppocr/losses/det_db_loss.py

@@ -11,6 +11,10 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+"""
+This code is refer from:
+https://github.com/WenmuZhou/DBNet.pytorch/blob/master/models/losses/DB_loss.py
+"""
 
 from __future__ import absolute_import
 from __future__ import division

ppocr/losses/det_pse_loss.py

@@ -11,6 +11,10 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+"""
+This code is refer from:
+https://github.com/whai362/PSENet/blob/python3/models/head/psenet_head.py
+"""
 
 import paddle
 from paddle import nn