Merge remote-tracking branch 'upstream/dygraph' into dy1

PPOCRLabel/README.md

@@ -13,7 +13,7 @@ PPOCRLabel is a semi-automatic graphic annotation tool suitable for OCR field, w
   - Undo function: In the process of drawing a four-point label box or after editing the box, press Ctrl+Z to undo the previous operation.
   - Fix image rotation and size problems, optimize the process of editing the mark frame (by [ninetailskim](https://github.com/ninetailskim)、 [edencfc](https://github.com/edencfc)).
 - 2021.1.11: Optimize the labeling experience (by [edencfc](https://github.com/edencfc)),
-  - Users can choose whether to pop up the label input dialog after drawing the detection box in "View - Pop-up Label Input Dialog". 
+  - Users can choose whether to pop up the label input dialog after drawing the detection box in "View - Pop-up Label Input Dialog".
   - The recognition result scrolls synchronously when users click related detection box.
   - Click to modify the recognition result.(If you can't change the result, please switch to the system default input method, or switch back to the original input method again)
 - 2020.12.18: Support re-recognition of a single label box (by [ninetailskim](https://github.com/ninetailskim) ), perfect shortcut keys.
@@ -52,7 +52,7 @@ python3 PPOCRLabel.py
 ```
 pip3 install pyqt5
 pip3 uninstall opencv-python # Uninstall opencv manually as it conflicts with pyqt
-pip3 install opencv-contrib-python-headless # Install the headless version of opencv
+pip3 install opencv-contrib-python-headless==4.2.0.32 # Install the headless version of opencv
 cd ./PPOCRLabel # Change the directory to the PPOCRLabel folder
 python3 PPOCRLabel.py
 ```
@@ -129,7 +129,7 @@ python3 PPOCRLabel.py
 
 - Default model: PPOCRLabel uses the Chinese and English ultra-lightweight OCR model in PaddleOCR by default, supports Chinese, English and number recognition, and multiple language detection.
 
-- Model language switching: Changing the built-in model language is supportable by clicking "PaddleOCR"-"Choose OCR Model" in the menu bar. Currently supported languages​include French, German, Korean, and Japanese. 
+- Model language switching: Changing the built-in model language is supportable by clicking "PaddleOCR"-"Choose OCR Model" in the menu bar. Currently supported languages​include French, German, Korean, and Japanese.
   For specific model download links, please refer to [PaddleOCR Model List](https://github.com/PaddlePaddle/PaddleOCR/blob/develop/doc/doc_en/models_list_en.md#multilingual-recognition-modelupdating)
 
 - Custom model: The model trained by users can be replaced by modifying PPOCRLabel.py in [PaddleOCR class instantiation](https://github.com/PaddlePaddle/PaddleOCR/blob/develop/PPOCRLabel/PPOCRLabel.py#L110) referring [Custom Model Code](https://github.com/PaddlePaddle/PaddleOCR/blob/develop/doc/doc_en/whl_en.md#use-custom-model)
@@ -162,11 +162,11 @@ For some data that are difficult to recognize, the recognition results will not
     ```
     pyrcc5 -o libs/resources.py resources.qrc
     ```
-- If you get an error ``` module 'cv2' has no attribute 'INTER_NEAREST'```, you need to delete all opencv related packages first, and then reinstall the headless version of opencv
+- If you get an error ``` module 'cv2' has no attribute 'INTER_NEAREST'```, you need to delete all opencv related packages first, and then reinstall the 4.2.0.32  version of headless opencv
     ```
-    pip install opencv-contrib-python-headless
+    pip install opencv-contrib-python-headless==4.2.0.32
     ```
-    
+
 ### Related
 
 1.[Tzutalin. LabelImg. Git code (2015)](https://github.com/tzutalin/labelImg)
\ No newline at end of file

PPOCRLabel/README_ch.md

@@ -52,7 +52,7 @@ python3 PPOCRLabel.py --lang ch
 ```
 pip3 install pyqt5
 pip3 uninstall opencv-python # 由于mac版本的opencv与pyqt有冲突，需先手动卸载opencv
-pip3 install opencv-contrib-python-headless # 安装headless版本的open-cv
+pip3 install opencv-contrib-python-headless==4.2.0.32 # 安装headless版本的open-cv
 cd ./PPOCRLabel # 将目录切换到PPOCRLabel文件夹下
 python3 PPOCRLabel.py --lang ch
 ```
@@ -136,22 +136,22 @@ PPOCRLabel支持三种保存方式：
 
 ### 错误提示
 - 如果同时使用whl包安装了paddleocr，其优先级大于通过paddleocr.py调用PaddleOCR类，whl包未更新时会导致程序异常。
-  
+
 - PPOCRLabel**不支持对中文文件名**的图片进行自动标注。
 
 - 针对Linux用户：如果您在打开软件过程中出现**objc[XXXXX]**开头的错误，证明您的opencv版本太高，建议安装4.2版本：
     ```
     pip install opencv-python==4.2.0.32
     ```
-    
+
 - 如果出现 ```Missing string id``` 开头的错误，需要重新编译资源：
     ```
     pyrcc5 -o libs/resources.py resources.qrc
     ```
-    
-- 如果出现``` module 'cv2' has no attribute 'INTER_NEAREST'```错误，需要首先删除所有opencv相关包，然后重新安装headless版本的opencv
+
+- 如果出现``` module 'cv2' has no attribute 'INTER_NEAREST'```错误，需要首先删除所有opencv相关包，然后重新安装4.2.0.32版本的headless opencv
     ```
-    pip install opencv-contrib-python-headless
+    pip install opencv-contrib-python-headless==4.2.0.32
     ```
 
 ### 参考资料

README_ch.md

@@ -8,8 +8,8 @@ PaddleOCR同时支持动态图与静态图两种编程范式
 - 静态图版本：develop分支
 
 **近期更新**
+- 2021.2.1 [FAQ](./doc/doc_ch/FAQ.md)新增5个高频问题，总数162个，每周一都会更新，欢迎大家持续关注。
 - 2021.1.26,28,29 PaddleOCR官方研发团队带来技术深入解读三日直播课，1月26日、28日、29日晚上19:30，[直播地址](https://live.bilibili.com/21689802)
-- 2021.1.25 [FAQ](./doc/doc_ch/FAQ.md)新增5个高频问题，总数157个，每周一都会更新，欢迎大家持续关注。
 - 2021.1.21 更新多语言识别模型，目前支持语种超过27种，[多语言模型下载](./doc/doc_ch/models_list.md)，包括中文简体、中文繁体、英文、法文、德文、韩文、日文、意大利文、西班牙文、葡萄牙文、俄罗斯文、阿拉伯文等，后续计划可以参考[多语言研发计划](https://github.com/PaddlePaddle/PaddleOCR/issues/1048)
 - 2020.12.15 更新数据合成工具[Style-Text](./StyleText/README_ch.md)，可以批量合成大量与目标场景类似的图像，在多个场景验证，效果明显提升。
 - 2020.11.25 更新半自动标注工具[PPOCRLabel](./PPOCRLabel/README_ch.md)，辅助开发者高效完成标注任务，输出格式与PP-OCR训练任务完美衔接。

StyleText/engine/predictors.py

@@ -38,7 +38,15 @@ class StyleTextRecPredictor(object):
         self.std = config["Predictor"]["std"]
         self.expand_result = config["Predictor"]["expand_result"]
 
-    def predict(self, style_input, text_input):
+    def reshape_to_same_height(self, img_list):
+        h = img_list[0].shape[0]
+        for idx in range(1, len(img_list)):
+            new_w = round(1.0 * img_list[idx].shape[1] /
+                          img_list[idx].shape[0] * h)
+            img_list[idx] = cv2.resize(img_list[idx], (new_w, h))
+        return img_list
+
+    def predict_single_image(self, style_input, text_input):
         style_input = self.rep_style_input(style_input, text_input)
         tensor_style_input = self.preprocess(style_input)
         tensor_text_input = self.preprocess(text_input)
@@ -64,6 +72,21 @@ class StyleTextRecPredictor(object):
             "fake_bg": fake_bg,
         }
 
+    def predict(self, style_input, text_input_list):
+        if not isinstance(text_input_list, (tuple, list)):
+            return self.predict_single_image(style_input, text_input_list)
+
+        synth_result_list = []
+        for text_input in text_input_list:
+            synth_result = self.predict_single_image(style_input, text_input)
+            synth_result_list.append(synth_result)
+
+        for key in synth_result:
+            res = [r[key] for r in synth_result_list]
+            res = self.reshape_to_same_height(res)
+            synth_result[key] = np.concatenate(res, axis=1)
+        return synth_result
+
     def preprocess(self, img):
         img = (img.astype('float32') * self.scale - self.mean) / self.std
         img_height, img_width, channel = img.shape

StyleText/engine/synthesisers.py

@@ -12,6 +12,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 import os
+import numpy as np
+import cv2
 
 from utils.config import ArgsParser, load_config, override_config
 from utils.logging import get_logger
@@ -36,8 +38,9 @@ class ImageSynthesiser(object):
         self.predictor = getattr(predictors, predictor_method)(self.config)
 
     def synth_image(self, corpus, style_input, language="en"):
-        corpus, text_input = self.text_drawer.draw_text(corpus, language)
-        synth_result = self.predictor.predict(style_input, text_input)
+        corpus_list, text_input_list = self.text_drawer.draw_text(
+            corpus, language, style_input_width=style_input.shape[1])
+        synth_result = self.predictor.predict(style_input, text_input_list)
         return synth_result
 
 
@@ -59,12 +62,15 @@ class DatasetSynthesiser(ImageSynthesiser):
         for i in range(self.output_num):
             style_data = self.style_sampler.sample()
             style_input = style_data["image"]
-            corpus_language, text_input_label = self.corpus_generator.generate(
-            )
-            text_input_label, text_input = self.text_drawer.draw_text(
-                text_input_label, corpus_language)
+            corpus_language, text_input_label = self.corpus_generator.generate()
+            text_input_label_list, text_input_list = self.text_drawer.draw_text(
+                text_input_label,
+                corpus_language,
+                style_input_width=style_input.shape[1])
 
-            synth_result = self.predictor.predict(style_input, text_input)
+            text_input_label = "".join(text_input_label_list)
+
+            synth_result = self.predictor.predict(style_input, text_input_list)
             fake_fusion = synth_result["fake_fusion"]
             self.writer.save_image(fake_fusion, text_input_label)
         self.writer.save_label()

StyleText/engine/text_drawers.py

 from PIL import Image, ImageDraw, ImageFont
 import numpy as np
+import cv2
 from utils.logging import get_logger
 
 
@@ -28,7 +29,11 @@ class StdTextDrawer(object):
         else:
             return int((self.height - 4)**2 / font_height)
 
-    def draw_text(self, corpus, language="en", crop=True):
+    def draw_text(self,
+                  corpus,
+                  language="en",
+                  crop=True,
+                  style_input_width=None):
         if language not in self.support_languages:
             self.logger.warning(
                 "language {} not supported, use en instead.".format(language))
@@ -37,21 +42,43 @@ class StdTextDrawer(object):
             width = min(self.max_width, len(corpus) * self.height) + 4
         else:
             width = len(corpus) * self.height + 4
-        bg = Image.new("RGB", (width, self.height), color=(127, 127, 127))
-        draw = ImageDraw.Draw(bg)
-
-        char_x = 2
-        font = self.font_dict[language]
-        for i, char_i in enumerate(corpus):
-            char_size = font.getsize(char_i)[0]
-            draw.text((char_x, 2), char_i, fill=(0, 0, 0), font=font)
-            char_x += char_size
-            if char_x >= width:
-                corpus = corpus[0:i + 1]
-                self.logger.warning("corpus length exceed limit: {}".format(
-                    corpus))
+
+        if style_input_width is not None:
+            width = min(width, style_input_width)
+
+        corpus_list = []
+        text_input_list = []
+
+        while len(corpus) != 0:
+            bg = Image.new("RGB", (width, self.height), color=(127, 127, 127))
+            draw = ImageDraw.Draw(bg)
+            char_x = 2
+            font = self.font_dict[language]
+            i = 0
+            while i < len(corpus):
+                char_i = corpus[i]
+                char_size = font.getsize(char_i)[0]
+                # split when char_x exceeds char size and index is not 0 (at least 1 char should be wroten on the image)
+                if char_x + char_size >= width and i != 0:
+                    text_input = np.array(bg).astype(np.uint8)
+                    text_input = text_input[:, 0:char_x, :]
+
+                    corpus_list.append(corpus[0:i])
+                    text_input_list.append(text_input)
+                    corpus = corpus[i:]
+                    break
+                draw.text((char_x, 2), char_i, fill=(0, 0, 0), font=font)
+                char_x += char_size
+
+                i += 1
+            # the whole text is shorter than style input
+            if i == len(corpus):
+                text_input = np.array(bg).astype(np.uint8)
+                text_input = text_input[:, 0:char_x, :]
+
+                corpus_list.append(corpus[0:i])
+                text_input_list.append(text_input)
+                corpus = corpus[i:]
                 break
 
-        text_input = np.array(bg).astype(np.uint8)
-        text_input = text_input[:, 0:char_x, :]
-        return corpus, text_input
+        return corpus_list, text_input_list

configs/cls/cls_mv3.yml

@@ -7,7 +7,6 @@ Global:
   save_epoch_step: 3
   # evaluation is run every 5000 iterations after the 4000th iteration
   eval_batch_step: [0, 1000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
   cal_metric_during_train: True
   pretrained_model:
   checkpoints:
@@ -93,4 +92,4 @@ Eval:
     shuffle: False
     drop_last: False
     batch_size_per_card: 512
-    num_workers: 4
+    num_workers: 4
\ No newline at end of file

configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml

@@ -7,7 +7,10 @@ Global:
   save_epoch_step: 1200
   # evaluation is run every 5000 iterations after the 4000th iteration
   eval_batch_step: [3000, 2000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
+  # 1. If pretrained_model is saved in static mode, such as classification pretrained model
+  #    from static branch, load_static_weights must be set as True.
+  # 2. If you want to finetune the pretrained models we provide in the docs,
+  #    you should set load_static_weights as False.
   load_static_weights: True
   cal_metric_during_train: False
   pretrained_model: ./pretrain_models/MobileNetV3_large_x0_5_pretrained

configs/det/ch_ppocr_v2.0/ch_det_res18_db_v2.0.yml

@@ -7,7 +7,10 @@ Global:
   save_epoch_step: 1200
   # evaluation is run every 5000 iterations after the 4000th iteration
   eval_batch_step: [3000, 2000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
+  # 1. If pretrained_model is saved in static mode, such as classification pretrained model
+  #    from static branch, load_static_weights must be set as True.
+  # 2. If you want to finetune the pretrained models we provide in the docs,
+  #    you should set load_static_weights as False.
   load_static_weights: True
   cal_metric_during_train: False
   pretrained_model: ./pretrain_models/ResNet18_vd_pretrained

configs/det/det_mv3_db.yml

@@ -7,7 +7,10 @@ Global:
   save_epoch_step: 1200
   # evaluation is run every 2000 iterations
   eval_batch_step: [0, 2000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
+  # 1. If pretrained_model is saved in static mode, such as classification pretrained model
+  #    from static branch, load_static_weights must be set as True.
+  # 2. If you want to finetune the pretrained models we provide in the docs,
+  #    you should set load_static_weights as False.
   load_static_weights: True
   cal_metric_during_train: False
   pretrained_model: ./pretrain_models/MobileNetV3_large_x0_5_pretrained

configs/det/det_mv3_east.yml

@@ -7,7 +7,10 @@ Global:
   save_epoch_step: 1000
   # evaluation is run every 5000 iterations after the 4000th iteration
   eval_batch_step: [4000, 5000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
+  # 1. If pretrained_model is saved in static mode, such as classification pretrained model
+  #    from static branch, load_static_weights must be set as True.
+  # 2. If you want to finetune the pretrained models we provide in the docs,
+  #    you should set load_static_weights as False.
   load_static_weights: True
   cal_metric_during_train: False
   pretrained_model: ./pretrain_models/MobileNetV3_large_x0_5_pretrained

configs/det/det_r50_vd_db.yml

@@ -7,7 +7,10 @@ Global:
   save_epoch_step: 1200
   # evaluation is run every 2000 iterations
   eval_batch_step: [0,2000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
+  # 1. If pretrained_model is saved in static mode, such as classification pretrained model
+  #    from static branch, load_static_weights must be set as True.
+  # 2. If you want to finetune the pretrained models we provide in the docs,
+  #    you should set load_static_weights as False.
   load_static_weights: True
   cal_metric_during_train: False
   pretrained_model: ./pretrain_models/ResNet50_vd_ssld_pretrained

configs/det/det_r50_vd_east.yml

@@ -7,7 +7,10 @@ Global:
   save_epoch_step: 1000
   # evaluation is run every 5000 iterations after the 4000th iteration
   eval_batch_step: [4000, 5000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
+  # 1. If pretrained_model is saved in static mode, such as classification pretrained model
+  #    from static branch, load_static_weights must be set as True.
+  # 2. If you want to finetune the pretrained models we provide in the docs,
+  #    you should set load_static_weights as False.
   load_static_weights: True
   cal_metric_during_train: False
   pretrained_model: ./pretrain_models/ResNet50_vd_pretrained/

configs/det/det_r50_vd_sast_icdar15.yml

@@ -7,7 +7,10 @@ Global:
   save_epoch_step: 1000
   # evaluation is run every 5000 iterations after the 4000th iteration
   eval_batch_step: [4000, 5000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
+  # 1. If pretrained_model is saved in static mode, such as classification pretrained model
+  #    from static branch, load_static_weights must be set as True.
+  # 2. If you want to finetune the pretrained models we provide in the docs,
+  #    you should set load_static_weights as False.
   load_static_weights: True
   cal_metric_during_train: False
   pretrained_model: ./pretrain_models/ResNet50_vd_ssld_pretrained/

configs/det/det_r50_vd_sast_totaltext.yml

@@ -7,7 +7,10 @@ Global:
   save_epoch_step: 1000
   # evaluation is run every 5000 iterations after the 4000th iteration
   eval_batch_step: [4000, 5000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
+  # 1. If pretrained_model is saved in static mode, such as classification pretrained model
+  #    from static branch, load_static_weights must be set as True.
+  # 2. If you want to finetune the pretrained models we provide in the docs,
+  #    you should set load_static_weights as False.
   load_static_weights: True
   cal_metric_during_train: False
   pretrained_model: ./pretrain_models/ResNet50_vd_ssld_pretrained/

configs/rec/ch_ppocr_v2.0/rec_chinese_common_train_v2.0.yml

@@ -7,7 +7,6 @@ Global:
   save_epoch_step: 3
   # evaluation is run every 5000 iterations after the 4000th iteration
   eval_batch_step: [0, 2000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
   cal_metric_during_train: True
   pretrained_model:
   checkpoints:

configs/rec/ch_ppocr_v2.0/rec_chinese_lite_train_v2.0.yml

@@ -7,7 +7,6 @@ Global:
   save_epoch_step: 3
   # evaluation is run every 5000 iterations after the 4000th iteration
   eval_batch_step: [0, 2000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
   cal_metric_during_train: True
   pretrained_model:
   checkpoints: 

configs/rec/rec_icdar15_train.yml

@@ -7,7 +7,6 @@ Global:
   save_epoch_step: 3
   # evaluation is run every 2000 iterations
   eval_batch_step: [0, 2000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
   cal_metric_during_train: True
   pretrained_model:
   checkpoints:

configs/rec/rec_mv3_none_bilstm_ctc.yml

@@ -7,7 +7,6 @@ Global:
   save_epoch_step: 3
   # evaluation is run every 2000 iterations
   eval_batch_step: [0, 2000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
   cal_metric_during_train: True
   pretrained_model:
   checkpoints:

configs/rec/rec_mv3_none_none_ctc.yml

@@ -7,7 +7,6 @@ Global:
   save_epoch_step: 3
   # evaluation is run every 2000 iterations
   eval_batch_step: [0, 2000]
-  # if pretrained_model is saved in static mode, load_static_weights must set to True
   cal_metric_during_train: True
   pretrained_model:
   checkpoints: