Merge pull request #184 from tink2123/add_cosine_decay

add cosine decay

configs/rec/rec_icdar15_train.yml

@@ -41,3 +41,7 @@ Optimizer:
   base_lr: 0.0005
   beta1: 0.9
   beta2: 0.999
+  decay:
+    function: cosine_decay
+    step_each_epoch: 20
+    total_epoch: 1000

doc/doc_ch/config.md

@@ -46,3 +46,18 @@
 |      img_set_dir          |    数据集路径             |       ./train_data        |                \                 |
 |      label_file_path      |    数据标签路径           |       ./train_data/rec_gt_train.txt| \    |
 |      infer_img            |    预测图像文件夹路径     |       ./infer_img | \|
+
+## 配置文件 Optimizer 系列参数介绍
+
+以 `rec_icdar15_train.yml` 为例
+
+|         字段             |            用途            |      默认值        |            备注             |
+| :---------------------: |  :---------------------:   | :--------------:  |   :--------------------:   |
+|         function        |         选择优化器          |  pocr.optimizer,AdamDecay  |  目前只支持Adam方式  |
+|         base_lr         |      设置初始学习率          |       0.0005      |               \             |
+|         beta1           |    设置一阶矩估计的指数衰减率  |       0.9         |               \             |
+|         beta2           |    设置二阶矩估计的指数衰减率  |     0.999         |               \             |
+|         decay           |         是否使用decay       |    \              |               \             |
+|      function(decay)    |         设置decay方式       |   cosine_decay    |       目前只支持cosin_decay  |
+|      step_each_epoch    |      每个epoch包含多少次迭代  |         20       | 计算方式：total_image_num / (batch_size_per_card * card_size) |
+|        total_epoch      |    总共迭代多少个epoch        |       1000      | 与Global.epoch_num 一致        |

doc/doc_en/config_en.md

@@ -46,3 +46,18 @@ Take `rec_chinese_reader.yml` as an example:
 |      img_set_dir          |    Image folder path             |       ./train_data        |                \                 |
 |      label_file_path      |    Groundtruth file path           |       ./train_data/rec_gt_train.txt| \    |
 |      infer_img            |    Result folder path     |       ./infer_img | \|
+
+## Introduction to Optimizer parameters of Configuration file
+
+Take `rec_icdar15_train.yml` as an example:
+
+|         Parameter             |            Use          |      Default        |            None             |
+| :---------------------: |  :---------------------:   | :--------------:  |   :--------------------:   |
+|         function        |         Select Optimizer function          |  pocr.optimizer,AdamDecay  |  Only support Adam  |
+|         base_lr         |      Set the base lr          |       0.0005      |               \             |
+|         beta1           |    Set the exponential decay rate for the 1st moment estimates  |       0.9         |               \             |
+|         beta2           |    Set the exponential decay rate for the 2nd moment estimates  |     0.999         |               \             |
+|         decay           |         Whether to use decay       |    \              |               \             |
+|      function(decay)    |         Set the decay function       |   cosine_decay    |           Only support cosine_decay            |
+|      step_each_epoch    |      The number of steps in an epoch.  |         20       | Calculation ：total_image_num / (batch_size_per_card * card_size) |
+|        total_epoch      |    The number of epochs      |       1000      | Consistent with Global.epoch_num      |

ppocr/optimizer.py

@@ -15,6 +15,9 @@ from __future__ import absolute_import
 from __future__ import division
 from __future__ import print_function
 import paddle.fluid as fluid
+from ppocr.utils.utility import initial_logger
+
+logger = initial_logger()
 
 
 def AdamDecay(params, parameter_list=None):
@@ -28,6 +31,18 @@ def AdamDecay(params, parameter_list=None):
     base_lr = params['base_lr']
     beta1 = params['beta1']
     beta2 = params['beta2']
+    if 'decay' in params:
+        params = params['decay']
+        decay_mode = params['function']
+        step_each_epoch = params['step_each_epoch']
+        total_epoch = params['total_epoch']
+        if decay_mode == "cosine_decay":
+            base_lr = fluid.layers.cosine_decay(
+                learning_rate=base_lr,
+                step_each_epoch=step_each_epoch,
+                epochs=total_epoch)
+        else:
+            logger.info("Only support Cosine decay currently")
     optimizer = fluid.optimizer.Adam(
         learning_rate=base_lr,
         beta1=beta1,