Skip to content

GitLab

Switch branch/tag
  1. 29 May, 2020 2 commits
  3. 28 May, 2020 3 commits
    • Hongkun Yu's avatar
      Deprecate old customized training loop for run_classifier.py as compile/fit... · abf60128
      Hongkun Yu authored 
      Deprecate old customized training loop for run_classifier.py as compile/fit fully satisfy needs/performance.

PiperOrigin-RevId: 313660745
      abf60128
    • Reed Wanderman-Milne's avatar
      Use float32 activation in Transformer. · 94b1efc1
      Reed Wanderman-Milne authored 
      Float32 is used if the model uses mixed precision with bfloat16. Float16 activation are unchanged.

The motivation is that BERT with the LAMB optimizer with a gelu activation has an unstable loss when gelu is in bfloat16. Unfortunately, it is not easy to check if the LAMB optimizer and gelu is used, and perhaps there are other cases that work better with float32 activations instead of bfloat16 activations, so we always do the activation in float32 instead of bfloat16.

PiperOrigin-RevId: 313618322
      94b1efc1
    • A. Unique TensorFlower's avatar
      Internal change · fbec2dbe
      A. Unique TensorFlower authored 
      PiperOrigin-RevId: 313536026
      fbec2dbe
  5. 26 May, 2020 2 commits
  7. 24 May, 2020 2 commits
  9. 23 May, 2020 2 commits
  11. 21 May, 2020 3 commits
  13. 20 May, 2020 2 commits
  15. 19 May, 2020 7 commits
  17. 18 May, 2020 3 commits
  19. 17 May, 2020 1 commit
  21. 15 May, 2020 1 commit
  23. 14 May, 2020 6 commits
  25. 13 May, 2020 2 commits
    • Chen Chen's avatar
      Internal change · 3c5d118c
      Chen Chen authored 
      PiperOrigin-RevId: 311428193
      3c5d118c
    • Scott Zhu's avatar
      Restructure the Keras class hierarchy for Network, Model and Sequential. · d08af90d
      Scott Zhu authored 
      The intention of this change is to reduce the code complexity within Keras class, especially for Network, which currently contains logic for both subclass Model and functional Model.

After this change, the subclass model and functional model become individual class and become self contained.

1. Model is now the base class for subclass model. It doesn't contains network structure management, and the topology will be created within __init__ and __call__, which is for user to implement. It also contains compile/fit/eval/predict, which is the basic functionality for model training.

2. Functional is created based on existing Network class. It extends the Model, which allows it leverage compile/fit/eval/predict. In addition, it also take input/output as init parameter and manage the network topology.

3. Sequential model is now a subclass of Functional, since it will use Functional's method to manage it topology (layer stacking).

Model(input, output) will create a Functional under the hood, and behave the same way as before.

PiperOrigin-RevId: 311232972
      d08af90d
  27. 12 May, 2020 4 commits