Skip to content

GitLab

Switch branch/tag
  1. 09 Oct, 2019 1 commit
    • Reed Wanderman-Milne's avatar
      Simply LayerNorm mixed precision logic. · 0257b276
      Reed Wanderman-Milne authored 
      Instead of needing to ensure variables are float32, casting inputs to float32, etc, instead dtype="float32" is passed to the layer constructor, which will do all that logic automatically.

The only difference is the output of LayerNorm is now float32 instead of float16, so an extra cast is needed elsewhere.

PiperOrigin-RevId: 273833286
      0257b276
  3. 07 Oct, 2019 1 commit
  5. 18 Sep, 2019 1 commit
  7. 17 Sep, 2019 1 commit
  9. 16 Sep, 2019 1 commit
  11. 09 Sep, 2019 2 commits
  13. 05 Sep, 2019 1 commit
  15. 04 Sep, 2019 1 commit
  17. 03 Sep, 2019 1 commit
  19. 30 Aug, 2019 2 commits
  21. 29 Aug, 2019 1 commit
  23. 28 Aug, 2019 2 commits
  25. 27 Aug, 2019 1 commit
  27. 26 Aug, 2019 1 commit
  29. 23 Aug, 2019 2 commits
  31. 22 Aug, 2019 3 commits
  33. 21 Aug, 2019 2 commits
  35. 20 Aug, 2019 2 commits
  37. 19 Aug, 2019 3 commits
  39. 16 Aug, 2019 2 commits
  41. 15 Aug, 2019 3 commits
  43. 09 Aug, 2019 1 commit
  45. 08 Aug, 2019 1 commit
    • Reed's avatar
      Fix fp16 Transformer model. (#7402) · 58340818
      Reed authored 
      Also, do Transformer inference in fp16, as well as training, when --dtype=fp16. In TF 2, layers now cannot run in multiple different dtypes, so we must use the same dtype for training and inference.
      58340818
  47. 07 Aug, 2019 1 commit
    • Hongkun Yu's avatar
      Merged commit includes the following changes: (#7398) · 45b708d4
      Hongkun Yu authored 
      262039434  by A. Unique TensorFlower<gardener@tensorflow.org>:

    Internal change

262024241  by hongkuny<hongkuny@google.com>:

    Adds __init__.py

--
262021128  by isaprykin<isaprykin@google.com>:

    Internal change

PiperOrigin-RevId: 262039434
      45b708d4
  49. 06 Aug, 2019 1 commit
  51. 05 Aug, 2019 1 commit
    • Igor's avatar
      Fix the ValueError: Error when checking model input on the new codepath (#7382) · ca7d215d
      Igor authored 
      * Fix the ValueError: Error when checking model input on the new codepath

Fixes the following error:

  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training.py", line 2428, in _standardize_user_data
    exception_prefix='input')
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/engine/training_utils.py", line 530, in standardize_input_data
    str(len(data)) + ' arrays: ' + str(data)[:200] + '...')
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays: [<tf.Tensor 'cond_8/Identity:0' shape=(None, None) dtype=int64>]..

Tested and reproduced by running trasformer_main_test (thanks to whoever wrote it, phew!)

* Remove the unnecessary TODO.
      ca7d215d
  53. 01 Aug, 2019 1 commit
    • Haoyu Zhang's avatar
      Merged commit includes the following changes: (#7354) · dc4c5f1a
      Haoyu Zhang authored 
      261171038  by gjn<gjn@google.com>:

    Remove weight_decay_rate 0 early exit check

    Removing this code path should be fine since this was actually not doing
    what it meant to do. Since weight_decay_rate is actually a tensor, the
    equality check was only looking at the id of the object and comparing to
    0. This should never be true. Evaluating a tensor is also not what we
    want to do at this point of the code. Thus it should be fine to simply
    remove this code.

--
261169862  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261153520  by haoyuzhang<haoyuzhang@google.com>:

    Internal change

261140302  by hongkuny<hongkuny@google.com>:

    Clean up

--

PiperOrigin-RevId: 261171038
      dc4c5f1a