Skip to content

GitLab

"test/srt/quant/test_w8a8_quantization.py" did not exist on "a7c47e0f028c2a9e67cbc99ab67692ec765d3dd0"
Switch branch/tag
  1. 29 May, 2020 2 commits
  3. 28 May, 2020 1 commit
    • Reed Wanderman-Milne's avatar
      Use float32 activation in Transformer. · 94b1efc1
      Reed Wanderman-Milne authored 
      Float32 is used if the model uses mixed precision with bfloat16. Float16 activation are unchanged.

The motivation is that BERT with the LAMB optimizer with a gelu activation has an unstable loss when gelu is in bfloat16. Unfortunately, it is not easy to check if the LAMB optimizer and gelu is used, and perhaps there are other cases that work better with float32 activations instead of bfloat16 activations, so we always do the activation in float32 instead of bfloat16.

PiperOrigin-RevId: 313618322
      94b1efc1
  5. 19 May, 2020 1 commit
  7. 18 May, 2020 1 commit
  9. 17 May, 2020 1 commit
  11. 12 May, 2020 3 commits
  13. 10 May, 2020 1 commit
  15. 05 May, 2020 1 commit
  17. 21 Apr, 2020 1 commit
  19. 20 Apr, 2020 1 commit
  21. 19 Apr, 2020 1 commit
  23. 17 Apr, 2020 4 commits
  25. 15 Apr, 2020 1 commit
  27. 13 Apr, 2020 1 commit
  29. 08 Apr, 2020 1 commit
  31. 01 Apr, 2020 1 commit
  33. 31 Mar, 2020 2 commits
  35. 27 Mar, 2020 1 commit
  37. 12 Mar, 2020 1 commit
  39. 09 Mar, 2020 1 commit
  41. 05 Mar, 2020 2 commits
  43. 03 Mar, 2020 2 commits
  45. 26 Feb, 2020 3 commits
  47. 25 Feb, 2020 1 commit
  49. 21 Feb, 2020 1 commit
  51. 08 Feb, 2020 1 commit
  53. 21 Jan, 2020 1 commit
    • Hongkun Yu's avatar
      Remove compute_output_shape. · 0e0a94a6
      Hongkun Yu authored 
      Keras: "manual" shape inference is only required if the layer is dynamic (otherwise we use TF's static shape inference capabilities)

PiperOrigin-RevId: 290821518
      0e0a94a6
  55. 08 Jan, 2020 1 commit
  57. 19 Dec, 2019 1 commit