Skip to content

GitLab

Switch branch/tag
  1. 03 Jun, 2020 5 commits
    • Hongkun Yu's avatar
      Revert "Revert "Add relative positional embedding to KerasBERT (#8617)" (#8619)" · a6c0e677
      Hongkun Yu authored 
      This reverts commit 4bb13e61.
      a6c0e677
    • Hongkun Yu's avatar
      Revert "Add relative positional embedding to KerasBERT (#8617)" (#8619) · 4bb13e61
      Hongkun Yu authored 
      This reverts commit c3c2386c.
      4bb13e61
    • xinliupitt's avatar
      Add relative positional embedding to KerasBERT (#8617) · c3c2386c
      xinliupitt authored 
      * root dir

* zone updated

* print mask

* preview emb

* tf print

* input only

* emb

* tf print

* emb after mask

* masked_softmax print

* print scores

* multi folder

* first pos emb

* check input shape

* add test temp

* import math

* two classes

* prints

* all get_pos replace

* make time scale private

* pos emb comments

* print input

* embedding_inputs

* tf shape

* dimention list

* tf_util

* print tf_util

* concise

* transformer pos change to layer

* keep length var

* length as input

* None as input

* print time signal

* print time signal

* remove print

* test input shape

* double check shape

* double check shape

* double check shape

* more test

* shape check

* shape check

* print 97 info

* print 97 info new

* test if sam

* assert same

* remove assert

* tf print same

* tf print diff

* output example

* output example

* output example

* formal test

* formal test length

* raise valurerror

* test valurerror

* double check

* comments

* remove prints

* rename relative

* delet naive test

* delete docs in xinliu branch

* code reformat

* import order

* indentation fix

* more files

* adjust char number

* disable not callable

* comment to length

* error of length unequal to input_shape

* root dir

* zone updated

* print mask

* preview emb

* tf print

* input only

* emb

* tf print

* emb after mask

* masked_softmax print

* print scores

* multi folder

* remove docs

* remove prints

* root dir

* zone updated

* print mask

* preview emb

* tf print

* input only

* emb

* tf print

* emb after mask

* masked_softmax print

* print scores

* multi folder

* remove docs

* apply revised 3 files

* rm prints
      c3c2386c
    • Maxim Neumann's avatar
      Update documentation and tests of model and network for regression tasks. · d90bed2e
      Maxim Neumann authored 
      PiperOrigin-RevId: 314486753
      d90bed2e
    • Tianqi Liu's avatar
      Internal change · 20897493
      Tianqi Liu authored 
      PiperOrigin-RevId: 314451720
      20897493
  3. 02 Jun, 2020 2 commits
    • xinliupitt's avatar
      Add relative positional embedding to KerasBERT (#8606) · 2db2501b
      xinliupitt authored 
      * root dir

* zone updated

* print mask

* preview emb

* tf print

* input only

* emb

* tf print

* emb after mask

* masked_softmax print

* print scores

* multi folder

* first pos emb

* check input shape

* add test temp

* import math

* two classes

* prints

* all get_pos replace

* make time scale private

* pos emb comments

* print input

* embedding_inputs

* tf shape

* dimention list

* tf_util

* print tf_util

* concise

* transformer pos change to layer

* keep length var

* length as input

* None as input

* print time signal

* print time signal

* remove print

* test input shape

* double check shape

* double check shape

* double check shape

* more test

* shape check

* shape check

* print 97 info

* print 97 info new

* test if sam

* assert same

* remove assert

* tf print same

* tf print diff

* output example

* output example

* output example

* formal test

* formal test length

* raise valurerror

* test valurerror

* double check

* comments

* remove prints

* rename relative

* delet naive test

* delete docs in xinliu branch

* code reformat

* import order

* indentation fix

* more files

* adjust char number

* disable not callable

* comment to length

* error of length unequal to input_shape
      2db2501b
    • Chen Chen's avatar
      Internal change · 8eb91073
      Chen Chen authored 
      PiperOrigin-RevId: 314373769
      8eb91073
  5. 30 May, 2020 1 commit
  7. 29 May, 2020 2 commits
  9. 28 May, 2020 1 commit
    • Reed Wanderman-Milne's avatar
      Use float32 activation in Transformer. · 94b1efc1
      Reed Wanderman-Milne authored 
      Float32 is used if the model uses mixed precision with bfloat16. Float16 activation are unchanged.

The motivation is that BERT with the LAMB optimizer with a gelu activation has an unstable loss when gelu is in bfloat16. Unfortunately, it is not easy to check if the LAMB optimizer and gelu is used, and perhaps there are other cases that work better with float32 activations instead of bfloat16 activations, so we always do the activation in float32 instead of bfloat16.

PiperOrigin-RevId: 313618322
      94b1efc1
  11. 21 May, 2020 2 commits
  13. 19 May, 2020 1 commit
  15. 18 May, 2020 1 commit
  17. 17 May, 2020 1 commit
  19. 15 May, 2020 1 commit
  21. 14 May, 2020 2 commits
  23. 13 May, 2020 2 commits
    • Chen Chen's avatar
      Internal change · 3c5d118c
      Chen Chen authored 
      PiperOrigin-RevId: 311428193
      3c5d118c
    • Scott Zhu's avatar
      Restructure the Keras class hierarchy for Network, Model and Sequential. · d08af90d
      Scott Zhu authored 
      The intention of this change is to reduce the code complexity within Keras class, especially for Network, which currently contains logic for both subclass Model and functional Model.

After this change, the subclass model and functional model become individual class and become self contained.

1. Model is now the base class for subclass model. It doesn't contains network structure management, and the topology will be created within __init__ and __call__, which is for user to implement. It also contains compile/fit/eval/predict, which is the basic functionality for model training.

2. Functional is created based on existing Network class. It extends the Model, which allows it leverage compile/fit/eval/predict. In addition, it also take input/output as init parameter and manage the network topology.

3. Sequential model is now a subclass of Functional, since it will use Functional's method to manage it topology (layer stacking).

Model(input, output) will create a Functional under the hood, and behave the same way as before.

PiperOrigin-RevId: 311232972
      d08af90d
  25. 12 May, 2020 3 commits
  27. 10 May, 2020 1 commit
  29. 09 May, 2020 1 commit
  31. 08 May, 2020 1 commit
  33. 05 May, 2020 1 commit
  35. 25 Apr, 2020 1 commit
  37. 21 Apr, 2020 1 commit
  39. 20 Apr, 2020 2 commits
  41. 19 Apr, 2020 1 commit
  43. 17 Apr, 2020 4 commits
  45. 15 Apr, 2020 1 commit
  47. 13 Apr, 2020 1 commit
  49. 08 Apr, 2020 1 commit