GitLab

(1) call() consume kwargs and implement _build_from_signature (layers added inside init_scope)
(2) make build/call_attention as public.

PiperOrigin-RevId: 322454619

PiperOrigin-RevId: 320240466

PiperOrigin-RevId: 317596394

PiperOrigin-RevId: 317330705

Proposes the full functionality of MultiHeadAttention layer. This change first goes to model garden NLP library.

PiperOrigin-RevId: 313847485

Float32 is used if the model uses mixed precision with bfloat16. Float16 activation are unchanged.

The motivation is that BERT with the LAMB optimizer with a gelu activation has an unstable loss when gelu is in bfloat16. Unfortunately, it is not easy to check if the LAMB optimizer and gelu is used, and perhaps there are other cases that work better with float32 activations instead of bfloat16 activations, so we always do the activation in float32 instead of bfloat16.

PiperOrigin-RevId: 313618322

PiperOrigin-RevId: 311165658

PiperOrigin-RevId: 311072125

PiperOrigin-RevId: 310767440

PiperOrigin-RevId: 310032518

PiperOrigin-RevId: 307689094

PiperOrigin-RevId: 304222530

PiperOrigin-RevId: 303407939

PiperOrigin-RevId: 299901483

PiperOrigin-RevId: 298692558

Removed with plans to re-add later once the feature stabilizes more.

PiperOrigin-RevId: 298486867

PiperOrigin-RevId: 297366405

PiperOrigin-RevId: 297366158

Application in graph mode still leads to some crashes.

PiperOrigin-RevId: 297144398

To debug the tf.function this API can be used: https://www.tensorflow.org/api_docs/python/tf/config/experimental_run_functions_eagerly

PiperOrigin-RevId: 296458870

PiperOrigin-RevId: 293958491

Keras: "manual" shape inference is only required if the layer is dynamic (otherwise we use TF's static shape inference capabilities)

PiperOrigin-RevId: 290821518

PiperOrigin-RevId: 281473612

PiperOrigin-RevId: 280123567

- Update classifier example.
- Add new converted checkpoints.
- Update benchmark,

PiperOrigin-RevId: 279762797