Commits · fc724b3e2b00a24d4ab5c9fad937c6c80edaa487 · ModelZoo / ResNet50_tensorflow

02 Jun, 2020 5 commits
- A minor change to model_training_utils of BERT. · fc724b3e
  A. Unique TensorFlower authored Jun 02, 2020
```
PiperOrigin-RevId: 314417394
```
  fc724b3e
- Internal change · 3d99cc89
  Maxim Neumann authored Jun 02, 2020
```
PiperOrigin-RevId: 314412294
```
  3d99cc89
- write_meta_graph=False as we do not care the graph. · 7580f0a2
  Hongkun Yu authored Jun 02, 2020
```
PiperOrigin-RevId: 314394246
```
  7580f0a2
- Internal change · 8eb91073
  Chen Chen authored Jun 02, 2020
```
PiperOrigin-RevId: 314373769
```
  8eb91073
- Add unit tests for bert_models.py · c25c3e88
  A. Unique TensorFlower authored Jun 01, 2020
```
PiperOrigin-RevId: 314270569
```
  c25c3e88
01 Jun, 2020 1 commit
- [XLNET] Fix tf.data bad usages. · 2488f2c7
  Hongkun Yu authored Jun 01, 2020
```
PiperOrigin-RevId: 314177321
```
  2488f2c7
30 May, 2020 1 commit
- adds a ClassificationHead layer · 2cf59d12
  Hongkun Yu authored May 29, 2020
```
PiperOrigin-RevId: 313906815
```
  2cf59d12
29 May, 2020 2 commits
- Proposes the full functionality of MultiHeadAttention layer. This change first... · 8a13ca4e
  Hongkun Yu authored May 29, 2020
```
Proposes the full functionality of MultiHeadAttention layer. This change first goes to model garden NLP library.

PiperOrigin-RevId: 313847485
```
  8a13ca4e
- Internal change · f2f26a8b
  Chen Chen authored May 29, 2020
```
PiperOrigin-RevId: 313812017
```
  f2f26a8b
28 May, 2020 3 commits

Deprecate old customized training loop for run_classifier.py as compile/fit... · abf60128

Hongkun Yu authored May 28, 2020

Deprecate old customized training loop for run_classifier.py as compile/fit fully satisfy needs/performance.

PiperOrigin-RevId: 313660745

abf60128

Use float32 activation in Transformer. · 94b1efc1

Reed Wanderman-Milne authored May 28, 2020

Float32 is used if the model uses mixed precision with bfloat16. Float16 activation are unchanged.

The motivation is that BERT with the LAMB optimizer with a gelu activation has an unstable loss when gelu is in bfloat16. Unfortunately, it is not easy to check if the LAMB optimizer and gelu is used, and perhaps there are other cases that work better with float32 activations instead of bfloat16 activations, so we always do the activation in float32 instead of bfloat16.

PiperOrigin-RevId: 313618322

94b1efc1

Internal change · fbec2dbe
A. Unique TensorFlower authored May 27, 2020
```
PiperOrigin-RevId: 313536026
```
fbec2dbe

26 May, 2020 2 commits
- Add learning_rate to the summary. · ee760015
  Maxim Neumann authored May 26, 2020
```
PiperOrigin-RevId: 313148142
```
  ee760015
- Add --sub_model_export_name to run_classifier and run_squad. · 38800db3
  André Susano Pinto authored May 26, 2020
```
This allows one to finetune a BERT model into a task before using it for
another task. E.g. SQuAD before finetune another QA type of tasks.

PiperOrigin-RevId: 313145768
```
  38800db3
24 May, 2020 2 commits
- [Clean up]: remove is_v2() check inside transformer. · b29fe6b7
  Hongkun Yu authored May 24, 2020
```
PiperOrigin-RevId: 312988874
```
  b29fe6b7
- Update BERT read me to show how to make predictions on the test set. · a2ed0385
  Hongkun Yu authored May 23, 2020
```
PiperOrigin-RevId: 312939899
```
  a2ed0385
23 May, 2020 2 commits
- Internal change · ca2e6ae0
  Hongkun Yu authored May 23, 2020
```
PiperOrigin-RevId: 312923051
```
  ca2e6ae0
- [Clean up] Remove adhoc checkpoint callback. · 2222cefc
  Hongkun Yu authored May 23, 2020
```
PiperOrigin-RevId: 312889153
```
  2222cefc
21 May, 2020 3 commits
- Internal change · ff138931
  A. Unique TensorFlower authored May 21, 2020
```
PiperOrigin-RevId: 312754139
```
  ff138931
- Makes token embedding projection consistent between Albert and BERT · b1eddf4f
  A. Unique TensorFlower authored May 21, 2020
```
PiperOrigin-RevId: 312751112
```
  b1eddf4f
- Transformer Encoder: when embedding width differs from hidden size, add a... · 26565d0d
  Hongkun Yu authored May 21, 2020
```
Transformer Encoder: when embedding width differs from hidden size, add a projection to hidden size.

PiperOrigin-RevId: 312708922
```
  26565d0d
20 May, 2020 2 commits
- Enable NHNet trainer tests on mirrored strategies. · 09b09eb5
  Hongkun Yu authored May 20, 2020
```
PiperOrigin-RevId: 312515585
```
  09b09eb5
- bugfix: unused and incorrect variable name · 31552f02
  Xiaoquan Kong authored May 20, 2020
  
  31552f02
19 May, 2020 7 commits
- Allows to customize feedforward layer in transformer_scaffold. · d9c4e454
  Chen Chen authored May 19, 2020
```
PiperOrigin-RevId: 312366167
```
  d9c4e454
- [Clean up] Remove enable_eager in the session config: Model garden is TF2 only now. · c2666cea
  Hongkun Yu authored May 19, 2020
```
Remove is_v2_0

PiperOrigin-RevId: 312336907
```
  c2666cea
- Internal change · 4ec2ee97
  Chen Chen authored May 19, 2020
```
PiperOrigin-RevId: 312313738
```
  4ec2ee97
- When not specified by user, make steps_per_loop be dependent on the devices available. · 866176ea
  André Susano Pinto authored May 19, 2020
```
A default of 1 all the times is bad for TPU users which end up not using the device effectively.
A larger default all the times is bad for GPU users.

So compromise and make this dependent on the devices available.

PiperOrigin-RevId: 312230371
```
  866176ea
- Internal change · 2a3971dd
  A. Unique TensorFlower authored May 18, 2020
```
PiperOrigin-RevId: 312209760
```
  2a3971dd
- Internal change · f5014889
  Ruoxin Sang authored May 18, 2020
```
PiperOrigin-RevId: 312194218
```
  f5014889
- Handle corner case in squad_lib when best_non_null_entry is None. · 25160730
  Saurabh Saxena authored May 18, 2020
```
PiperOrigin-RevId: 312193729
```
  25160730
18 May, 2020 3 commits
- Internal Change · 6f7a682d
  Chen Chen authored May 18, 2020
```
PiperOrigin-RevId: 312116965
```
  6f7a682d
- Add NHNet quality benchmark. · e81d423d
  Hongkun Yu authored May 18, 2020
```
PiperOrigin-RevId: 312099777
```
  e81d423d
- Internal change · 457064e1
  A. Unique TensorFlower authored May 18, 2020
```
PiperOrigin-RevId: 312096778
```
  457064e1
17 May, 2020 1 commit
- Update nlp.modeling.layers.ReZeroTransformer, to have the same interface with... · ef61bae9
  A. Unique TensorFlower authored May 16, 2020
```
Update nlp.modeling.layers.ReZeroTransformer, to have the same interface with nlp.modeling.layers.Transformer

PiperOrigin-RevId: 311937563
```
  ef61bae9
15 May, 2020 1 commit
- Internal change · 840a493a
  Hongkun Yu authored May 15, 2020
```
PiperOrigin-RevId: 311773503
```
  840a493a
14 May, 2020 5 commits
- Add bert's instruction about using a checkpoint on local storage · 293b173f
  nuka137 authored May 15, 2020
  
  293b173f
- Internal change · 3dccfae1
  Hongkun Yu authored May 14, 2020
```
PiperOrigin-RevId: 311602262
```
  3dccfae1
- Internal change · 8c408bbe
  Chen Chen authored May 14, 2020
```
PiperOrigin-RevId: 311597242
```
  8c408bbe
- Deprecate run_customized_training_loop() method. · 7cdb82e3
  Chen Chen authored May 14, 2020
```
PiperOrigin-RevId: 311577663
```
  7cdb82e3
- Add network and BERT model to perform per-token classification (e.g., for... · 25542676
  Jeremiah Harmsen authored May 14, 2020
```
Add network and BERT model to perform per-token classification (e.g., for named entity recognition tasks).

PiperOrigin-RevId: 311480326
```
  25542676