Skip to content

GitLab

"git@developer.sourcefind.cn:OpenDAS/cubvh.git" did not exist on "d2efe16db959ff44c2aa08caec6af417eeb56fe5"
Commit d92f67be authored by A. Unique TensorFlower's avatar A. Unique TensorFlower
Browse files

Minor cleanup in `projects/mobilebert/distillation.py`. 

PiperOrigin-RevId: 367244904
parent 75b52393
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 1 deletion
official/nlp/projects/mobilebert/distillation.py
View file @ d92f67be
...@@ -378,7 +378,6 @@ class BertDistillationTask(policies.ProgressivePolicy, base_task.Task): ...@@ -378,7 +378,6 @@ class BertDistillationTask(policies.ProgressivePolicy, base_task.Task):
# Shape: [batch, max_predictions_per_seq, vocab_size] # Shape: [batch, max_predictions_per_seq, vocab_size]
lm_label = tf.one_hot(indices=lm_label, depth=vocab_size, on_value=1.0, lm_label = tf.one_hot(indices=lm_label, depth=vocab_size, on_value=1.0,
off_value=0.0, axis=-1, dtype=tf.float32) off_value=0.0, axis=-1, dtype=tf.float32)
lm_label_weights = labels['masked_lm_weights']
gt_ratio = distill_config.distill_ground_truth_ratio gt_ratio = distill_config.distill_ground_truth_ratio
if gt_ratio != 1.0: if gt_ratio != 1.0:
teacher_mlm_logits = outputs['teacher_pretrainer_output']['mlm_logits'] teacher_mlm_logits = outputs['teacher_pretrainer_output']['mlm_logits']
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment