Unverified Commit 103e614b authored by Harry Mellor's avatar Harry Mellor Committed by GitHub
Browse files

Fix pipeline parallel with embed scaling in the Transformers modelling backend (#35094)


Signed-off-by: default avatarHarry Mellor <19981378+hmellor@users.noreply.github.com>
parent 54e2f83d
......@@ -191,6 +191,7 @@ class Base(
self.attention_instances = self.create_attention_instances()
# Input embeddings
self.embed_scale = None
input_embeddings = self.model.get_input_embeddings()
if not isinstance(input_embeddings, PPMissingLayer):
# Some models scale embeddings inside the input embedding layer
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment