"official/vision/examples/starter/example_task.py" did not exist on "9c8cbd0ce88a59565369934c5e3e7ca688302a7f"
  • Deepak Narayanan's avatar
    Support for pipeline parallelism in T5 model · 46c74b4c
    Deepak Narayanan authored
    - Accumulate encoder hidden state gradient to handle skip connection
    - Correctly compute the number of layers in encoder / decoder for T5 model
    - Ensure e weights are initialized the same way in embeddings
    - Synchronize embedding gradients across encoder and decoder for T5 model
    - Support for checkpoint loading and saving
    46c74b4c
initialize.py 8.52 KB