Skip to content

GitLab

"megatron/model/fused_bias_gelu.py" did not exist on "8ec5d6780e546aaa6338b2d3271f291d4ecc3127"
Switch branch/tag
  1. 30 Jul, 2021 1 commit
    • Deepak Narayanan's avatar
      Support for pipeline parallelism in T5 model · 46c74b4c
      Deepak Narayanan authored 
      - Accumulate encoder hidden state gradient to handle skip connection
- Correctly compute the number of layers in encoder / decoder for T5 model
- Ensure e weights are initialized the same way in embeddings
- Synchronize embedding gradients across encoder and decoder for T5 model
- Support for checkpoint loading and saving
      46c74b4c
  3. 18 May, 2021 1 commit
  5. 27 Jan, 2021 1 commit
  7. 22 Jan, 2021 1 commit
  9. 08 Jan, 2021 1 commit