GitLab

* Fix t5 shard on TPU Pods

The current script doesn't work properly on a TPU pod because the global batch is not divided correctly per host.
This pull request fixes this issue by dividing the global batch to each host before it is shared on each host.

* fix style
Co-authored-by: ahmed-elnaggar <ahmed.elnaggar@allianz.com>