Skip to content

GitLab

"docs/en/advanced_guides/datasets/kitti_det.md" did not exist on "75c5c6cd590386bd1539a686c5fd2cc45c5480d5"
Switch branch/tag
Find file Normal viewHistoryPermalink
README.md 334 Bytes
Newer Older
xingjinliang's avatar
升级megatron_v0.10  
xingjinliang committed
1
2
3
4
5
6
7
8
9
10
11 
## How to use pytorch FSDP2?

Add these flag to enable Torch FSDP2.

```
--use-torch-fsdp2
--no-gradient-accumulation-fusion
--ckpt-format torch_dist
```

It is worth noting that CUDA_MAX_CONNECTIONS=1 should not be enabled to ensure that the communication of FSDP and the computation on the primary stream can be fully parallelized.