======================= Modified FLAGS detected ======================= FLAGS(name='FLAGS_cudnn_batchnorm_spatial_persistent', current_value=True, default_value=False) FLAGS(name='FLAGS_selected_gpus', current_value='7', default_value='') ======================================================================= I0727 11:19:01.149477 2901 tcp_utils.cc:107] Retry to connect to 127.0.0.1:52742 while the server is not yet listening. I0727 11:19:04.149612 2901 tcp_utils.cc:130] Successfully connected to 127.0.0.1:52742 I0727 11:19:04.179198 2901 process_group_nccl.cc:120] ProcessGroupNCCL pg_timeout_ 1800000 W0727 11:19:04.197528 2901 gpu_resources.cc:119] Please NOTE: device: 7, GPU Compute Capability: 90.2, Driver API Version: 50724.2, Runtime API Version: 50724.2 eval model:: 0%| | 0/500 [00:00 >) ---------------------- Error Message Summary: ---------------------- FatalError: `Termination signal` is detected by the operating system. [TimeInfo: *** Aborted at 1722504078 (unix time) try "date -d @1722504078" if you are using GNU date ***] [SignalInfo: *** SIGTERM (@0x148) received by PID 451 (TID 0x7fe5d8c1c640) from PID 328 ***] ======================= Modified FLAGS detected ======================= FLAGS(name='FLAGS_selected_gpus', current_value='7', default_value='') FLAGS(name='FLAGS_cudnn_batchnorm_spatial_persistent', current_value=True, default_value=False) ======================================================================= I0801 17:43:09.999272 774 tcp_utils.cc:130] Successfully connected to 10.8.145.246:62911 I0801 17:43:10.035256 774 process_group_nccl.cc:120] ProcessGroupNCCL pg_timeout_ 1800000 Traceback (most recent call last): File "/workspace/dbnet/PaddleOCR-release-2.5/tools/train.py", line 198, in main(config, device, logger, vdl_writer) File "/workspace/dbnet/PaddleOCR-release-2.5/tools/train.py", line 53, in main train_dataloader = build_dataloader(config, 'Train', device, logger) File "/workspace/dbnet/PaddleOCR-release-2.5/ppocr/data/__init__.py", line 65, in build_dataloader dataset = eval(module_name)(config, mode, logger, seed) File "/workspace/dbnet/PaddleOCR-release-2.5/ppocr/data/simple_dataset.py", line 47, in __init__ self.data_lines = self.get_image_info_list(label_file_list, ratio_list) File "/workspace/dbnet/PaddleOCR-release-2.5/ppocr/data/simple_dataset.py", line 61, in get_image_info_list with open(file, "rb") as f: FileNotFoundError: [Errno 2] No such file or directory: '/datasets/icdar2015/text_localization/train_icdar2015_label.txt' ======================= Modified FLAGS detected ======================= FLAGS(name='FLAGS_selected_gpus', current_value='7', default_value='') FLAGS(name='FLAGS_cudnn_batchnorm_spatial_persistent', current_value=True, default_value=False) ======================================================================= I0801 17:45:05.537649 1093 tcp_utils.cc:130] Successfully connected to 10.8.145.246:49295 I0801 17:45:05.558270 1093 process_group_nccl.cc:120] ProcessGroupNCCL pg_timeout_ 1800000 W0801 17:45:05.576316 1093 gpu_resources.cc:119] Please NOTE: device: 7, GPU Compute Capability: 90.2, Driver API Version: 50724.2, Runtime API Version: 50724.2 eval model:: 0%| | 0/500 [00:00