2024-05-21 18:02:28,789 INFO StreamThr :375 [internal.py:wandb_internal():85] W&B internal server running at pid: 375, started at: 2024-05-21 18:02:28.782926 2024-05-21 18:02:28,790 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status 2024-05-21 18:02:28,792 INFO WriterThread:375 [datastore.py:open_for_write():87] open: /home/RepViT/wandb/offline-run-20240521_180228-j521cilf/run-j521cilf.wandb 2024-05-21 18:02:28,904 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: run_start 2024-05-21 18:02:32,301 DEBUG HandlerThread:375 [system_info.py:__init__():26] System info init 2024-05-21 18:02:32,301 DEBUG HandlerThread:375 [system_info.py:__init__():41] System info init done 2024-05-21 18:02:32,301 INFO HandlerThread:375 [system_monitor.py:start():194] Starting system monitor 2024-05-21 18:02:32,301 INFO SystemMonitor:375 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-05-21 18:02:32,302 INFO HandlerThread:375 [system_monitor.py:probe():214] Collecting system info 2024-05-21 18:02:32,303 INFO SystemMonitor:375 [interfaces.py:start():188] Started cpu monitoring 2024-05-21 18:02:32,304 INFO SystemMonitor:375 [interfaces.py:start():188] Started disk monitoring 2024-05-21 18:02:32,305 INFO SystemMonitor:375 [interfaces.py:start():188] Started memory monitoring 2024-05-21 18:02:32,306 INFO SystemMonitor:375 [interfaces.py:start():188] Started network monitoring 2024-05-21 18:02:32,345 DEBUG HandlerThread:375 [system_info.py:probe():150] Probing system 2024-05-21 18:02:32,351 DEBUG HandlerThread:375 [system_info.py:_probe_git():135] Probing git 2024-05-21 18:02:32,370 DEBUG HandlerThread:375 [system_info.py:_probe_git():143] Probing git done 2024-05-21 18:02:32,370 DEBUG HandlerThread:375 [system_info.py:probe():198] Probing system done 2024-05-21 18:02:32,370 DEBUG HandlerThread:375 [system_monitor.py:probe():223] {'os': 'Linux-3.10.0-957.el7.x86_64-x86_64-with-glibc2.17', 'python': '3.10.12', 'heartbeatAt': '2024-05-21T10:02:32.345751', 'startedAt': '2024-05-21T10:02:28.676023', 'docker': None, 'cuda': None, 'args': ('--model', 'repvit_m0_9', '--data-path', './imagenet', '--dist-eval'), 'state': 'running', 'program': '/home/RepViT/main.py', 'codePathLocal': 'main.py', 'codePath': 'main.py', 'git': {'remote': 'https://github.com/THU-MIG/RepViT.git', 'commit': '338506bc71d7b4e008cb6f1d94a559e8d8f969c6'}, 'email': None, 'root': '/home/RepViT', 'host': 'dfd3245b928e', 'username': 'root', 'executable': '/usr/local/bin/python', 'cpu_count': 64, 'cpu_count_logical': 128, 'cpu_freq': {'current': 1999.9429999999995, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}, {'current': 1999.943, 'min': 0.0, 'max': 0.0}], 'disk': {'/': {'total': 12935.566589355469, 'used': 11000.176765441895}}, 'memory': {'total': 503.6071319580078}} 2024-05-21 18:02:32,370 INFO HandlerThread:375 [system_monitor.py:probe():224] Finished collecting system info 2024-05-21 18:02:32,370 INFO HandlerThread:375 [system_monitor.py:probe():227] Publishing system info 2024-05-21 18:02:32,382 INFO HandlerThread:375 [system_monitor.py:probe():229] Finished publishing system info 2024-05-21 18:02:32,602 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: python_packages 2024-05-21 18:02:33,818 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:02:33,818 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:02:38,819 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:02:38,820 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:02:43,821 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:02:43,821 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:02:48,822 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:02:48,823 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:02:53,824 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:02:53,824 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:02:58,825 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:02:58,826 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:03,827 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:03,827 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:08,828 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:08,828 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:13,829 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:13,830 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:18,831 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:18,831 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:23,832 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:23,833 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:28,834 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:28,835 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:32,307 DEBUG SystemMonitor:375 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-05-21 18:03:33,836 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:33,836 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:38,837 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:38,838 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:43,839 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:43,839 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:48,840 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:48,841 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:53,841 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:53,842 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:03:58,843 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:03:58,844 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:03,844 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:03,845 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:08,846 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:08,846 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:13,847 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:13,848 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:18,849 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:18,850 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:23,851 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:23,852 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:28,853 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:28,855 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:33,856 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:33,862 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:38,863 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:38,864 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:43,865 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:43,866 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:48,867 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:48,867 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:53,868 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:53,869 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:04:58,869 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:04:58,870 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:03,871 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:03,871 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:08,872 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:08,873 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:13,874 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:13,896 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:18,896 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:18,897 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:23,898 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:23,899 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:28,900 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:28,901 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:33,901 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:33,920 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:38,921 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:38,922 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:43,923 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:43,924 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:48,925 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:48,926 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:53,927 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:53,927 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:05:58,928 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:05:58,929 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:03,930 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:03,931 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:08,932 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:08,933 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:13,933 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:13,934 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:18,935 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:18,935 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:23,936 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:23,937 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:28,938 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:28,938 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:33,939 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:33,940 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:38,941 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:38,941 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:43,942 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:43,943 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:48,943 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:48,944 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:53,945 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:53,945 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:06:58,946 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:06:58,947 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:07:03,832 DEBUG SenderThread:375 [sender.py:send():378] send: exit 2024-05-21 18:07:03,833 INFO SenderThread:375 [sender.py:send_exit():585] handling exit code: 255 2024-05-21 18:07:03,833 INFO SenderThread:375 [sender.py:send_exit():587] handling runtime: 274 2024-05-21 18:07:03,851 INFO SenderThread:375 [sender.py:_save_file():1389] saving file wandb-summary.json with policy end 2024-05-21 18:07:03,851 INFO SenderThread:375 [sender.py:send_exit():593] send defer 2024-05-21 18:07:03,851 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,851 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 0 2024-05-21 18:07:03,852 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,852 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 0 2024-05-21 18:07:03,852 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 1 2024-05-21 18:07:03,852 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,852 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 1 2024-05-21 18:07:03,852 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,852 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 1 2024-05-21 18:07:03,852 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 2 2024-05-21 18:07:03,853 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,853 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 2 2024-05-21 18:07:03,853 INFO HandlerThread:375 [system_monitor.py:finish():203] Stopping system monitor 2024-05-21 18:07:03,853 DEBUG SystemMonitor:375 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-05-21 18:07:03,854 INFO HandlerThread:375 [interfaces.py:finish():200] Joined cpu monitor 2024-05-21 18:07:03,854 DEBUG SystemMonitor:375 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-05-21 18:07:03,855 INFO HandlerThread:375 [interfaces.py:finish():200] Joined disk monitor 2024-05-21 18:07:03,858 INFO HandlerThread:375 [interfaces.py:finish():200] Joined memory monitor 2024-05-21 18:07:03,858 INFO HandlerThread:375 [interfaces.py:finish():200] Joined network monitor 2024-05-21 18:07:03,859 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,859 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 2 2024-05-21 18:07:03,859 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 3 2024-05-21 18:07:03,859 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,859 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 3 2024-05-21 18:07:03,859 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,859 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 3 2024-05-21 18:07:03,859 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 4 2024-05-21 18:07:03,860 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,860 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 4 2024-05-21 18:07:03,860 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,860 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 4 2024-05-21 18:07:03,860 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 5 2024-05-21 18:07:03,860 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,860 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 5 2024-05-21 18:07:03,861 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,861 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 5 2024-05-21 18:07:03,861 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 6 2024-05-21 18:07:03,861 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,861 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 6 2024-05-21 18:07:03,861 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,861 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 6 2024-05-21 18:07:03,861 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 7 2024-05-21 18:07:03,862 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: status_report 2024-05-21 18:07:03,862 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,862 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 7 2024-05-21 18:07:03,862 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: status_report 2024-05-21 18:07:03,862 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,862 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 7 2024-05-21 18:07:03,862 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 8 2024-05-21 18:07:03,862 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,862 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 8 2024-05-21 18:07:03,863 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,863 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 8 2024-05-21 18:07:03,863 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 9 2024-05-21 18:07:03,863 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,863 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 9 2024-05-21 18:07:03,863 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,863 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 9 2024-05-21 18:07:03,863 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 10 2024-05-21 18:07:03,864 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,864 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 10 2024-05-21 18:07:03,864 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,864 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 10 2024-05-21 18:07:03,864 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 11 2024-05-21 18:07:03,864 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,864 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 11 2024-05-21 18:07:03,864 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,865 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 11 2024-05-21 18:07:03,865 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 12 2024-05-21 18:07:03,865 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,865 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 12 2024-05-21 18:07:03,865 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,865 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 12 2024-05-21 18:07:03,865 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 13 2024-05-21 18:07:03,865 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,866 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 13 2024-05-21 18:07:03,866 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,866 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 13 2024-05-21 18:07:03,866 INFO SenderThread:375 [sender.py:transition_state():613] send defer: 14 2024-05-21 18:07:03,866 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: defer 2024-05-21 18:07:03,866 INFO HandlerThread:375 [handler.py:handle_request_defer():184] handle defer: 14 2024-05-21 18:07:03,867 DEBUG SenderThread:375 [sender.py:send():378] send: final 2024-05-21 18:07:03,867 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: defer 2024-05-21 18:07:03,867 INFO SenderThread:375 [sender.py:send_request_defer():609] handle sender defer: 14 2024-05-21 18:07:03,867 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: poll_exit 2024-05-21 18:07:03,868 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: poll_exit 2024-05-21 18:07:03,868 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: poll_exit 2024-05-21 18:07:03,869 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: poll_exit 2024-05-21 18:07:03,869 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: server_info 2024-05-21 18:07:03,869 DEBUG SenderThread:375 [sender.py:send_request():405] send_request: server_info 2024-05-21 18:07:03,870 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: get_summary 2024-05-21 18:07:03,870 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: sampled_history 2024-05-21 18:07:03,870 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: internal_messages 2024-05-21 18:07:03,871 INFO MainThread:375 [wandb_run.py:_footer_history_summary_info():3994] rendering history 2024-05-21 18:07:03,871 INFO MainThread:375 [wandb_run.py:_footer_history_summary_info():4026] rendering summary 2024-05-21 18:07:03,871 DEBUG HandlerThread:375 [handler.py:handle_request():158] handle_request: shutdown 2024-05-21 18:07:03,872 INFO HandlerThread:375 [handler.py:finish():882] shutting down handler 2024-05-21 18:07:04,869 INFO WriterThread:375 [datastore.py:close():296] close: /home/RepViT/wandb/offline-run-20240521_180228-j521cilf/run-j521cilf.wandb 2024-05-21 18:07:04,870 INFO SenderThread:375 [sender.py:finish():1545] shutting down sender