resnet50-onnxruntime-fp16.log

INFO:LANUCH:******************* Pip Package Installing *******************
INFO:PerfEngine:******************* Backend Env Initization *******************
INFO:BackendStore:Loading Compile Backend: DCU
INFO:BackendStore:Loading Runtime Backend: DCU
/usr/local/lib/python3.10/site-packages/tensorflow/python/keras/engine/training_arrays_v1.py:37: UserWarning: A NumPy version >=1.23.5 and <2.3.0 is required for this version of SciPy (detected version 1.23.0)
  from scipy.sparse import issparse  # pylint: disable=g-import-not-at-top
INFO:PerfEngine:******************************************* Start to test model: resnet50-onnxruntime-fp16. *******************************************
INFO:PerfEngine:******************************************* Running Backend Compilation... *******************************************
INFO:PerfEngine:Running Backend Preoptimization...
INFO:DatasetStore:Loading Dataset: open_imagenet
INFO:Imagenet:Initial...

0it [00:00, ?it/s]
9297it [00:00, 92966.99it/s]
31650it [00:00, 169762.59it/s]
50000it [00:00, 177855.12it/s]
INFO:Imagenet:reduced image list, 45000 images not found
INFO:Imagenet:loaded 5000 images, cache=0, took=0.3sec
INFO:Imagenet:Preprocessing...
INFO:Imagenet:Rebatching batch size to: 1 ...

  0%|          | 0/5000 [00:00<?, ?it/s]
  4%|▍         | 193/5000 [00:00<00:02, 1924.98it/s]
  8%|▊         | 388/5000 [00:00<00:02, 1938.29it/s]
 12%|█▏        | 582/5000 [00:00<00:02, 1919.19it/s]
 16%|█▌        | 775/5000 [00:00<00:02, 1922.31it/s]
 19%|█▉        | 968/5000 [00:00<00:02, 1921.08it/s]
 23%|██▎       | 1161/5000 [00:00<00:02, 1916.49it/s]
 27%|██▋       | 1357/5000 [00:00<00:01, 1929.47it/s]
 31%|███       | 1555/5000 [00:00<00:01, 1942.96it/s]
 35%|███▌      | 1753/5000 [00:00<00:01, 1953.38it/s]
 39%|███▉      | 1951/5000 [00:01<00:01, 1960.21it/s]
 43%|████▎     | 2148/5000 [00:01<00:01, 1937.94it/s]
 47%|████▋     | 2346/5000 [00:01<00:01, 1948.89it/s]
 51%|█████     | 2541/5000 [00:01<00:01, 1945.09it/s]
 55%|█████▍    | 2736/5000 [00:01<00:01, 1934.50it/s]
 59%|█████▊    | 2933/5000 [00:01<00:01, 1943.05it/s]
 63%|██████▎   | 3129/5000 [00:01<00:00, 1945.57it/s]
 67%|██████▋   | 3326/5000 [00:01<00:00, 1950.60it/s]
 70%|███████   | 3523/5000 [00:01<00:00, 1953.93it/s]
 74%|███████▍  | 3719/5000 [00:01<00:00, 1955.51it/s]
 78%|███████▊  | 3915/5000 [00:02<00:00, 1944.63it/s]
 82%|████████▏ | 4112/5000 [00:02<00:00, 1949.95it/s]
 86%|████████▌ | 4308/5000 [00:02<00:00, 1947.67it/s]
 90%|█████████ | 4506/5000 [00:02<00:00, 1954.55it/s]
 94%|█████████▍| 4702/5000 [00:02<00:00, 1950.45it/s]
 98%|█████████▊| 4899/5000 [00:02<00:00, 1955.93it/s]
100%|██████████| 5000/5000 [00:02<00:00, 1944.71it/s]
INFO:PerfEngine:Start to compile the model...
INFO:PerfEngine:******************************************* Running Accuracy Checker... *******************************************
INFO:Imagenet:Rebatching batch size to: 1 ...
INFO:TestAccuracy:Start to calculate accuracy...

  0%|          | 0/5000 [00:00<?, ?it/s]
  0%|          | 1/5000 [00:03<4:20:34,  3.13s/it]
  1%|          | 32/5000 [00:03<05:59, 13.84it/s] 
  1%|▏         | 63/5000 [00:03<02:38, 31.22it/s]
  2%|▏         | 94/5000 [00:03<01:32, 52.77it/s]
  2%|▎         | 125/5000 [00:03<01:02, 78.35it/s]
  3%|▎         | 156/5000 [00:03<00:45, 107.15it/s]
  4%|▎         | 187/5000 [00:03<00:35, 137.34it/s]
  4%|▍         | 218/5000 [00:03<00:28, 167.80it/s]
  5%|▍         | 249/5000 [00:03<00:24, 196.33it/s]
  6%|▌         | 280/5000 [00:04<00:21, 220.36it/s]
  6%|▌         | 311/5000 [00:04<00:19, 241.76it/s]
  7%|▋         | 342/5000 [00:04<00:17, 259.01it/s]
  7%|▋         | 373/5000 [00:04<00:16, 272.36it/s]
  8%|▊         | 404/5000 [00:04<00:16, 282.58it/s]
  9%|▊         | 435/5000 [00:04<00:15, 289.95it/s]
  9%|▉         | 466/5000 [00:04<00:15, 295.51it/s]
 10%|▉         | 497/5000 [00:04<00:15, 299.45it/s]
 11%|█         | 528/5000 [00:04<00:14, 302.40it/s]
 11%|█         | 559/5000 [00:04<00:14, 304.45it/s]
 12%|█▏        | 590/5000 [00:05<00:14, 306.08it/s]
 12%|█▏        | 621/5000 [00:05<00:14, 306.97it/s]
 13%|█▎        | 652/5000 [00:05<00:14, 307.69it/s]
 14%|█▎        | 683/5000 [00:05<00:14, 306.11it/s]
 14%|█▍        | 714/5000 [00:05<00:13, 306.89it/s]
 15%|█▍        | 745/5000 [00:05<00:13, 307.44it/s]
 16%|█▌        | 776/5000 [00:05<00:13, 307.91it/s]
 16%|█▌        | 807/5000 [00:05<00:13, 308.34it/s]
 17%|█▋        | 838/5000 [00:05<00:13, 308.44it/s]
 17%|█▋        | 869/5000 [00:05<00:13, 308.67it/s]
 18%|█▊        | 900/5000 [00:06<00:13, 308.70it/s]
 19%|█▊        | 931/5000 [00:06<00:13, 308.85it/s]
 19%|█▉        | 963/5000 [00:06<00:13, 309.19it/s]
 20%|█▉        | 994/5000 [00:06<00:12, 309.23it/s]
 20%|██        | 1025/5000 [00:06<00:12, 309.36it/s]
 21%|██        | 1056/5000 [00:06<00:12, 309.54it/s]
 22%|██▏       | 1087/5000 [00:06<00:12, 307.45it/s]
 22%|██▏       | 1118/5000 [00:06<00:12, 308.13it/s]
 23%|██▎       | 1149/5000 [00:06<00:12, 308.50it/s]
 24%|██▎       | 1181/5000 [00:06<00:12, 309.04it/s]
 24%|██▍       | 1213/5000 [00:07<00:12, 309.39it/s]
 25%|██▍       | 1244/5000 [00:07<00:12, 309.53it/s]
 26%|██▌       | 1275/5000 [00:07<00:12, 309.62it/s]
 26%|██▌       | 1307/5000 [00:07<00:11, 309.74it/s]
 27%|██▋       | 1338/5000 [00:07<00:11, 309.61it/s]
 27%|██▋       | 1369/5000 [00:07<00:11, 309.69it/s]
 28%|██▊       | 1400/5000 [00:07<00:11, 309.73it/s]
 29%|██▊       | 1432/5000 [00:07<00:11, 309.87it/s]
 29%|██▉       | 1463/5000 [00:07<00:11, 309.86it/s]
 30%|██▉       | 1494/5000 [00:07<00:11, 309.70it/s]
 30%|███       | 1525/5000 [00:08<00:11, 307.48it/s]
 31%|███       | 1556/5000 [00:08<00:11, 308.10it/s]
 32%|███▏      | 1587/5000 [00:08<00:11, 308.55it/s]
 32%|███▏      | 1618/5000 [00:08<00:10, 308.95it/s]
 33%|███▎      | 1649/5000 [00:08<00:10, 309.12it/s]
 34%|███▎      | 1680/5000 [00:08<00:10, 309.13it/s]
 34%|███▍      | 1711/5000 [00:08<00:10, 309.26it/s]
 35%|███▍      | 1742/5000 [00:08<00:10, 308.93it/s]
 35%|███▌      | 1773/5000 [00:08<00:10, 309.02it/s]
 36%|███▌      | 1804/5000 [00:08<00:10, 309.23it/s]
 37%|███▋      | 1835/5000 [00:09<00:10, 309.22it/s]
 37%|███▋      | 1866/5000 [00:09<00:10, 309.41it/s]
 38%|███▊      | 1897/5000 [00:09<00:10, 309.41it/s]
 39%|███▊      | 1928/5000 [00:09<00:10, 307.00it/s]
 39%|███▉      | 1959/5000 [00:09<00:09, 307.79it/s]
 40%|███▉      | 1990/5000 [00:09<00:09, 308.18it/s]
 40%|████      | 2021/5000 [00:09<00:09, 308.48it/s]
 41%|████      | 2052/5000 [00:09<00:09, 308.74it/s]
 42%|████▏     | 2083/5000 [00:09<00:09, 308.75it/s]
 42%|████▏     | 2114/5000 [00:09<00:09, 308.91it/s]
 43%|████▎     | 2146/5000 [00:10<00:09, 309.24it/s]
 44%|████▎     | 2177/5000 [00:10<00:09, 308.92it/s]
 44%|████▍     | 2208/5000 [00:10<00:09, 307.90it/s]
 45%|████▍     | 2239/5000 [00:10<00:08, 308.33it/s]
 45%|████▌     | 2270/5000 [00:10<00:08, 308.52it/s]
 46%|████▌     | 2301/5000 [00:10<00:08, 308.84it/s]
 47%|████▋     | 2332/5000 [00:10<00:08, 306.61it/s]
 47%|████▋     | 2363/5000 [00:10<00:08, 307.22it/s]
 48%|████▊     | 2394/5000 [00:10<00:08, 307.90it/s]
 48%|████▊     | 2425/5000 [00:10<00:08, 308.31it/s]
 49%|████▉     | 2456/5000 [00:11<00:08, 308.52it/s]
 50%|████▉     | 2487/5000 [00:11<00:08, 308.53it/s]
 50%|█████     | 2518/5000 [00:11<00:08, 308.52it/s]
 51%|█████     | 2549/5000 [00:11<00:07, 308.82it/s]
 52%|█████▏    | 2580/5000 [00:11<00:07, 309.11it/s]
 52%|█████▏    | 2611/5000 [00:11<00:07, 309.17it/s]
 53%|█████▎    | 2642/5000 [00:11<00:07, 309.28it/s]
 53%|█████▎    | 2673/5000 [00:11<00:07, 309.05it/s]
 54%|█████▍    | 2704/5000 [00:11<00:07, 308.93it/s]
 55%|█████▍    | 2735/5000 [00:11<00:07, 309.02it/s]
 55%|█████▌    | 2766/5000 [00:12<00:07, 307.03it/s]
 56%|█████▌    | 2797/5000 [00:12<00:07, 307.68it/s]
 57%|█████▋    | 2828/5000 [00:12<00:07, 308.27it/s]
 57%|█████▋    | 2859/5000 [00:12<00:06, 308.59it/s]
 58%|█████▊    | 2890/5000 [00:12<00:06, 308.66it/s]
 58%|█████▊    | 2921/5000 [00:12<00:06, 308.82it/s]
 59%|█████▉    | 2952/5000 [00:12<00:06, 309.11it/s]
 60%|█████▉    | 2983/5000 [00:12<00:06, 309.19it/s]
 60%|██████    | 3014/5000 [00:12<00:06, 309.21it/s]
 61%|██████    | 3045/5000 [00:12<00:06, 309.12it/s]
 62%|██████▏   | 3076/5000 [00:13<00:06, 309.25it/s]
 62%|██████▏   | 3107/5000 [00:13<00:06, 309.13it/s]
 63%|██████▎   | 3138/5000 [00:13<00:06, 309.08it/s]
 63%|██████▎   | 3169/5000 [00:13<00:05, 307.63it/s]
 64%|██████▍   | 3200/5000 [00:13<00:05, 307.91it/s]
 65%|██████▍   | 3231/5000 [00:13<00:05, 308.30it/s]
 65%|██████▌   | 3262/5000 [00:13<00:05, 308.55it/s]
 66%|██████▌   | 3293/5000 [00:13<00:05, 308.59it/s]
 66%|██████▋   | 3324/5000 [00:13<00:05, 308.48it/s]
 67%|██████▋   | 3355/5000 [00:13<00:05, 308.82it/s]
 68%|██████▊   | 3386/5000 [00:14<00:05, 308.85it/s]
 68%|██████▊   | 3417/5000 [00:14<00:05, 308.55it/s]
 69%|██████▉   | 3448/5000 [00:14<00:05, 308.76it/s]
 70%|██████▉   | 3479/5000 [00:14<00:04, 308.75it/s]
 70%|███████   | 3510/5000 [00:14<00:04, 308.82it/s]
 71%|███████   | 3541/5000 [00:14<00:04, 308.87it/s]
 71%|███████▏  | 3572/5000 [00:14<00:04, 306.68it/s]
 72%|███████▏  | 3603/5000 [00:14<00:04, 307.56it/s]
 73%|███████▎  | 3634/5000 [00:14<00:04, 308.08it/s]
 73%|███████▎  | 3665/5000 [00:15<00:04, 308.37it/s]
 74%|███████▍  | 3696/5000 [00:15<00:04, 308.71it/s]
 75%|███████▍  | 3727/5000 [00:15<00:04, 308.81it/s]
 75%|███████▌  | 3758/5000 [00:15<00:04, 308.83it/s]
 76%|███████▌  | 3789/5000 [00:15<00:03, 308.78it/s]
 76%|███████▋  | 3820/5000 [00:15<00:03, 308.94it/s]
 77%|███████▋  | 3851/5000 [00:15<00:03, 309.03it/s]
 78%|███████▊  | 3882/5000 [00:15<00:03, 308.38it/s]
 78%|███████▊  | 3913/5000 [00:15<00:03, 308.53it/s]
 79%|███████▉  | 3944/5000 [00:15<00:03, 308.86it/s]
 80%|███████▉  | 3975/5000 [00:16<00:03, 308.78it/s]
 80%|████████  | 4006/5000 [00:16<00:03, 306.48it/s]
 81%|████████  | 4037/5000 [00:16<00:03, 307.35it/s]
 81%|████████▏ | 4068/5000 [00:16<00:03, 307.86it/s]
 82%|████████▏ | 4099/5000 [00:16<00:02, 308.16it/s]
 83%|████████▎ | 4130/5000 [00:16<00:02, 308.57it/s]
 83%|████████▎ | 4161/5000 [00:16<00:02, 308.62it/s]
 84%|████████▍ | 4192/5000 [00:16<00:02, 308.83it/s]
 84%|████████▍ | 4223/5000 [00:16<00:02, 309.13it/s]
 85%|████████▌ | 4254/5000 [00:16<00:02, 309.04it/s]
 86%|████████▌ | 4285/5000 [00:17<00:02, 309.01it/s]
 86%|████████▋ | 4316/5000 [00:17<00:02, 309.17it/s]
 87%|████████▋ | 4347/5000 [00:17<00:02, 309.18it/s]
 88%|████████▊ | 4378/5000 [00:17<00:02, 309.38it/s]
 88%|████████▊ | 4409/5000 [00:17<00:01, 306.77it/s]
 89%|████████▉ | 4440/5000 [00:17<00:01, 307.39it/s]
 89%|████████▉ | 4471/5000 [00:17<00:01, 307.95it/s]
 90%|█████████ | 4502/5000 [00:17<00:01, 308.31it/s]
 91%|█████████ | 4533/5000 [00:17<00:01, 308.65it/s]
 91%|█████████▏| 4564/5000 [00:17<00:01, 308.70it/s]
 92%|█████████▏| 4595/5000 [00:18<00:01, 308.77it/s]
 93%|█████████▎| 4627/5000 [00:18<00:01, 309.21it/s]
 93%|█████████▎| 4658/5000 [00:18<00:01, 309.16it/s]
 94%|█████████▍| 4689/5000 [00:18<00:01, 309.08it/s]
 94%|█████████▍| 4720/5000 [00:18<00:00, 309.00it/s]
 95%|█████████▌| 4751/5000 [00:18<00:00, 308.87it/s]
 96%|█████████▌| 4782/5000 [00:18<00:00, 308.29it/s]
 96%|█████████▋| 4813/5000 [00:18<00:00, 306.30it/s]
 97%|█████████▋| 4844/5000 [00:18<00:00, 307.38it/s]
 98%|█████████▊| 4875/5000 [00:18<00:00, 307.91it/s]
 98%|█████████▊| 4906/5000 [00:19<00:00, 308.05it/s]
 99%|█████████▊| 4937/5000 [00:19<00:00, 308.37it/s]
 99%|█████████▉| 4968/5000 [00:19<00:00, 308.58it/s]
100%|█████████▉| 4999/5000 [00:19<00:00, 308.92it/s]
100%|██████████| 5000/5000 [00:19<00:00, 258.69it/s]
INFO:TestAccuracy:Batch size is 1, Accuracy: 0.7698
INFO:PerfEngine:******************************************* Runing QPS Checker... *******************************************
INFO:BackendDCU:Batch size is 1, QPS: 373, Avg Latency:2.68, Tail Latency:5.21
INFO:BackendDCU:Batch size is 2, QPS: 586, Avg Latency:3.41, Tail Latency:3.43
INFO:BackendDCU:Batch size is 4, QPS: 867, Avg Latency:4.61, Tail Latency:4.72
INFO:BackendDCU:Batch size is 8, QPS: 1271, Avg Latency:6.29, Tail Latency:6.45
INFO:BackendDCU:Batch size is 16, QPS: 1729, Avg Latency:9.25, Tail Latency:11.55
INFO:BackendDCU:Batch size is 32, QPS: 2047, Avg Latency:15.63, Tail Latency:17.64
INFO:BackendDCU:Batch size is 64, QPS: 2088, Avg Latency:30.65, Tail Latency:33.75
INFO:BackendDCU:Batch size is 128, QPS: 2084, Avg Latency:61.41, Tail Latency:63.78
INFO:BackendDCU:Batch size is 256, QPS: 1996, Avg Latency:128.23, Tail Latency:131.11
INFO:BackendDCU:Batch size is 512, QPS: 1955, Avg Latency:261.82, Tail Latency:264.08
INFO:BackendDCU:Batch size is 1024, QPS: 1883, Avg Latency:543.67, Tail Latency:546.07
INFO:PerfEngine:Testing Finish. Report is saved in path: [ general_perf/reports/DCU/resnet50-onnxruntime-fp16/result-fp16.json ]
INFO:PerfEngine:PDF Version is saved in path: [ general_perf/reports/DCU/resnet50-onnxruntime-fp16/RESNET50-ONNXRUNTIME-FP16-TO-FP16.JSON.pdf ]