Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
widedeep-tf-fp16.log 21.7 KB
Newer Older
sunzhq2's avatar
init  
sunzhq2 committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80 
INFO:LANUCH:******************* Pip Package Installing *******************
INFO:PerfEngine:******************* Backend Env Initization *******************
INFO:BackendStore:Loading Compile Backend: DCU
INFO:BackendStore:Loading Runtime Backend: DCU
/usr/local/lib/python3.10/site-packages/tensorflow/python/keras/engine/training_arrays_v1.py:37: UserWarning: A NumPy version >=1.23.5 and <2.3.0 is required for this version of SciPy (detected version 1.23.0)
  from scipy.sparse import issparse  # pylint: disable=g-import-not-at-top
INFO:PerfEngine:******************************************* Start to test model: widedeep-tf-fp16. *******************************************
INFO:PerfEngine:******************************************* Running Backend Compilation... *******************************************
INFO:PerfEngine:Running Backend Preoptimization...
INFO:DatasetStore:Loading Dataset: open_criteo_kaggle
INFO:CriteoKaggle:Initial...

  0%|          | 0/2000000 [00:00<?, ?it/s]
  1%|          | 16213/2000000 [00:00<00:12, 162114.31it/s]
  2%|▏         | 33330/2000000 [00:00<00:11, 167434.26it/s]
  3%|▎         | 50409/2000000 [00:00<00:11, 168963.62it/s]
  3%|▎         | 67553/2000000 [00:00<00:11, 169938.10it/s]
  4%|▍         | 84676/2000000 [00:00<00:11, 170398.44it/s]
  5%|▌         | 101777/2000000 [00:00<00:11, 170603.65it/s]
  6%|▌         | 118939/2000000 [00:00<00:11, 170933.96it/s]
  7%|▋         | 136138/2000000 [00:00<00:10, 171268.24it/s]
  8%|▊         | 153288/2000000 [00:00<00:10, 171338.19it/s]
  9%|▊         | 170470/2000000 [00:01<00:10, 171483.70it/s]
  9%|▉         | 187642/2000000 [00:01<00:10, 171550.18it/s]
 10%|█         | 204798/2000000 [00:01<00:10, 171328.31it/s]
 11%|█         | 222094/2000000 [00:01<00:10, 171818.22it/s]
 12%|█▏        | 239276/2000000 [00:01<00:10, 171545.73it/s]
 13%|█▎        | 256521/2000000 [00:01<00:10, 171813.98it/s]
 14%|█▎        | 273703/2000000 [00:01<00:10, 171452.90it/s]
 15%|█▍        | 290849/2000000 [00:01<00:10, 170739.40it/s]
 15%|█▌        | 308236/2000000 [00:01<00:09, 171673.35it/s]
 16%|█▋        | 325405/2000000 [00:01<00:09, 170573.33it/s]
 17%|█▋        | 342914/2000000 [00:02<00:09, 171917.53it/s]
 18%|█▊        | 360175/2000000 [00:02<00:09, 172121.88it/s]
 19%|█▉        | 377389/2000000 [00:02<00:09, 171574.40it/s]
 20%|█▉        | 394666/2000000 [00:02<00:09, 171928.16it/s]
 21%|██        | 411860/2000000 [00:02<00:09, 170518.94it/s]
 21%|██▏       | 429424/2000000 [00:02<00:09, 172039.90it/s]
 22%|██▏       | 446700/2000000 [00:02<00:09, 172251.96it/s]
 23%|██▎       | 463968/2000000 [00:02<00:08, 172375.99it/s]
 24%|██▍       | 481209/2000000 [00:02<00:08, 172381.17it/s]
 25%|██▍       | 498476/2000000 [00:02<00:08, 172464.50it/s]
 26%|██▌       | 515756/2000000 [00:03<00:08, 172561.86it/s]
 27%|██▋       | 533025/2000000 [00:03<00:08, 172596.00it/s]
 28%|██▊       | 550286/2000000 [00:03<00:08, 172525.15it/s]
 28%|██▊       | 567572/2000000 [00:03<00:08, 172620.76it/s]
 29%|██▉       | 584864/2000000 [00:03<00:08, 172708.92it/s]
 30%|███       | 602136/2000000 [00:03<00:08, 172683.41it/s]
 31%|███       | 619405/2000000 [00:03<00:07, 172632.11it/s]
 32%|███▏      | 636711/2000000 [00:03<00:07, 172756.26it/s]
 33%|███▎      | 654010/2000000 [00:03<00:07, 172824.18it/s]
 34%|███▎      | 671293/2000000 [00:03<00:07, 172824.25it/s]
 34%|███▍      | 688576/2000000 [00:04<00:07, 172644.16it/s]
 35%|███▌      | 705841/2000000 [00:04<00:07, 172612.66it/s]
 36%|███▌      | 723103/2000000 [00:04<00:07, 172555.34it/s]
 37%|███▋      | 740359/2000000 [00:04<00:07, 172545.93it/s]
 38%|███▊      | 757614/2000000 [00:04<00:07, 172537.58it/s]
 39%|███▊      | 774868/2000000 [00:04<00:07, 172374.94it/s]
 40%|███▉      | 792147/2000000 [00:04<00:07, 172496.55it/s]
 40%|████      | 809419/2000000 [00:04<00:06, 172561.12it/s]
 41%|████▏     | 826689/2000000 [00:04<00:06, 172600.22it/s]
 42%|████▏     | 843953/2000000 [00:04<00:06, 172610.10it/s]
 43%|████▎     | 861215/2000000 [00:05<00:06, 172376.83it/s]
 44%|████▍     | 878484/2000000 [00:05<00:06, 172467.52it/s]
 45%|████▍     | 895767/2000000 [00:05<00:06, 172573.63it/s]
 46%|████▌     | 913037/2000000 [00:05<00:06, 172609.97it/s]
 47%|████▋     | 930311/2000000 [00:05<00:06, 172647.41it/s]
 47%|████▋     | 947578/2000000 [00:05<00:06, 172652.13it/s]
 48%|████▊     | 964844/2000000 [00:05<00:06, 172482.85it/s]
 49%|████▉     | 982118/2000000 [00:05<00:05, 172556.03it/s]
 50%|████▉     | 999382/2000000 [00:05<00:05, 172579.28it/s]
 51%|█████     | 1016647/2000000 [00:05<00:05, 172596.10it/s]
 52%|█████▏    | 1033940/2000000 [00:06<00:05, 172693.69it/s]
 53%|█████▎    | 1051224/2000000 [00:06<00:05, 172735.77it/s]
 53%|█████▎    | 1068517/2000000 [00:06<00:05, 172791.36it/s]
 54%|█████▍    | 1085797/2000000 [00:06<00:05, 172475.39it/s]
 55%|█████▌    | 1103077/2000000 [00:06<00:05, 172570.96it/s]
 56%|█████▌    | 1120362/2000000 [00:06<00:05, 172653.29it/s]
 57%|█████▋    | 1137676/2000000 [00:06<00:04, 172795.05it/s]
 58%|█████▊    | 1154981/2000000 [00:06<00:04, 172870.19it/s]
 59%|█████▊    | 1172283/2000000 [00:06<00:04, 172911.08it/s]
 59%|█████▉    | 1189575/2000000 [00:06<00:04, 172800.99it/s]
 60%|██████    | 1206876/2000000 [00:07<00:04, 172861.11it/s]
 61%|██████    | 1224163/2000000 [00:07<00:04, 172574.26it/s]
 62%|██████▏   | 1241454/2000000 [00:07<00:04, 172672.04it/s]
 63%|██████▎   | 1258722/2000000 [00:07<00:04, 172640.80it/s]
 64%|██████▍   | 1276001/2000000 [00:07<00:04, 172683.23it/s]
 65%|██████▍   | 1293287/2000000 [00:07<00:04, 172732.57it/s]
 66%|██████▌   | 1310573/2000000 [00:07<00:03, 172766.63it/s]
 66%|██████▋   | 1327850/2000000 [00:07<00:03, 172747.43it/s]
 67%|██████▋   | 1345125/2000000 [00:07<00:03, 172730.59it/s]
 68%|██████▊   | 1362427/2000000 [00:07<00:03, 172813.36it/s]
 69%|██████▉   | 1379709/2000000 [00:08<00:03, 172464.16it/s]
 70%|██████▉   | 1397023/2000000 [00:08<00:03, 172664.42it/s]
 71%|███████   | 1414321/2000000 [00:08<00:03, 172756.69it/s]
 72%|███████▏  | 1431624/2000000 [00:08<00:03, 172836.82it/s]
 72%|███████▏  | 1448932/2000000 [00:08<00:03, 172906.81it/s]
 73%|███████▎  | 1466239/2000000 [00:08<00:03, 172952.09it/s]
 74%|███████▍  | 1483536/2000000 [00:08<00:02, 172953.06it/s]
 75%|███████▌  | 1500832/2000000 [00:08<00:02, 172914.26it/s]
 76%|███████▌  | 1518124/2000000 [00:08<00:02, 172844.72it/s]
 77%|███████▋  | 1535422/2000000 [00:08<00:02, 172883.48it/s]
 78%|███████▊  | 1552711/2000000 [00:09<00:02, 172478.28it/s]
 79%|███████▊  | 1570029/2000000 [00:09<00:02, 172684.38it/s]
 79%|███████▉  | 1587349/2000000 [00:09<00:02, 172836.31it/s]
 80%|████████  | 1604657/2000000 [00:09<00:02, 172905.07it/s]
 81%|████████  | 1621958/2000000 [00:09<00:02, 172933.49it/s]
 82%|████████▏ | 1639252/2000000 [00:09<00:02, 172927.33it/s]
 83%|████████▎ | 1656551/2000000 [00:09<00:01, 172943.95it/s]
 84%|████████▎ | 1673848/2000000 [00:09<00:01, 172948.72it/s]
 85%|████████▍ | 1691154/2000000 [00:09<00:01, 172979.30it/s]
 85%|████████▌ | 1708452/2000000 [00:09<00:01, 172889.43it/s]
 86%|████████▋ | 1725741/2000000 [00:10<00:01, 172797.98it/s]
 87%|████████▋ | 1743021/2000000 [00:10<00:01, 172285.26it/s]
 88%|████████▊ | 1760284/2000000 [00:10<00:01, 172383.42it/s]
 89%|████████▉ | 1777537/2000000 [00:10<00:01, 172425.89it/s]
 90%|████████▉ | 1794828/2000000 [00:10<00:01, 172569.63it/s]
 91%|█████████ | 1812115/2000000 [00:10<00:01, 172658.23it/s]
 91%|█████████▏| 1829405/2000000 [00:10<00:00, 172729.40it/s]
 92%|█████████▏| 1846694/2000000 [00:10<00:00, 172776.25it/s]
 93%|█████████▎| 1863992/2000000 [00:10<00:00, 172833.33it/s]
 94%|█████████▍| 1881276/2000000 [00:10<00:00, 172753.23it/s]
 95%|█████████▍| 1898552/2000000 [00:11<00:00, 172753.95it/s]
 96%|█████████▌| 1915834/2000000 [00:11<00:00, 172771.46it/s]
 97%|█████████▋| 1933134/2000000 [00:11<00:00, 172837.21it/s]
 98%|█████████▊| 1950433/2000000 [00:11<00:00, 172878.89it/s]
 98%|█████████▊| 1967721/2000000 [00:11<00:00, 172204.64it/s]
 99%|█████████▉| 1985004/2000000 [00:11<00:00, 172388.94it/s]
100%|██████████| 2000000/2000000 [00:11<00:00, 172347.95it/s]
INFO:CriteoKaggle:Preprocessing...
INFO:CriteoKaggle:Rebatching batch size to: 1 ...

  0%|          | 0/2000000 [00:00<?, ?it/s]
  3%|▎         | 61038/2000000 [00:00<00:03, 610358.75it/s]
  6%|▌         | 124243/2000000 [00:00<00:03, 623101.87it/s]
  9%|▉         | 187585/2000000 [00:00<00:02, 627802.58it/s]
 13%|█▎        | 251426/2000000 [00:00<00:02, 631982.68it/s]
 16%|█▌        | 314985/2000000 [00:00<00:02, 633277.16it/s]
 19%|█▉        | 378404/2000000 [00:00<00:02, 633581.05it/s]
 22%|██▏       | 442269/2000000 [00:00<00:02, 635232.92it/s]
 25%|██▌       | 505997/2000000 [00:00<00:02, 635879.97it/s]
 28%|██▊       | 569759/2000000 [00:00<00:02, 636416.99it/s]
 32%|███▏      | 633401/2000000 [00:01<00:02, 636400.31it/s]
 35%|███▍      | 697042/2000000 [00:01<00:02, 635711.86it/s]
 38%|███▊      | 760614/2000000 [00:01<00:01, 635670.75it/s]
 41%|████      | 824702/2000000 [00:01<00:01, 637243.47it/s]
 44%|████▍     | 888427/2000000 [00:01<00:01, 636551.88it/s]
 48%|████▊     | 952083/2000000 [00:01<00:01, 635434.12it/s]
 51%|█████     | 1016273/2000000 [00:01<00:01, 637372.14it/s]
 54%|█████▍    | 1080011/2000000 [00:01<00:01, 635477.50it/s]
 57%|█████▋    | 1144235/2000000 [00:01<00:01, 637499.45it/s]
 60%|██████    | 1207987/2000000 [00:01<00:01, 635662.17it/s]
 64%|██████▎   | 1272042/2000000 [00:02<00:01, 637121.07it/s]
 67%|██████▋   | 1336156/2000000 [00:02<00:01, 638318.24it/s]
 70%|██████▉   | 1399990/2000000 [00:02<00:00, 635109.90it/s]
 73%|███████▎  | 1464169/2000000 [00:02<00:00, 637099.25it/s]
 76%|███████▋  | 1527883/2000000 [00:02<00:00, 636159.99it/s]
 80%|███████▉  | 1591846/2000000 [00:02<00:00, 637192.80it/s]
 83%|████████▎ | 1655654/2000000 [00:02<00:00, 637452.17it/s]
 86%|████████▌ | 1719401/2000000 [00:02<00:00, 635973.06it/s]
 89%|████████▉ | 1783734/2000000 [00:02<00:00, 638170.15it/s]
 92%|█████████▏| 1847950/2000000 [00:02<00:00, 639360.41it/s]
 96%|█████████▌| 1911888/2000000 [00:03<00:00, 634141.15it/s]
 99%|█████████▉| 1975949/2000000 [00:03<00:00, 636061.79it/s]
100%|██████████| 2000000/2000000 [00:03<00:00, 635577.02it/s]
INFO:PerfEngine:Start to compile the model...
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:PerfEngine:******************************************* Running Accuracy Checker... *******************************************
INFO:CriteoKaggle:Rebatching batch size to: 1024 ...

  0%|          | 0/1953 [00:00<?, ?it/s]
100%|██████████| 1953/1953 [00:00<00:00, 583521.56it/s]
INFO:TestAccuracy:Start to calculate accuracy...

  0%|          | 0/1953 [00:00<?, ?it/s]WARNING:absl:`new_categorical_placeholder:0` is not a valid tf.function parameter name. Sanitizing to `new_categorical_placeholder_0`.
WARNING:absl:`new_numeric_placeholder:0` is not a valid tf.function parameter name. Sanitizing to `new_numeric_placeholder_0`.

  0%|          | 1/1953 [00:04<2:17:51,  4.24s/it]WARNING:absl:`new_categorical_placeholder:0` is not a valid tf.function parameter name. Sanitizing to `new_categorical_placeholder_0`.
WARNING:absl:`new_numeric_placeholder:0` is not a valid tf.function parameter name. Sanitizing to `new_numeric_placeholder_0`.
WARNING:absl:`new_categorical_placeholder:0` is not a valid tf.function parameter name. Sanitizing to `new_categorical_placeholder_0`.

  1%|▏         | 27/1953 [00:04<03:41,  8.70it/s] 
  3%|▎         | 53/1953 [00:04<01:35, 19.86it/s]
  4%|▍         | 79/1953 [00:04<00:54, 34.17it/s]
  5%|▌         | 105/1953 [00:04<00:35, 51.84it/s]
  7%|▋         | 131/1953 [00:04<00:25, 72.70it/s]
  8%|▊         | 157/1953 [00:04<00:18, 96.10it/s]
  9%|▉         | 183/1953 [00:04<00:14, 120.79it/s]
 11%|█         | 209/1953 [00:05<00:12, 145.23it/s]
 12%|█▏        | 235/1953 [00:05<00:10, 168.14it/s]
 13%|█▎        | 261/1953 [00:05<00:08, 188.23it/s]
 15%|█▍        | 287/1953 [00:05<00:08, 205.01it/s]
 16%|█▌        | 313/1953 [00:05<00:07, 218.52it/s]
 17%|█▋        | 339/1953 [00:05<00:07, 228.93it/s]
 19%|█▊        | 365/1953 [00:05<00:06, 236.25it/s]
 20%|██        | 391/1953 [00:05<00:06, 242.25it/s]
 21%|██▏       | 417/1953 [00:05<00:06, 246.40it/s]
 23%|██▎       | 443/1953 [00:05<00:06, 249.67it/s]
 24%|██▍       | 469/1953 [00:06<00:05, 251.58it/s]
 25%|██▌       | 495/1953 [00:06<00:05, 253.10it/s]
 27%|██▋       | 521/1953 [00:06<00:05, 254.12it/s]
 28%|██▊       | 547/1953 [00:06<00:05, 254.28it/s]
 29%|██▉       | 573/1953 [00:06<00:05, 254.97it/s]
 31%|███       | 599/1953 [00:06<00:05, 255.56it/s]
 32%|███▏      | 625/1953 [00:06<00:05, 255.90it/s]
 33%|███▎      | 651/1953 [00:06<00:05, 256.11it/s]
 35%|███▍      | 677/1953 [00:06<00:04, 256.05it/s]
 36%|███▌      | 703/1953 [00:06<00:04, 255.79it/s]
 37%|███▋      | 729/1953 [00:07<00:04, 255.83it/s]
 39%|███▊      | 755/1953 [00:07<00:04, 256.06it/s]
 40%|███▉      | 781/1953 [00:07<00:04, 256.21it/s]
 41%|████▏     | 807/1953 [00:07<00:04, 256.35it/s]
 43%|████▎     | 833/1953 [00:07<00:04, 256.53it/s]
 44%|████▍     | 859/1953 [00:07<00:04, 256.44it/s]
 45%|████▌     | 885/1953 [00:07<00:04, 256.66it/s]
 47%|████▋     | 911/1953 [00:07<00:04, 256.80it/s]
 48%|████▊     | 937/1953 [00:07<00:03, 256.80it/s]
 49%|████▉     | 963/1953 [00:07<00:03, 256.92it/s]
 51%|█████     | 989/1953 [00:08<00:03, 256.90it/s]
 52%|█████▏    | 1015/1953 [00:08<00:03, 256.62it/s]
 53%|█████▎    | 1041/1953 [00:08<00:03, 256.07it/s]
 55%|█████▍    | 1067/1953 [00:08<00:03, 256.42it/s]
 56%|█████▌    | 1093/1953 [00:08<00:03, 256.54it/s]
 57%|█████▋    | 1119/1953 [00:08<00:03, 256.99it/s]
 59%|█████▊    | 1145/1953 [00:08<00:03, 257.16it/s]
 60%|█████▉    | 1171/1953 [00:08<00:03, 257.38it/s]
 61%|██████▏   | 1197/1953 [00:08<00:02, 257.32it/s]
 63%|██████▎   | 1223/1953 [00:09<00:02, 257.24it/s]
 64%|██████▍   | 1249/1953 [00:09<00:02, 257.32it/s]
 65%|██████▌   | 1275/1953 [00:09<00:02, 257.37it/s]
 67%|██████▋   | 1301/1953 [00:09<00:02, 257.24it/s]
 68%|██████▊   | 1327/1953 [00:09<00:02, 257.30it/s]
 69%|██████▉   | 1353/1953 [00:09<00:02, 257.24it/s]
 71%|███████   | 1379/1953 [00:09<00:02, 257.29it/s]
 72%|███████▏  | 1405/1953 [00:09<00:02, 257.07it/s]
 73%|███████▎  | 1431/1953 [00:09<00:02, 257.16it/s]
 75%|███████▍  | 1457/1953 [00:09<00:01, 257.22it/s]
 76%|███████▌  | 1483/1953 [00:10<00:01, 257.08it/s]
 77%|███████▋  | 1509/1953 [00:10<00:01, 257.09it/s]
 79%|███████▊  | 1535/1953 [00:10<00:01, 257.29it/s]
 80%|███████▉  | 1561/1953 [00:10<00:01, 257.32it/s]
 81%|████████▏ | 1587/1953 [00:10<00:01, 257.24it/s]
 83%|████████▎ | 1613/1953 [00:10<00:01, 257.18it/s]
 84%|████████▍ | 1639/1953 [00:10<00:01, 256.96it/s]
 85%|████████▌ | 1665/1953 [00:10<00:01, 257.10it/s]
 87%|████████▋ | 1691/1953 [00:10<00:01, 257.00it/s]
 88%|████████▊ | 1717/1953 [00:10<00:00, 256.90it/s]
 89%|████████▉ | 1743/1953 [00:11<00:00, 257.03it/s]
 91%|█████████ | 1769/1953 [00:11<00:00, 257.01it/s]
 92%|█████████▏| 1795/1953 [00:11<00:00, 256.82it/s]
 93%|█████████▎| 1821/1953 [00:11<00:00, 256.82it/s]
 95%|█████████▍| 1847/1953 [00:11<00:00, 256.85it/s]
 96%|█████████▌| 1873/1953 [00:11<00:00, 256.88it/s]
 97%|█████████▋| 1899/1953 [00:11<00:00, 256.90it/s]
 99%|█████████▊| 1925/1953 [00:11<00:00, 256.80it/s]
100%|█████████▉| 1951/1953 [00:11<00:00, 256.82it/s]
100%|██████████| 1953/1953 [00:11<00:00, 164.94it/s]
INFO:TestAccuracy:Batch size is 1024, Accuracy: 0.77395
INFO:PerfEngine:******************************************* Runing QPS Checker... *******************************************
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 1024, QPS: 1066666, Avg Latency:0.96, Tail Latency:1.04
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 20000, QPS: 3200000, Avg Latency:6.25, Tail Latency:7.07
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 40000, QPS: 3159557, Avg Latency:12.66, Tail Latency:12.8
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 80000, QPS: 3898635, Avg Latency:20.52, Tail Latency:22.77
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 120000, QPS: 3271537, Avg Latency:36.68, Tail Latency:38.92
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 140000, QPS: 3170289, Avg Latency:44.16, Tail Latency:46.42
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 160000, QPS: 3132341, Avg Latency:51.08, Tail Latency:53.33
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 180000, QPS: 3092783, Avg Latency:58.2, Tail Latency:60.19
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 200000, QPS: 3504468, Avg Latency:57.07, Tail Latency:59.95
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 220000, QPS: 3054707, Avg Latency:72.02, Tail Latency:74.48
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 240000, QPS: 3099173, Avg Latency:77.44, Tail Latency:80.01
INFO:tensorflow:Saver not created because there are no variables in the graph to restore
INFO:absl:Fingerprint not found. Saved model loading will continue.
INFO:BackendDCU:Batch size is 260000, QPS: 3584723, Avg Latency:72.53, Tail Latency:76.46
INFO:PerfEngine:Testing Finish. Report is saved in path: [ general_perf/reports/DCU/widedeep-tf-fp16/result-fp16.json ]
INFO:PerfEngine:PDF Version is saved in path: [ general_perf/reports/DCU/widedeep-tf-fp16/WIDEDEEP-TF-FP16-TO-FP16.JSON.pdf ]
Exception ignored in: <function AtomicFunction.__del__ at 0x7f584d658b80>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/tensorflow/python/eager/polymorphic_function/atomic_function.py", line 218, in __del__
TypeError: 'NoneType' object is not subscriptable
Exception ignored in: <function AtomicFunction.__del__ at 0x7f584d658b80>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/tensorflow/python/eager/polymorphic_function/atomic_function.py", line 218, in __del__
TypeError: 'NoneType' object is not subscriptable
Exception ignored in: <function AtomicFunction.__del__ at 0x7f584d658b80>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/tensorflow/python/eager/polymorphic_function/atomic_function.py", line 218, in __del__
TypeError: 'NoneType' object is not subscriptable