Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
bert-onnxruntime-fp16.log 23.9 KB
Newer Older
sunzhq2's avatar
init  
sunzhq2 committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43 
INFO:LANUCH:******************* Pip Package Installing *******************
INFO:PerfEngine:******************* Backend Env Initization *******************
INFO:BackendStore:Loading Compile Backend: DCU
INFO:BackendStore:Loading Runtime Backend: DCU
INFO:PerfEngine:******************************************* Start to test model: bert-onnxruntime-fp16. *******************************************
INFO:PerfEngine:******************************************* Running Backend Compilation... *******************************************
INFO:PerfEngine:Running Backend Preoptimization...
INFO:DatasetStore:Loading Dataset: open_squad
INFO:SQUAD:Initial...
INFO:SQUAD:Preprocessing...
INFO:SQUAD:Rebatching batch size to: 10833 ...

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:00<00:00, 93.58it/s]
INFO:PerfEngine:Start to compile the model...
2024-10-29 10:22:36.670387581 [W:onnxruntime:, session_state.cc:1169 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-10-29 10:22:36.670406950 [W:onnxruntime:, session_state.cc:1171 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
INFO:PerfEngine:******************************************* Running Accuracy Checker... *******************************************
INFO:SQUAD:Rebatching batch size to: 4 ...

  0%|          | 0/2708 [00:00<?, ?it/s]
100%|██████████| 2708/2708 [00:00<00:00, 132671.91it/s]
INFO:TestAccuracy:Start to calculate accuracy...

  0%|          | 0/2708 [00:00<?, ?it/s]
  0%|          | 1/2708 [00:00<21:28,  2.10it/s]
  0%|          | 9/2708 [00:00<02:14, 20.07it/s]
  1%|          | 18/2708 [00:00<01:13, 36.43it/s]
  1%|          | 27/2708 [00:00<00:54, 49.14it/s]
  1%|▏         | 36/2708 [00:00<00:45, 58.36it/s]
  2%|▏         | 45/2708 [00:01<00:40, 65.12it/s]
  2%|▏         | 54/2708 [00:01<00:37, 70.06it/s]
  2%|▏         | 63/2708 [00:01<00:35, 73.67it/s]
  3%|▎         | 72/2708 [00:01<00:34, 75.88it/s]
  3%|▎         | 81/2708 [00:01<00:33, 77.65it/s]
  3%|▎         | 90/2708 [00:01<00:33, 78.96it/s]
  4%|▎         | 99/2708 [00:01<00:32, 79.92it/s]
  4%|▍         | 108/2708 [00:01<00:32, 80.53it/s]
  4%|▍         | 117/2708 [00:01<00:32, 80.26it/s]
  5%|▍         | 126/2708 [00:02<00:32, 80.24it/s]
  5%|▍         | 135/2708 [00:02<00:32, 80.33it/s]
  5%|▌         | 144/2708 [00:02<00:31, 80.77it/s]
  6%|▌         | 153/2708 [00:02<00:31, 80.70it/s]
  6%|▌         | 162/2708 [00:02<00:31, 81.03it/s]
  6%|▋         | 171/2708 [00:02<00:31, 81.22it/s]
  7%|▋         | 180/2708 [00:02<00:31, 80.73it/s]
  7%|▋         | 189/2708 [00:02<00:31, 80.84it/s]
  7%|▋         | 198/2708 [00:02<00:30, 81.13it/s]
  8%|▊         | 207/2708 [00:03<00:30, 81.32it/s]
  8%|▊         | 216/2708 [00:03<00:30, 81.27it/s]
  8%|▊         | 225/2708 [00:03<00:30, 80.80it/s]
  9%|▊         | 234/2708 [00:03<00:30, 80.67it/s]
  9%|▉         | 243/2708 [00:03<00:30, 80.99it/s]
  9%|▉         | 252/2708 [00:03<00:30, 81.18it/s]
 10%|▉         | 261/2708 [00:03<00:30, 81.37it/s]
 10%|▉         | 270/2708 [00:03<00:29, 81.73it/s]
 10%|█         | 279/2708 [00:03<00:29, 82.01it/s]
 11%|█         | 288/2708 [00:04<00:29, 81.50it/s]
 11%|█         | 297/2708 [00:04<00:29, 81.85it/s]
 11%|█▏        | 306/2708 [00:04<00:29, 82.13it/s]
 12%|█▏        | 315/2708 [00:04<00:29, 82.19it/s]
 12%|█▏        | 324/2708 [00:04<00:29, 82.07it/s]
 12%|█▏        | 333/2708 [00:04<00:29, 81.59it/s]
 13%|█▎        | 342/2708 [00:04<00:29, 81.14it/s]
 13%|█▎        | 351/2708 [00:04<00:28, 81.34it/s]
 13%|█▎        | 360/2708 [00:04<00:28, 81.54it/s]
 14%|█▎        | 369/2708 [00:04<00:28, 81.58it/s]
 14%|█▍        | 378/2708 [00:05<00:28, 81.51it/s]
 14%|█▍        | 387/2708 [00:05<00:28, 81.78it/s]
 15%|█▍        | 396/2708 [00:05<00:28, 81.55it/s]
 15%|█▍        | 405/2708 [00:05<00:28, 81.62it/s]
 15%|█▌        | 414/2708 [00:05<00:28, 81.73it/s]
 16%|█▌        | 423/2708 [00:05<00:27, 81.89it/s]
 16%|█▌        | 432/2708 [00:05<00:27, 81.73it/s]
 16%|█▋        | 441/2708 [00:05<00:27, 81.98it/s]
 17%|█▋        | 450/2708 [00:05<00:27, 80.99it/s]
 17%|█▋        | 459/2708 [00:06<00:27, 81.30it/s]
 17%|█▋        | 468/2708 [00:06<00:27, 81.43it/s]
 18%|█▊        | 477/2708 [00:06<00:27, 81.58it/s]
 18%|█▊        | 486/2708 [00:06<00:27, 81.81it/s]
 18%|█▊        | 495/2708 [00:06<00:27, 81.89it/s]
 19%|█▊        | 504/2708 [00:06<00:27, 81.35it/s]
 19%|█▉        | 513/2708 [00:06<00:26, 81.55it/s]
 19%|█▉        | 522/2708 [00:06<00:26, 81.84it/s]
 20%|█▉        | 531/2708 [00:06<00:26, 81.82it/s]
 20%|█▉        | 540/2708 [00:07<00:26, 81.59it/s]
 20%|██        | 549/2708 [00:07<00:26, 81.67it/s]
 21%|██        | 558/2708 [00:07<00:27, 78.50it/s]
 21%|██        | 567/2708 [00:07<00:26, 79.94it/s]
 21%|██▏       | 576/2708 [00:07<00:26, 80.86it/s]
 22%|██▏       | 585/2708 [00:07<00:26, 81.40it/s]
 22%|██▏       | 594/2708 [00:07<00:25, 81.91it/s]
 22%|██▏       | 603/2708 [00:07<00:25, 82.13it/s]
 23%|██▎       | 612/2708 [00:07<00:25, 82.05it/s]
 23%|██▎       | 621/2708 [00:08<00:25, 82.23it/s]
 23%|██▎       | 630/2708 [00:08<00:25, 82.55it/s]
 24%|██▎       | 639/2708 [00:08<00:25, 82.74it/s]
 24%|██▍       | 648/2708 [00:08<00:24, 82.74it/s]
 24%|██▍       | 657/2708 [00:08<00:24, 82.89it/s]
 25%|██▍       | 666/2708 [00:08<00:24, 81.89it/s]
 25%|██▍       | 675/2708 [00:08<00:24, 82.37it/s]
 25%|██▌       | 684/2708 [00:08<00:24, 82.61it/s]
 26%|██▌       | 693/2708 [00:08<00:24, 82.80it/s]
 26%|██▌       | 702/2708 [00:09<00:24, 82.80it/s]
 26%|██▋       | 711/2708 [00:09<00:24, 82.91it/s]
 27%|██▋       | 720/2708 [00:09<00:24, 82.36it/s]
 27%|██▋       | 729/2708 [00:09<00:23, 82.69it/s]
 27%|██▋       | 738/2708 [00:09<00:23, 82.89it/s]
 28%|██▊       | 747/2708 [00:09<00:23, 82.96it/s]
 28%|██▊       | 756/2708 [00:09<00:23, 83.12it/s]
 28%|██▊       | 765/2708 [00:09<00:23, 83.13it/s]
 29%|██▊       | 774/2708 [00:09<00:23, 82.10it/s]
 29%|██▉       | 783/2708 [00:10<00:23, 82.24it/s]
 29%|██▉       | 792/2708 [00:10<00:23, 82.08it/s]
 30%|██▉       | 801/2708 [00:10<00:23, 82.01it/s]
 30%|██▉       | 810/2708 [00:10<00:23, 82.19it/s]
 30%|███       | 819/2708 [00:10<00:22, 82.24it/s]
 31%|███       | 828/2708 [00:10<00:23, 81.51it/s]
 31%|███       | 837/2708 [00:10<00:22, 81.79it/s]
 31%|███       | 846/2708 [00:10<00:22, 81.99it/s]
 32%|███▏      | 855/2708 [00:10<00:22, 82.03it/s]
 32%|███▏      | 864/2708 [00:11<00:22, 82.14it/s]
 32%|███▏      | 873/2708 [00:11<00:22, 82.27it/s]
 33%|███▎      | 882/2708 [00:11<00:22, 81.32it/s]
 33%|███▎      | 891/2708 [00:11<00:22, 81.65it/s]
 33%|███▎      | 900/2708 [00:11<00:22, 81.88it/s]
 34%|███▎      | 909/2708 [00:11<00:21, 82.03it/s]
 34%|███▍      | 918/2708 [00:11<00:21, 81.85it/s]
 34%|███▍      | 927/2708 [00:11<00:21, 81.99it/s]
 35%|███▍      | 936/2708 [00:11<00:21, 81.75it/s]
 35%|███▍      | 945/2708 [00:12<00:21, 81.73it/s]
 35%|███▌      | 954/2708 [00:12<00:21, 81.79it/s]
 36%|███▌      | 963/2708 [00:12<00:21, 82.03it/s]
 36%|███▌      | 972/2708 [00:12<00:21, 82.17it/s]
 36%|███▌      | 981/2708 [00:12<00:21, 82.09it/s]
 37%|███▋      | 990/2708 [00:12<00:21, 81.57it/s]
 37%|███▋      | 999/2708 [00:12<00:21, 81.36it/s]
 37%|███▋      | 1008/2708 [00:12<00:20, 81.71it/s]
 38%|███▊      | 1017/2708 [00:12<00:20, 81.98it/s]
 38%|███▊      | 1026/2708 [00:13<00:20, 82.18it/s]
 38%|███▊      | 1035/2708 [00:13<00:20, 82.29it/s]
 39%|███▊      | 1044/2708 [00:13<00:20, 81.75it/s]
 39%|███▉      | 1053/2708 [00:13<00:20, 81.97it/s]
 39%|███▉      | 1062/2708 [00:13<00:20, 82.15it/s]
 40%|███▉      | 1071/2708 [00:13<00:19, 81.98it/s]
 40%|███▉      | 1080/2708 [00:13<00:19, 82.21it/s]
 40%|████      | 1089/2708 [00:13<00:19, 82.28it/s]
 41%|████      | 1098/2708 [00:13<00:19, 81.92it/s]
 41%|████      | 1107/2708 [00:14<00:19, 81.49it/s]
 41%|████      | 1116/2708 [00:14<00:19, 81.83it/s]
 42%|████▏     | 1125/2708 [00:14<00:19, 82.21it/s]
 42%|████▏     | 1134/2708 [00:14<00:19, 82.38it/s]
 42%|████▏     | 1143/2708 [00:14<00:18, 82.45it/s]
 43%|████▎     | 1152/2708 [00:14<00:19, 81.83it/s]
 43%|████▎     | 1161/2708 [00:14<00:18, 81.98it/s]
 43%|████▎     | 1170/2708 [00:14<00:18, 81.98it/s]
 44%|████▎     | 1179/2708 [00:14<00:18, 82.12it/s]
 44%|████▍     | 1188/2708 [00:14<00:18, 82.24it/s]
 44%|████▍     | 1197/2708 [00:15<00:18, 82.35it/s]
 45%|████▍     | 1206/2708 [00:15<00:19, 78.96it/s]
 45%|████▍     | 1215/2708 [00:15<00:18, 79.78it/s]
 45%|████▌     | 1224/2708 [00:15<00:18, 80.61it/s]
 46%|████▌     | 1233/2708 [00:15<00:18, 80.99it/s]
 46%|████▌     | 1242/2708 [00:15<00:18, 81.23it/s]
 46%|████▌     | 1251/2708 [00:15<00:17, 81.59it/s]
 47%|████▋     | 1260/2708 [00:15<00:17, 81.24it/s]
 47%|████▋     | 1269/2708 [00:15<00:17, 81.62it/s]
 47%|████▋     | 1278/2708 [00:16<00:17, 81.85it/s]
 48%|████▊     | 1287/2708 [00:16<00:17, 81.90it/s]
 48%|████▊     | 1296/2708 [00:16<00:17, 81.85it/s]
 48%|████▊     | 1305/2708 [00:16<00:17, 81.88it/s]
 49%|████▊     | 1314/2708 [00:16<00:17, 81.42it/s]
 49%|████▉     | 1323/2708 [00:16<00:17, 81.26it/s]
 49%|████▉     | 1332/2708 [00:16<00:16, 81.50it/s]
 50%|████▉     | 1341/2708 [00:16<00:16, 81.62it/s]
 50%|████▉     | 1350/2708 [00:16<00:16, 81.78it/s]
 50%|█████     | 1359/2708 [00:17<00:16, 81.64it/s]
 51%|█████     | 1368/2708 [00:17<00:16, 81.69it/s]
 51%|█████     | 1377/2708 [00:17<00:16, 81.91it/s]
 51%|█████     | 1386/2708 [00:17<00:16, 81.90it/s]
 52%|█████▏    | 1395/2708 [00:17<00:16, 81.84it/s]
 52%|█████▏    | 1404/2708 [00:17<00:15, 82.08it/s]
 52%|█████▏    | 1413/2708 [00:17<00:15, 81.78it/s]
 53%|█████▎    | 1422/2708 [00:17<00:15, 81.78it/s]
 53%|█████▎    | 1431/2708 [00:17<00:15, 81.37it/s]
 53%|█████▎    | 1440/2708 [00:18<00:15, 81.53it/s]
 54%|█████▎    | 1449/2708 [00:18<00:15, 81.80it/s]
 54%|█████▍    | 1458/2708 [00:18<00:15, 81.84it/s]
 54%|█████▍    | 1467/2708 [00:18<00:15, 81.76it/s]
 55%|█████▍    | 1476/2708 [00:18<00:15, 81.64it/s]
 55%|█████▍    | 1485/2708 [00:18<00:14, 81.64it/s]
 55%|█████▌    | 1494/2708 [00:18<00:14, 81.70it/s]
 56%|█████▌    | 1503/2708 [00:18<00:14, 81.79it/s]
 56%|█████▌    | 1512/2708 [00:18<00:14, 81.96it/s]
 56%|█████▌    | 1521/2708 [00:19<00:14, 81.66it/s]
 56%|█████▋    | 1530/2708 [00:19<00:14, 81.57it/s]
 57%|█████▋    | 1539/2708 [00:19<00:14, 80.99it/s]
 57%|█████▋    | 1548/2708 [00:19<00:14, 81.45it/s]
 57%|█████▋    | 1557/2708 [00:19<00:14, 81.79it/s]
 58%|█████▊    | 1566/2708 [00:19<00:13, 82.02it/s]
 58%|█████▊    | 1575/2708 [00:19<00:13, 81.74it/s]
 58%|█████▊    | 1584/2708 [00:19<00:13, 81.94it/s]
 59%|█████▉    | 1593/2708 [00:19<00:13, 81.93it/s]
 59%|█████▉    | 1602/2708 [00:20<00:13, 82.27it/s]
 59%|█████▉    | 1611/2708 [00:20<00:13, 82.26it/s]
 60%|█████▉    | 1620/2708 [00:20<00:13, 82.10it/s]
 60%|██████    | 1629/2708 [00:20<00:13, 81.82it/s]
 60%|██████    | 1638/2708 [00:20<00:13, 81.94it/s]
 61%|██████    | 1647/2708 [00:20<00:12, 81.95it/s]
 61%|██████    | 1656/2708 [00:20<00:12, 81.53it/s]
 61%|██████▏   | 1665/2708 [00:20<00:12, 81.80it/s]
 62%|██████▏   | 1674/2708 [00:20<00:12, 81.75it/s]
 62%|██████▏   | 1683/2708 [00:21<00:12, 81.26it/s]
 62%|██████▏   | 1692/2708 [00:21<00:12, 81.48it/s]
 63%|██████▎   | 1701/2708 [00:21<00:12, 81.62it/s]
 63%|██████▎   | 1710/2708 [00:21<00:12, 81.94it/s]
 63%|██████▎   | 1719/2708 [00:21<00:12, 81.95it/s]
 64%|██████▍   | 1728/2708 [00:21<00:11, 82.09it/s]
 64%|██████▍   | 1737/2708 [00:21<00:11, 81.55it/s]
 64%|██████▍   | 1746/2708 [00:21<00:11, 81.82it/s]
 65%|██████▍   | 1755/2708 [00:21<00:11, 82.07it/s]
 65%|██████▌   | 1764/2708 [00:22<00:11, 81.98it/s]
 65%|██████▌   | 1773/2708 [00:22<00:11, 81.92it/s]
 66%|██████▌   | 1782/2708 [00:22<00:11, 82.11it/s]
 66%|██████▌   | 1791/2708 [00:22<00:11, 81.53it/s]
 66%|██████▋   | 1800/2708 [00:22<00:11, 81.44it/s]
 67%|██████▋   | 1809/2708 [00:22<00:11, 81.56it/s]
 67%|██████▋   | 1818/2708 [00:22<00:10, 81.83it/s]
 67%|██████▋   | 1827/2708 [00:22<00:10, 82.05it/s]
 68%|██████▊   | 1836/2708 [00:22<00:10, 81.83it/s]
 68%|██████▊   | 1845/2708 [00:23<00:10, 78.67it/s]
 68%|██████▊   | 1854/2708 [00:23<00:10, 79.93it/s]
 69%|██████▉   | 1863/2708 [00:23<00:10, 80.98it/s]
 69%|██████▉   | 1872/2708 [00:23<00:10, 80.68it/s]
 69%|██████▉   | 1881/2708 [00:23<00:10, 81.09it/s]
 70%|██████▉   | 1890/2708 [00:23<00:10, 81.49it/s]
 70%|███████   | 1899/2708 [00:23<00:09, 81.14it/s]
 70%|███████   | 1908/2708 [00:23<00:09, 81.58it/s]
 71%|███████   | 1917/2708 [00:23<00:09, 81.66it/s]
 71%|███████   | 1926/2708 [00:24<00:09, 81.72it/s]
 71%|███████▏  | 1935/2708 [00:24<00:09, 81.78it/s]
 72%|███████▏  | 1944/2708 [00:24<00:09, 82.03it/s]
 72%|███████▏  | 1953/2708 [00:24<00:09, 81.21it/s]
 72%|███████▏  | 1962/2708 [00:24<00:09, 81.64it/s]
 73%|███████▎  | 1971/2708 [00:24<00:08, 82.10it/s]
 73%|███████▎  | 1980/2708 [00:24<00:08, 81.62it/s]
 73%|███████▎  | 1989/2708 [00:24<00:08, 81.95it/s]
 74%|███████▍  | 1998/2708 [00:24<00:08, 82.10it/s]
 74%|███████▍  | 2007/2708 [00:25<00:08, 81.51it/s]
 74%|███████▍  | 2016/2708 [00:25<00:08, 81.82it/s]
 75%|███████▍  | 2025/2708 [00:25<00:08, 82.04it/s]
 75%|███████▌  | 2034/2708 [00:25<00:08, 82.24it/s]
 75%|███████▌  | 2043/2708 [00:25<00:08, 82.33it/s]
 76%|███████▌  | 2052/2708 [00:25<00:07, 82.23it/s]
 76%|███████▌  | 2061/2708 [00:25<00:07, 81.86it/s]
 76%|███████▋  | 2070/2708 [00:25<00:07, 82.06it/s]
 77%|███████▋  | 2079/2708 [00:25<00:07, 82.19it/s]
 77%|███████▋  | 2088/2708 [00:26<00:07, 81.83it/s]
 77%|███████▋  | 2097/2708 [00:26<00:07, 82.05it/s]
 78%|███████▊  | 2106/2708 [00:26<00:07, 82.03it/s]
 78%|███████▊  | 2115/2708 [00:26<00:07, 81.73it/s]
 78%|███████▊  | 2124/2708 [00:26<00:07, 81.96it/s]
 79%|███████▉  | 2133/2708 [00:26<00:07, 81.95it/s]
 79%|███████▉  | 2142/2708 [00:26<00:06, 82.15it/s]
 79%|███████▉  | 2151/2708 [00:26<00:06, 82.24it/s]
 80%|███████▉  | 2160/2708 [00:26<00:06, 82.38it/s]
 80%|████████  | 2169/2708 [00:27<00:06, 81.95it/s]
 80%|████████  | 2178/2708 [00:27<00:06, 82.13it/s]
 81%|████████  | 2187/2708 [00:27<00:06, 82.29it/s]
 81%|████████  | 2196/2708 [00:27<00:06, 82.58it/s]
 81%|████████▏ | 2205/2708 [00:27<00:06, 81.67it/s]
 82%|████████▏ | 2214/2708 [00:27<00:06, 81.91it/s]
 82%|████████▏ | 2223/2708 [00:27<00:05, 81.68it/s]
 82%|████████▏ | 2232/2708 [00:27<00:05, 81.91it/s]
 83%|████████▎ | 2241/2708 [00:27<00:05, 82.16it/s]
 83%|████████▎ | 2250/2708 [00:27<00:05, 82.27it/s]
 83%|████████▎ | 2259/2708 [00:28<00:05, 82.34it/s]
 84%|████████▍ | 2268/2708 [00:28<00:05, 82.25it/s]
 84%|████████▍ | 2277/2708 [00:28<00:05, 81.90it/s]
 84%|████████▍ | 2286/2708 [00:28<00:05, 82.21it/s]
 85%|████████▍ | 2295/2708 [00:28<00:05, 82.34it/s]
 85%|████████▌ | 2304/2708 [00:28<00:04, 82.18it/s]
 85%|████████▌ | 2313/2708 [00:28<00:04, 81.67it/s]
 86%|████████▌ | 2322/2708 [00:28<00:04, 81.96it/s]
 86%|████████▌ | 2331/2708 [00:28<00:04, 81.67it/s]
 86%|████████▋ | 2340/2708 [00:29<00:04, 81.94it/s]
 87%|████████▋ | 2349/2708 [00:29<00:04, 81.95it/s]
 87%|████████▋ | 2358/2708 [00:29<00:04, 82.37it/s]
 87%|████████▋ | 2367/2708 [00:29<00:04, 82.40it/s]
 88%|████████▊ | 2376/2708 [00:29<00:04, 82.20it/s]
 88%|████████▊ | 2385/2708 [00:29<00:03, 81.63it/s]
 88%|████████▊ | 2394/2708 [00:29<00:03, 81.69it/s]
 89%|████████▊ | 2403/2708 [00:29<00:03, 81.92it/s]
 89%|████████▉ | 2412/2708 [00:29<00:03, 82.14it/s]
 89%|████████▉ | 2421/2708 [00:30<00:03, 81.56it/s]
 90%|████████▉ | 2430/2708 [00:30<00:03, 81.85it/s]
 90%|█████████ | 2439/2708 [00:30<00:03, 81.45it/s]
 90%|█████████ | 2448/2708 [00:30<00:03, 81.74it/s]
 91%|█████████ | 2457/2708 [00:30<00:03, 82.02it/s]
 91%|█████████ | 2466/2708 [00:30<00:02, 82.19it/s]
 91%|█████████▏| 2475/2708 [00:30<00:02, 82.53it/s]
 92%|█████████▏| 2484/2708 [00:30<00:02, 82.55it/s]
 92%|█████████▏| 2493/2708 [00:30<00:02, 79.06it/s]
 92%|█████████▏| 2502/2708 [00:31<00:02, 80.34it/s]
 93%|█████████▎| 2511/2708 [00:31<00:02, 81.24it/s]
 93%|█████████▎| 2520/2708 [00:31<00:02, 81.56it/s]
 93%|█████████▎| 2529/2708 [00:31<00:02, 81.59it/s]
 94%|█████████▎| 2538/2708 [00:31<00:02, 81.74it/s]
 94%|█████████▍| 2547/2708 [00:31<00:01, 81.52it/s]
 94%|█████████▍| 2556/2708 [00:31<00:01, 81.77it/s]
 95%|█████████▍| 2565/2708 [00:31<00:01, 81.85it/s]
 95%|█████████▌| 2574/2708 [00:31<00:01, 82.03it/s]
 95%|█████████▌| 2583/2708 [00:32<00:01, 82.21it/s]
 96%|█████████▌| 2592/2708 [00:32<00:01, 82.30it/s]
 96%|█████████▌| 2601/2708 [00:32<00:01, 81.75it/s]
 96%|█████████▋| 2610/2708 [00:32<00:01, 82.01it/s]
 97%|█████████▋| 2619/2708 [00:32<00:01, 82.22it/s]
 97%|█████████▋| 2628/2708 [00:32<00:00, 82.29it/s]
 97%|█████████▋| 2637/2708 [00:32<00:00, 82.07it/s]
 98%|█████████▊| 2646/2708 [00:32<00:00, 82.00it/s]
 98%|█████████▊| 2655/2708 [00:32<00:00, 81.72it/s]
 98%|█████████▊| 2664/2708 [00:33<00:00, 81.56it/s]
 99%|█████████▊| 2673/2708 [00:33<00:00, 81.87it/s]
 99%|█████████▉| 2682/2708 [00:33<00:00, 81.89it/s]
 99%|█████████▉| 2691/2708 [00:33<00:00, 82.25it/s]
100%|█████████▉| 2700/2708 [00:33<00:00, 82.39it/s]
100%|██████████| 2708/2708 [00:33<00:00, 80.64it/s]
INFO:TestAccuracy:Batch size is 4, F1: 80.15325, Exact Match:72.95975
INFO:PerfEngine:******************************************* Runing QPS Checker... *******************************************
2024-10-29 10:23:39.494164276 [W:onnxruntime:, session_state.cc:1169 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-10-29 10:23:39.494180436 [W:onnxruntime:, session_state.cc:1171 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
INFO:BackendDCU:Batch size is 4, QPS: 375, Avg Latency:10.66, Tail Latency:12.69
2024-10-29 10:23:41.914490180 [W:onnxruntime:, session_state.cc:1169 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-10-29 10:23:41.914506778 [W:onnxruntime:, session_state.cc:1171 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
INFO:BackendDCU:Batch size is 8, QPS: 448, Avg Latency:17.82, Tail Latency:19.7
2024-10-29 10:23:45.306684295 [W:onnxruntime:, session_state.cc:1169 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-10-29 10:23:45.306699721 [W:onnxruntime:, session_state.cc:1171 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
INFO:BackendDCU:Batch size is 16, QPS: 486, Avg Latency:32.92, Tail Latency:34.87
2024-10-29 10:23:50.633233948 [W:onnxruntime:, session_state.cc:1169 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-10-29 10:23:50.633250336 [W:onnxruntime:, session_state.cc:1171 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
INFO:BackendDCU:Batch size is 32, QPS: 514, Avg Latency:62.17, Tail Latency:63.92
2024-10-29 10:23:59.768382259 [W:onnxruntime:, session_state.cc:1169 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-10-29 10:23:59.768398238 [W:onnxruntime:, session_state.cc:1171 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
INFO:BackendDCU:Batch size is 64, QPS: 537, Avg Latency:119.1, Tail Latency:121.3
2024-10-29 10:24:16.353528795 [W:onnxruntime:, session_state.cc:1169 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-10-29 10:24:16.353545488 [W:onnxruntime:, session_state.cc:1171 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
INFO:BackendDCU:Batch size is 128, QPS: 514, Avg Latency:248.69, Tail Latency:277.44
INFO:PerfEngine:Testing Finish. Report is saved in path: [ general_perf/reports/DCU/bert-onnxruntime-fp16/result-fp16.json ]
INFO:PerfEngine:PDF Version is saved in path: [ general_perf/reports/DCU/bert-onnxruntime-fp16/BERT-ONNXRUNTIME-FP16-TO-FP16.JSON.pdf ]
Writing predictions to: /home/workspace/ByteMLPerf/byte_infer_perf/general_perf/reports/DCU/predictions.json