INFO:LANUCH:******************* Pip Package Installing *******************
INFO:PerfEngine:******************* Backend Env Initization *******************
INFO:BackendStore:Loading Compile Backend: DCU
INFO:BackendStore:Loading Runtime Backend: DCU
INFO:PerfEngine:******************************************* Start to test model: bert-onnxruntime-fp32. *******************************************
INFO:PerfEngine:******************************************* Running Backend Compilation... *******************************************
INFO:PerfEngine:Running Backend Preoptimization...
INFO:DatasetStore:Loading Dataset: open_squad
INFO:SQUAD:Initial...
INFO:SQUAD:Preprocessing...
INFO:SQUAD:Rebatching batch size to: 10833 ...

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:00<00:00, 98.69it/s]
INFO:PerfEngine:Start to compile the model...
2024-10-29 10:25:05.286401290 [W:onnxruntime:, session_state.cc:1169 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-10-29 10:25:05.286419867 [W:onnxruntime:, session_state.cc:1171 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
INFO:PerfEngine:******************************************* Running Accuracy Checker... *******************************************
INFO:SQUAD:Rebatching batch size to: 4 ...

  0%|          | 0/2708 [00:00<?, ?it/s]
100%|██████████| 2708/2708 [00:00<00:00, 135118.25it/s]
INFO:TestAccuracy:Start to calculate accuracy...

  0%|          | 0/2708 [00:00<?, ?it/s]
  0%|          | 1/2708 [00:00<20:41,  2.18it/s]
  0%|          | 6/2708 [00:00<03:20, 13.45it/s]
  0%|          | 11/2708 [00:00<02:00, 22.46it/s]
  1%|          | 16/2708 [00:00<01:31, 29.27it/s]
  1%|          | 21/2708 [00:00<01:17, 34.54it/s]
  1%|          | 26/2708 [00:00<01:09, 38.46it/s]
  1%|          | 31/2708 [00:01<01:04, 41.31it/s]
  1%|▏         | 36/2708 [00:01<01:01, 43.38it/s]
  2%|▏         | 41/2708 [00:01<00:59, 44.78it/s]
  2%|▏         | 46/2708 [00:01<00:58, 45.80it/s]
  2%|▏         | 51/2708 [00:01<00:57, 46.37it/s]
  2%|▏         | 56/2708 [00:01<00:56, 46.81it/s]
  2%|▏         | 61/2708 [00:01<00:56, 47.12it/s]
  2%|▏         | 66/2708 [00:01<00:55, 47.36it/s]
  3%|▎         | 71/2708 [00:01<00:55, 47.33it/s]
  3%|▎         | 76/2708 [00:02<00:55, 47.29it/s]
  3%|▎         | 81/2708 [00:02<00:55, 47.49it/s]
  3%|▎         | 86/2708 [00:02<00:54, 47.76it/s]
  3%|▎         | 91/2708 [00:02<00:54, 47.91it/s]
  4%|▎         | 96/2708 [00:02<00:54, 48.06it/s]
  4%|▎         | 101/2708 [00:02<00:54, 48.05it/s]
  4%|▍         | 106/2708 [00:02<00:54, 48.08it/s]
  4%|▍         | 111/2708 [00:02<00:54, 48.02it/s]
  4%|▍         | 116/2708 [00:02<00:53, 48.02it/s]
  4%|▍         | 121/2708 [00:02<00:53, 48.03it/s]
  5%|▍         | 126/2708 [00:03<00:53, 47.89it/s]
  5%|▍         | 131/2708 [00:03<00:53, 47.95it/s]
  5%|▌         | 136/2708 [00:03<00:53, 48.05it/s]
  5%|▌         | 141/2708 [00:03<00:53, 47.76it/s]
  5%|▌         | 146/2708 [00:03<00:53, 47.85it/s]
  6%|▌         | 151/2708 [00:03<00:53, 48.00it/s]
  6%|▌         | 156/2708 [00:03<00:53, 47.98it/s]
  6%|▌         | 161/2708 [00:03<00:53, 47.98it/s]
  6%|▌         | 166/2708 [00:03<00:52, 48.06it/s]
  6%|▋         | 171/2708 [00:03<00:52, 48.06it/s]
  6%|▋         | 176/2708 [00:04<00:52, 47.94it/s]
  7%|▋         | 181/2708 [00:04<00:52, 47.99it/s]
  7%|▋         | 186/2708 [00:04<00:52, 48.02it/s]
  7%|▋         | 191/2708 [00:04<00:52, 48.05it/s]
  7%|▋         | 196/2708 [00:04<00:52, 48.05it/s]
  7%|▋         | 201/2708 [00:04<00:52, 48.06it/s]
  8%|▊         | 206/2708 [00:04<00:52, 47.80it/s]
  8%|▊         | 211/2708 [00:04<00:52, 47.84it/s]
  8%|▊         | 216/2708 [00:04<00:52, 47.90it/s]
  8%|▊         | 221/2708 [00:05<00:51, 47.92it/s]
  8%|▊         | 226/2708 [00:05<00:51, 48.05it/s]
  9%|▊         | 231/2708 [00:05<00:51, 47.85it/s]
  9%|▊         | 236/2708 [00:05<00:51, 47.84it/s]
  9%|▉         | 241/2708 [00:05<00:51, 47.91it/s]
  9%|▉         | 246/2708 [00:05<00:51, 47.92it/s]
  9%|▉         | 251/2708 [00:05<00:51, 48.00it/s]
  9%|▉         | 256/2708 [00:05<00:51, 47.91it/s]
 10%|▉         | 261/2708 [00:05<00:51, 47.95it/s]
 10%|▉         | 266/2708 [00:05<00:50, 47.96it/s]
 10%|█         | 271/2708 [00:06<00:50, 47.82it/s]
 10%|█         | 276/2708 [00:06<00:50, 47.86it/s]
 10%|█         | 281/2708 [00:06<00:50, 47.92it/s]
 11%|█         | 286/2708 [00:06<00:50, 47.93it/s]
 11%|█         | 291/2708 [00:06<00:50, 48.00it/s]
 11%|█         | 296/2708 [00:06<00:50, 48.12it/s]
 11%|█         | 301/2708 [00:06<00:49, 48.22it/s]
 11%|█▏        | 306/2708 [00:06<00:49, 48.20it/s]
 11%|█▏        | 311/2708 [00:06<00:49, 48.18it/s]
 12%|█▏        | 316/2708 [00:07<00:49, 48.17it/s]
 12%|█▏        | 321/2708 [00:07<00:49, 48.11it/s]
 12%|█▏        | 326/2708 [00:07<00:49, 48.11it/s]
 12%|█▏        | 331/2708 [00:07<00:49, 48.14it/s]
 12%|█▏        | 336/2708 [00:07<00:49, 47.89it/s]
 13%|█▎        | 341/2708 [00:07<00:49, 47.87it/s]
 13%|█▎        | 346/2708 [00:07<00:49, 47.94it/s]
 13%|█▎        | 351/2708 [00:07<00:49, 47.98it/s]
 13%|█▎        | 356/2708 [00:07<00:49, 48.00it/s]
 13%|█▎        | 361/2708 [00:07<00:48, 48.05it/s]
 14%|█▎        | 366/2708 [00:08<00:48, 48.03it/s]
 14%|█▎        | 371/2708 [00:08<00:48, 48.06it/s]
 14%|█▍        | 376/2708 [00:08<00:48, 48.14it/s]
 14%|█▍        | 381/2708 [00:08<00:48, 48.18it/s]
 14%|█▍        | 386/2708 [00:08<00:48, 48.18it/s]
 14%|█▍        | 391/2708 [00:08<00:48, 48.10it/s]
 15%|█▍        | 396/2708 [00:08<00:47, 48.19it/s]
 15%|█▍        | 401/2708 [00:08<00:48, 47.95it/s]
 15%|█▍        | 406/2708 [00:08<00:47, 48.07it/s]
 15%|█▌        | 411/2708 [00:08<00:47, 48.05it/s]
 15%|█▌        | 416/2708 [00:09<00:47, 48.10it/s]
 16%|█▌        | 421/2708 [00:09<00:47, 48.13it/s]
 16%|█▌        | 426/2708 [00:09<00:47, 48.14it/s]
 16%|█▌        | 431/2708 [00:09<00:47, 48.16it/s]
 16%|█▌        | 436/2708 [00:09<00:47, 48.21it/s]
 16%|█▋        | 441/2708 [00:09<00:46, 48.24it/s]
 16%|█▋        | 446/2708 [00:09<00:47, 48.06it/s]
 17%|█▋        | 451/2708 [00:09<00:46, 48.07it/s]
 17%|█▋        | 456/2708 [00:09<00:46, 48.12it/s]
 17%|█▋        | 461/2708 [00:10<00:46, 48.08it/s]
 17%|█▋        | 466/2708 [00:10<00:46, 47.82it/s]
 17%|█▋        | 471/2708 [00:10<00:46, 47.94it/s]
 18%|█▊        | 476/2708 [00:10<00:46, 47.96it/s]
 18%|█▊        | 481/2708 [00:10<00:46, 48.14it/s]
 18%|█▊        | 486/2708 [00:10<00:46, 48.18it/s]
 18%|█▊        | 491/2708 [00:10<00:46, 48.20it/s]
 18%|█▊        | 496/2708 [00:10<00:45, 48.19it/s]
 19%|█▊        | 501/2708 [00:10<00:45, 48.07it/s]
 19%|█▊        | 506/2708 [00:10<00:45, 48.00it/s]
 19%|█▉        | 511/2708 [00:11<00:45, 48.04it/s]
 19%|█▉        | 516/2708 [00:11<00:45, 48.16it/s]
 19%|█▉        | 521/2708 [00:11<00:45, 48.16it/s]
 19%|█▉        | 526/2708 [00:11<00:45, 48.22it/s]
 20%|█▉        | 531/2708 [00:11<00:45, 48.01it/s]
 20%|█▉        | 536/2708 [00:11<00:45, 48.06it/s]
 20%|█▉        | 541/2708 [00:11<00:45, 47.97it/s]
 20%|██        | 546/2708 [00:11<00:44, 48.09it/s]
 20%|██        | 551/2708 [00:11<00:44, 48.07it/s]
 21%|██        | 556/2708 [00:12<00:46, 46.60it/s]
 21%|██        | 561/2708 [00:12<00:45, 47.07it/s]
 21%|██        | 566/2708 [00:12<00:45, 47.42it/s]
 21%|██        | 571/2708 [00:12<00:44, 47.60it/s]
 21%|██▏       | 576/2708 [00:12<00:44, 47.74it/s]
 21%|██▏       | 581/2708 [00:12<00:44, 47.84it/s]
 22%|██▏       | 586/2708 [00:12<00:44, 47.92it/s]
 22%|██▏       | 591/2708 [00:12<00:44, 47.82it/s]
 22%|██▏       | 596/2708 [00:12<00:44, 47.96it/s]
 22%|██▏       | 601/2708 [00:12<00:43, 48.05it/s]
 22%|██▏       | 606/2708 [00:13<00:43, 47.82it/s]
 23%|██▎       | 611/2708 [00:13<00:43, 47.95it/s]
 23%|██▎       | 616/2708 [00:13<00:43, 48.02it/s]
 23%|██▎       | 621/2708 [00:13<00:43, 48.01it/s]
 23%|██▎       | 626/2708 [00:13<00:43, 48.07it/s]
 23%|██▎       | 631/2708 [00:13<00:43, 48.14it/s]
 23%|██▎       | 636/2708 [00:13<00:43, 48.04it/s]
 24%|██▎       | 641/2708 [00:13<00:43, 48.06it/s]
 24%|██▍       | 646/2708 [00:13<00:42, 48.04it/s]
 24%|██▍       | 651/2708 [00:14<00:42, 47.91it/s]
 24%|██▍       | 656/2708 [00:14<00:43, 47.65it/s]
 24%|██▍       | 661/2708 [00:14<00:42, 47.72it/s]
 25%|██▍       | 666/2708 [00:14<00:42, 47.82it/s]
 25%|██▍       | 671/2708 [00:14<00:42, 47.88it/s]
 25%|██▍       | 676/2708 [00:14<00:42, 47.91it/s]
 25%|██▌       | 681/2708 [00:14<00:42, 48.00it/s]
 25%|██▌       | 686/2708 [00:14<00:42, 48.04it/s]
 26%|██▌       | 691/2708 [00:14<00:42, 48.02it/s]
 26%|██▌       | 696/2708 [00:14<00:41, 48.08it/s]
 26%|██▌       | 701/2708 [00:15<00:41, 48.17it/s]
 26%|██▌       | 706/2708 [00:15<00:41, 48.17it/s]
 26%|██▋       | 711/2708 [00:15<00:41, 48.15it/s]
 26%|██▋       | 716/2708 [00:15<00:41, 47.97it/s]
 27%|██▋       | 721/2708 [00:15<00:41, 47.65it/s]
 27%|██▋       | 726/2708 [00:15<00:41, 47.87it/s]
 27%|██▋       | 731/2708 [00:15<00:41, 47.92it/s]
 27%|██▋       | 736/2708 [00:15<00:41, 47.96it/s]
 27%|██▋       | 741/2708 [00:15<00:41, 47.97it/s]
 28%|██▊       | 746/2708 [00:15<00:40, 48.11it/s]
 28%|██▊       | 751/2708 [00:16<00:40, 48.17it/s]
 28%|██▊       | 756/2708 [00:16<00:40, 48.12it/s]
 28%|██▊       | 761/2708 [00:16<00:40, 48.12it/s]
 28%|██▊       | 766/2708 [00:16<00:40, 48.08it/s]
 28%|██▊       | 771/2708 [00:16<00:40, 47.93it/s]
 29%|██▊       | 776/2708 [00:16<00:40, 47.95it/s]
 29%|██▉       | 781/2708 [00:16<00:40, 47.98it/s]
 29%|██▉       | 786/2708 [00:16<00:40, 47.79it/s]
 29%|██▉       | 791/2708 [00:16<00:40, 47.88it/s]
 29%|██▉       | 796/2708 [00:17<00:40, 47.79it/s]
 30%|██▉       | 801/2708 [00:17<00:39, 47.86it/s]
 30%|██▉       | 806/2708 [00:17<00:39, 47.91it/s]
 30%|██▉       | 811/2708 [00:17<00:39, 48.05it/s]
 30%|███       | 816/2708 [00:17<00:39, 47.99it/s]
 30%|███       | 821/2708 [00:17<00:39, 47.86it/s]
 31%|███       | 826/2708 [00:17<00:39, 47.95it/s]
 31%|███       | 831/2708 [00:17<00:39, 47.93it/s]
 31%|███       | 836/2708 [00:17<00:38, 48.04it/s]
 31%|███       | 841/2708 [00:17<00:38, 48.05it/s]
 31%|███       | 846/2708 [00:18<00:38, 48.00it/s]
 31%|███▏      | 851/2708 [00:18<00:38, 47.80it/s]
 32%|███▏      | 856/2708 [00:18<00:38, 47.84it/s]
 32%|███▏      | 861/2708 [00:18<00:38, 47.96it/s]
 32%|███▏      | 866/2708 [00:18<00:38, 48.01it/s]
 32%|███▏      | 871/2708 [00:18<00:38, 48.03it/s]
 32%|███▏      | 876/2708 [00:18<00:38, 47.88it/s]
 33%|███▎      | 881/2708 [00:18<00:38, 47.83it/s]
 33%|███▎      | 886/2708 [00:18<00:38, 47.87it/s]
 33%|███▎      | 891/2708 [00:19<00:37, 47.90it/s]
 33%|███▎      | 896/2708 [00:19<00:37, 47.99it/s]
 33%|███▎      | 901/2708 [00:19<00:37, 47.93it/s]
 33%|███▎      | 906/2708 [00:19<00:37, 47.96it/s]
 34%|███▎      | 911/2708 [00:19<00:37, 47.94it/s]
 34%|███▍      | 916/2708 [00:19<00:37, 47.67it/s]
 34%|███▍      | 921/2708 [00:19<00:37, 47.88it/s]
 34%|███▍      | 926/2708 [00:19<00:37, 47.91it/s]
 34%|███▍      | 931/2708 [00:19<00:37, 47.84it/s]
 35%|███▍      | 936/2708 [00:19<00:37, 47.80it/s]
 35%|███▍      | 941/2708 [00:20<00:36, 47.87it/s]
 35%|███▍      | 946/2708 [00:20<00:36, 47.81it/s]