Intel NCS2 vs. Intel NCS1

As of year 2024, it looks NCSDK is NOT maintained. To clarify the relationships among OpenVINO, NCSDK master and NCSDK ncsdk2 is a kind of complicated.

1. OpenVINO Runtime NO Longer Supports Movidius VPU

1.1 OpenVINO Toolkit Release Notes 2023.0

OpenVINO NOT Support NCS2

And it looks Long Term Support (LTS) version of OpenVINO ending in December 2024 now still supports for the devices listed above.

Therefore, in order to get my old devices: 1 NCS2 and 1 NCS1 working in the year of 2024, I’ll have to resort to Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS).

1.2 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3

OpenVINO LTS Supports Movidius

1.3 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3

Ever since LTS 2023.3, OpenVINO started emphasize GenAI. On its release notes, nowhere is talking about NCS2 or NCS1 or anything related to Movidius VPU. According to OpenVINO Toolkit Release Notes 2023.0 in the above, I think ever since 2023, OpenVINO turns its direction to GenAI, and NEVER support for Movidius VPU any more. Therefore, most probably, including this Long-Term Support version, Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3 does NOT have any support to Movidius VPU devices.

OpenVINO GenAI Since 2023

1.4 OpenVINO 2024.2 @ Intel® Distribution of OpenVINO™ Toolkit Release Notes 2024.2

Not to mention the MOST UP-TO-DATE version 2024.2, which seems to be NOT a LTS. Of course, it does NOT support Movidius VPU devices.

1.5 Conclusion

Go for:

2. NCSDK Does NOT Support AArch64

2.1 get_mvcmd.sh is the key.

2.2 Firmware Is Actually Downloaded But Without AArch64 Support

1
2
3
4
5
6
7
8
➜  ncsdk ls NCSDK-2.10.01.01
ncsdk-armv7l ncsdk-x86_64 version.txt
➜ ncsdk ll NCSDK-2.10.01.01/ncsdk-x86_64/fw
Permissions Size User Date Modified Name
.rwxrwxrwx 1.1M root 26 Jan 2019 MvNCAPI-ma2450.mvcmd
➜ ncsdk ll NCSDK-2.10.01.01/ncsdk-armv7l/fw
Permissions Size User Date Modified Name
.rwxrwxrwx 1.1M root 26 Jan 2019 MvNCAPI-ma2450.mvcmd
1
2
3
4
5
6
7
8
➜  ncsdk ls NCSDK-1.12.01.01
install-ncsdk.sh ncsdk-armv7l ncsdk-x86_64 ncsdk.conf requirements.txt requirements_apt.txt tests uninstall-ncsdk.sh version.txt
➜ ncsdk ll NCSDK-1.12.01.01/ncsdk-x86_64/fw
Permissions Size User Date Modified Name
.rwxrwxrwx 866k root 5 Jan 2018 MvNCAPI.mvcmd
➜ ncsdk ll NCSDK-1.12.01.01/ncsdk-armv7l/fw
Permissions Size User Date Modified Name
.rwxrwxrwx 866k root 5 Jan 2018 MvNCAPI.mvcmd

2.3 ncsdk-aarch64 for NCS1

Fortunately, I found ncsdk-aarch64. However, it only provides ncsdk-aarch64’s firmware from NCSDK-1.12.00.01, which seems to be also okay for NCSDK-1.12.01.01.

Let’s try it out.

2.3.1 Find the Right Version

You will see a folder named NCSDK-1.12.01.01 (or NCSDK-1.12.00.01), under which there is the key folder ncsdk-aarch64, which actually contains the Aarch64 firmware for NCS1.

1
2
3
4
5
➜  ncsdk-aarch64 git:(master) ✗ ls
api docs examples install-opencv.sh install.sh LICENSE Makefile NCSDK-1.12.01.01 ncsdk.conf README.md uninstall-opencv.sh uninstall.sh
➜ ncsdk-aarch64 git:(master) ✗ cd NCSDK-1.12.01.01
➜ NCSDK-1.12.01.01 git:(master) ✗ ls
install-ncsdk.sh ncsdk-aarch64 ncsdk-armv7l ncsdk-x86_64 ncsdk.conf requirements.txt requirements_apt.txt tests uninstall-ncsdk.sh version.txt

2.3.2 Installation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
➜  ncsdk-aarch64 git:(master) ✗ cd api/src
➜ src git:(master) ✗ sudo make basicinstall
mkdir -p /usr/local/include/
mkdir -p /usr/local/lib/
cp obj-aarch64/libmvnc.so.0 /usr/local/lib/
ln -fs libmvnc.so.0 /usr/local/lib/libmvnc.so
cp ../include/*.h /usr/local/include/
mkdir -p /usr/local/lib/mvnc
cp mvnc/MvNCAPI.mvcmd /usr/local/lib/mvnc/
mkdir -p /etc/udev/rules.d/
cp 97-usbboot.rules /etc/udev/rules.d/
➜ src git:(master) ✗ sudo make pythoninstall
mkdir -p /usr/local/lib/python3.11/dist-packages
cp -r ../python/mvnc /usr/local/lib/python3.11/dist-packages/
➜ src git:(master) ✗ sudo make postinstall
udevadm control --reload-rules
udevadm trigger
ldconfig

2.3.3 Demonstrate Examples on My Raspberry Pi 5

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
➜  examples git:(master) ✗ make
make -C apps/.
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps'
make -C hello_ncs_cpp/.
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_cpp'

making hello_ncs_cpp
g++ cpp/hello_ncs.cpp -o cpp/hello_ncs_cpp -lmvnc
Created cpp/hello_ncs_cpp executable
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_cpp'
make -C hello_ncs_py/.
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_py'
nothing to make, use 'make run' to run.
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_py'
make -C multistick_cpp/.
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/multistick_cpp'

making googlenet
(cd ../../caffe/GoogLeNet; make compile; cd ../../apps/multistick_cpp; cp ../../caffe/GoogLeNet/graph ./googlenet.graph;)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[4]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[4]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'

making prototxt
Prototxt file already exists

making caffemodel
caffemodel file already exists

making compile
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

Layer inception_3b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_3b/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4a/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4a/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4b/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4c/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4c/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4d/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4d/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4e/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4e/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_5a/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_5a/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_5b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_5b/pool_proj forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making squeezenet
(cd ../../caffe/SqueezeNet; make compile; cd ../../apps/multistick_cpp; cp ../../caffe/SqueezeNet/graph ./squeezenet.graph;)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[4]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[4]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
LICENSE file already exists

making prototxt
Prototxt file already exists

making caffemodel
caffemodel file already exists

making compile
mvNCCompile -w squeezenet_v1.0.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making multistick_cpp
cp googlenet.graph cpp/googlenet.graph;
cp squeezenet.graph cpp/squeezenet.graph;
g++ cpp/multistick.cpp cpp/fp16.c -o cpp/multistick_cpp -lmvnc
cpp/multistick.cpp: In function ‘bool DoInferenceOnImageFile(void*, const char*, int, float*)’:
cpp/multistick.cpp:307:1: warning: control reaches end of non-void function [-Wreturn-type]
307 | }
| ^
Created cpp/multistick_cpp executable
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/multistick_cpp'
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps'
make -C caffe/.
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe'
make -C AlexNet/.
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/AlexNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
chmod +x run.py

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
tensor = tensor.tostring()
Time to Execute : 107.91 ms
USB: Myriad Execution Finished
Time to Execute : 103.26 ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
Bandwidth time
# Name MFLOPs (MB/s) (ms)
======================================================================================================================================================================================================================================
0 data 0.0126457.6 0.002
1 conv1 210.8 7434.1 4.811
2 norm1 0.0 721.2 0.768
3 pool1 0.6 1253.9 0.442
4 pool1_s0 0.0 1372.8 0.049
5 conv2_p0 223.9 596.2 6.099
6 pool1_s1 0.0 1455.7 0.046
7 conv2_p1 223.9 585.2 6.213
8 norm2 0.0 697.8 0.510
9 pool2 0.4 1308.0 0.272
10 conv3 299.0 302.0 8.048
11 conv3_s0 0.0 1582.4 0.039
12 conv4_p0 112.1 543.7 3.219
13 conv3_s1 0.0 1492.2 0.041
14 conv4_p1 112.1 560.1 3.125
15 conv4_p0_s0 0.0 1314.0 0.047
16 conv5_p0 74.8 578.5 2.659
17 conv4_p0_s1 0.0 1317.5 0.047
18 conv5_p1 74.8 580.0 2.652
19 pool5 0.1 918.4 0.090
20 fc6 75.5 2144.1 33.589
21 fc7 33.6 2132.6 15.008
22 fc8 8.2 2651.8 2.949
23 prob 0.0 7.6 0.253
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total inference time 90.98
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w bvlc_alexnet.caffemodel -i ../../data/images/cat.jpg -s 12 -id 281 deploy.prototxt -M 110 -S 255 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result: (1000,)
1) 281 0.512
Expected: (1000,)
1) 281 0.522
------------------------------------------------------------
Obtained values
------------------------------------------------------------
Obtained Min Pixel Accuracy: 2.385406941175461% (max allowed=2%), Fail
Obtained Average Pixel Accuracy: 0.0052463514293776825% (max allowed=1%), Pass
Obtained Percentage of wrong values: 0.1% (max allowed=0%), Fail
Obtained Pixel-wise L2 error: 0.09619102055778048% (max allowed=1%), Pass
Obtained Global Sum Difference: 0.027384519577026367
------------------------------------------------------------

making compile
mvNCCompile -w bvlc_alexnet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/AlexNet'
make -C GoogLeNet/.
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
Layer inception_3b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_3b/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4a/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4a/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4b/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4c/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4c/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4d/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4d/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4e/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4e/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_5a/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_5a/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_5b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_5b/pool_proj forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
tensor = tensor.tostring()
Time to Execute : 130.42 ms
USB: Myriad Execution Finished
Time to Execute : 108.35 ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
Bandwidth time
# Name MFLOPs (MB/s) (ms)
======================================================================================================================================================================================================================================
0 data 0.0 46160.0 0.006
1 conv1/7x7_s2 236.0 2466.8 5.713
2 pool1/3x3_s2 1.8 1340.9 1.142
3 pool1/norm1 0.0 701.9 0.546
4 conv2/3x3_reduce 25.7 478.8 0.816
5 conv2/3x3 693.6 307.6 11.891
6 conv2/norm2 0.0 787.5 1.458
7 pool2/3x3_s2 1.4 1390.2 0.826
8 inception_3a/1x1 19.3 535.1 0.581
9 inception_3a/3x3_reduce 28.9 459.3 0.702
10 inception_3a/3x3 173.4 314.3 4.788
11 inception_3a/5x5_reduce 4.8 1025.6 0.286
12 inception_3a/5x5 20.1 678.0 0.921
13 inception_3a/pool 1.4 578.0 0.497
14 inception_3a/pool_proj 9.6 640.6 0.467
15 inception_3b/1x1 51.4 444.3 1.003
16 inception_3b/3x3_reduce 51.4 446.2 0.998
17 inception_3b/3x3 346.8 257.6 8.339
18 inception_3b/5x5_reduce 12.8 877.2 0.454
19 inception_3b/5x5 120.4 522.8 2.577
20 inception_3b/pool 1.8 660.6 0.580
21 inception_3b/pool_proj 25.7 618.7 0.669
22 pool3/3x3_s2 0.8 1215.8 0.590
23 inception_4a/1x1 36.1 358.6 0.991
24 inception_4a/3x3_reduce 18.1 498.7 0.536
25 inception_4a/3x3 70.4 296.2 2.259
26 inception_4a/5x5_reduce 3.0 756.2 0.257
27 inception_4a/5x5 7.5 412.6 0.457
28 inception_4a/pool 0.8 505.0 0.355
29 inception_4a/pool_proj 12.0 547.8 0.435
30 inception_4b/1x1 32.1 330.5 1.053
31 inception_4b/3x3_reduce 22.5 375.9 0.800
32 inception_4b/3x3 88.5 285.9 2.838
33 inception_4b/5x5_reduce 4.8 573.7 0.375
34 inception_4b/5x5 15.1 309.1 0.973
35 inception_4b/pool 0.9 532.7 0.359
36 inception_4b/pool_proj 12.8 537.0 0.473
37 inception_4c/1x1 25.7 402.8 0.786
38 inception_4c/3x3_reduce 25.7 389.8 0.812
39 inception_4c/3x3 115.6 260.4 3.831
40 inception_4c/5x5_reduce 4.8 584.9 0.367
41 inception_4c/5x5 15.1 308.1 0.976
42 inception_4c/pool 0.9 627.2 0.305
43 inception_4c/pool_proj 12.8 551.2 0.461
44 inception_4d/1x1 22.5 413.7 0.728
45 inception_4d/3x3_reduce 28.9 472.7 0.702
46 inception_4d/3x3 146.3 399.4 3.008
47 inception_4d/5x5_reduce 6.4 637.4 0.349
48 inception_4d/5x5 20.1 388.8 1.028
49 inception_4d/pool 0.9 610.5 0.314
50 inception_4d/pool_proj 12.8 566.9 0.448
51 inception_4e/1x1 53.0 297.6 1.531
52 inception_4e/3x3_reduce 33.1 258.2 1.389
53 inception_4e/3x3 180.6 280.8 5.067
54 inception_4e/5x5_reduce 6.6 493.5 0.465
55 inception_4e/5x5 40.1 356.2 1.405
56 inception_4e/pool 0.9 600.5 0.329
57 inception_4e/pool_proj 26.5 430.8 0.758
58 pool4/3x3_s2 0.4 1221.7 0.255
59 inception_5a/1x1 20.9 614.2 0.789
60 inception_5a/3x3_reduce 13.0 553.5 0.599
61 inception_5a/3x3 45.2 550.4 1.851
62 inception_5a/5x5_reduce 2.6 336.2 0.382
63 inception_5a/5x5 10.0 427.3 0.646
64 inception_5a/pool 0.4 405.1 0.192
65 inception_5a/pool_proj 10.4 484.7 0.580
66 inception_5b/1x1 31.3 611.9 1.124
67 inception_5b/3x3_reduce 15.7 608.1 0.629
68 inception_5b/3x3 65.0 569.9 2.516
69 inception_5b/5x5_reduce 3.9 362.9 0.424
70 inception_5b/5x5 15.1 438.7 0.937
71 inception_5b/pool 0.4 483.8 0.161
72 inception_5b/pool_proj 10.4 502.0 0.560
73 pool5/7x7_s1 0.1 460.8 0.208
74 loss3/classifier 2.0 2497.0 0.783
75 prob 0.0 9.5 0.200
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total inference time 95.91
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w bvlc_googlenet.caffemodel -i ../../data/images/nps_electric_guitar.png -s 12 -id 546 deploy.prototxt -S 255 -M 110 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

Layer inception_3b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_3b/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4a/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4a/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4b/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4c/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4c/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4d/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4d/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4e/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4e/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_5a/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_5a/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_5b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_5b/pool_proj forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result: (1000,)
1) 546 0.995
Expected: (1000,)
1) 546 0.9946
------------------------------------------------------------
Obtained values
------------------------------------------------------------
Obtained Min Pixel Accuracy: 0.049091799883171916% (max allowed=2%), Pass
Obtained Average Pixel Accuracy: 7.989402774910559e-05% (max allowed=1%), Pass
Obtained Percentage of wrong values: 0.0% (max allowed=0%), Pass
Obtained Pixel-wise L2 error: 0.0017114860971367098% (max allowed=1%), Pass
Obtained Global Sum Difference: 0.0007946491241455078
------------------------------------------------------------

making compile
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

Layer inception_3b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_3b/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4a/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4a/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4b/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4c/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4c/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4d/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4d/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_4e/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_4e/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_5a/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_5a/pool_proj forced to im2col_v2, because its output is used in concat
Layer inception_5b/1x1 forced to im2col_v2, because its output is used in concat
Layer inception_5b/pool_proj forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'
make -C SqueezeNet/.
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
LICENSE file already exists

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
tensor = tensor.tostring()
Time to Execute : 78.37 ms
USB: Myriad Execution Finished
Time to Execute : 60.05 ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
Bandwidth time
# Name MFLOPs (MB/s) (ms)
======================================================================================================================================================================================================================================
0 data 0.0 96516.2 0.003
1 conv1 347.7 1634.5 8.861
2 pool1 2.6 1445.0 1.561
3 fire2/squeeze1x1 9.3 1222.9 0.455
4 fire2/expand1x1 6.2 152.1 0.620
5 fire2/expand3x3 55.8 475.0 1.789
6 fire3/squeeze1x1 12.4 1466.3 0.506
7 fire3/expand1x1 6.2 158.4 0.596
8 fire3/expand3x3 55.8 475.9 1.785
9 fire4/squeeze1x1 24.8 976.6 0.764
10 fire4/expand1x1 24.8 174.3 1.106
11 fire4/expand3x3 223.0 391.4 4.430
12 pool4 1.7 1268.5 1.164
13 fire5/squeeze1x1 11.9 788.7 0.471
14 fire5/expand1x1 6.0 151.4 0.347
15 fire5/expand3x3 53.7 376.4 1.257
16 fire6/squeeze1x1 17.9 699.1 0.543
17 fire6/expand1x1 13.4 159.6 0.531
18 fire6/expand3x3 120.9 256.4 2.973
19 fire7/squeeze1x1 26.9 827.4 0.688
20 fire7/expand1x1 13.4 159.9 0.530
21 fire7/expand3x3 120.9 259.7 2.935
22 fire8/squeeze1x1 35.8 735.0 0.790
23 fire8/expand1x1 23.9 155.7 0.775
24 fire8/expand3x3 215.0 201.3 5.396
25 pool8 0.8 1265.2 0.563
26 fire9/squeeze1x1 11.1 587.7 0.387
27 fire9/expand1x1 5.5 153.4 0.341
28 fire9/expand3x3 49.8 288.2 1.636
29 conv10 173.1 331.1 3.448
30 pool10 0.3 671.0 0.480
31 prob 0.0 9.5 0.202
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total inference time 47.94
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w squeezenet_v1.0.caffemodel -i ../../data/images/cat.jpg -s 12 -id 281 deploy.prototxt -S 255 -M 120 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result: (1000, 1, 1)
1) 281 0.6177
Expected: (1000, 1, 1)
1) 281 0.616
------------------------------------------------------------
Obtained values
------------------------------------------------------------
Obtained Min Pixel Accuracy: 0.6933439057320356% (max allowed=2%), Pass
Obtained Average Pixel Accuracy: 0.0016120149666676298% (max allowed=1%), Pass
Obtained Percentage of wrong values: 0.0% (max allowed=0%), Pass
Obtained Pixel-wise L2 error: 0.029018872980330863% (max allowed=1%), Pass
Obtained Global Sum Difference: 0.009933412075042725
------------------------------------------------------------

making compile
mvNCCompile -w squeezenet_v1.0.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe'
make -C data/.
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data'
Possible Make targets
make help - shows this message
make clean - Removes all temp files from all directories
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data'
make -C tensorflow/.
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/tensorflow'
make -C inception_v1/.
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1'
test -f output/inception-v1.meta || ((wget http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz && tar zxf inception_v1_2016_08_28.tar.gz && rm inception_v1_2016_08_28.tar.gz) && ./inception-v1.py)
--2024-07-29 13:06:28-- http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz
Resolving download.tensorflow.org (download.tensorflow.org)... 2607:f8b0:400a:803::201b, 2607:f8b0:400a:805::201b, 2607:f8b0:400a:807::201b, ...
Connecting to download.tensorflow.org (download.tensorflow.org)|2607:f8b0:400a:803::201b|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24642554 (24M) [application/x-tar]
Saving to: ‘inception_v1_2016_08_28.tar.gz’

inception_v1_2016_08_28.tar.gz 100%[====================================================================================================================================>] 23.50M 98.9MB/s in 0.2s

2024-07-29 13:06:28 (98.9 MB/s) - ‘inception_v1_2016_08_28.tar.gz’ saved [24642554/24642554]

Traceback (most recent call last):
File "/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1/./inception-v1.py", line 6, in <module>
from tensorflow.contrib.slim.nets import inception
ModuleNotFoundError: No module named 'tensorflow.contrib'
make[2]: *** [Makefile:47: weights] Error 1
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1'
make[1]: *** [Makefile:12: inception_v1/.] Error 2
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/tensorflow'
make: *** [Makefile:12: tensorflow/.] Error 2

I'm gonna giving up the issue of Tensorflow version incompatibility, for Tensorflow 1.x has NOT been maintained for years. I mean example inception_v1 and inception_v3 are compatible with Tensorflow 1.x, rather than 2.x.

1
2
➜  examples git:(master) ✗ ls tensorflow 
inception_v1 inception_v3 Makefile readme.md

2.4 ncsdk2-aarch64 for NCS2

2.4.1 Download the Right NCSDK-2.10.01.01.tar.gz

  • git clone https://github.com/LE0xUL/ncsdk2_aarch64.git won’t provide the correct NCSDK-2.10.01.01.tar.gz.
1
2
3
4
5
6
7
8
➜  ncsdk2_aarch64 git:(aarch64) ✗ ll
Permissions Size User Date Modified Name
drwxr-xr-x - lvision 29 Jul 15:34 .git
.rw-r--r-- 564 lvision 29 Jul 12:16 .gitattributes
.rw-r--r-- 212 lvision 29 Jul 12:16 .gitignore
......
.rw-r--r-- 132 lvision 29 Jul 12:16 NCSDK-2.10.01.01.tar.gz
......

As you can see, the git cloned NCSDK-2.10.01.01.tar.gz is of size 132 bytes ONLY.

1
2
3
➜  ncsdk2_aarch64 git:(aarch64) ✗ ll NCSDK-2.10.01.01.tar.gz 
Permissions Size User Date Modified Name
.rw-r--r-- 2.4M lvision 29 Jul 16:12 NCSDK-2.10.01.01.tar.gz

After extraction, you’ll also see the most important folder named ncsdk-aarch64, which actually contains the Aarch64 firmware for NCS2.

1
2
3
4
5
6
➜  ncsdk2_aarch64 git:(aarch64) ✗ ls
api docs extras install-utilities.sh LICENSE NCSDK-2.10.01.01 ncsdk.conf requirements.txt requirements_apt_raspbian.txt uninstall-opencv.sh version.txt
ATTRIBUTIONS examples install-opencv.sh install.sh Makefile NCSDK-2.10.01.01.tar.gz README.md requirements_apt.txt tensorflow-1.11.0-cp35-none-linux_aarch64.whl uninstall.sh
➜ ncsdk2_aarch64 git:(aarch64) ✗ cd NCSDK-2.10.01.01
➜ NCSDK-2.10.01.01 git:(aarch64) ✗ ls
ncsdk-aarch64 ncsdk-armv7l ncsdk-x86_64 version.txt

2.4.2 Installation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
➜  ncsdk2_aarch64 git:(aarch64) ✗ cd api/src 
➜ src git:(aarch64) ✗ sudo make basicinstall
NCSDK FW successfully installed
mkdir -p /usr/local/include/
mkdir -p /usr/local/include/mvnc2
mkdir -p /usr/local/lib/
cp obj-aarch64/libmvnc.so.0 /usr/local/lib/
ln -fs libmvnc.so.0 /usr/local/lib/libmvnc.so
cp ../include/mvnc.h /usr/local/include/mvnc2
ln -fs /usr/local/include/mvnc2/mvnc.h /usr/local/include/mvnc.h
mkdir -p /usr/local/lib/mvnc
cp mvnc/MvNCAPI-*.mvcmd /usr/local/lib/mvnc/
mkdir -p /etc/udev/rules.d/
cp 97-ncs2.rules /etc/udev/rules.d/
➜ src git:(aarch64) ✗ sudo make pythoninstall
mkdir -p /usr/local/lib/python3.11/dist-packages
cp -r ../python/mvnc /usr/local/lib/python3.11/dist-packages/
➜ src git:(aarch64) ✗ sudo make postinstall
udevadm control --reload-rules
udevadm trigger
ldconfig

2.4.3 Demonstrate Examples on My Raspberry Pi 5

Let’s just try hello_ncs.py.

1
2
3
4
5
6
7
8
9
10
➜  hello_ncs_py git:(aarch64) ✗ python hello_ncs.py 
D: [ 0] ncDeviceCreate:307 ncDeviceCreate index 0

D: [ 0] ncDeviceCreate:307 ncDeviceCreate index 1

D: [ 0] ncDeviceOpen:523 File path /usr/local/lib/mvnc/MvNCAPI-ma2480.mvcmd

W: [ 0] ncDeviceOpen:527 ncDeviceOpen() XLinkBootRemote returned error 3

Error - Could not open NCS device.

It looks I dealt with this issue in my old blog OpenVINO on Raspberry Pi 4 with Movidius Neural Compute Stick II, where I’d already provided the solution:

OpenVINO should be utilized instead of NCSDK ncsdk2.

3. Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3 for NCS2

3.1 Fix an Error

File /opt/intel/OpenVINO/openvino/thirdparty/ocv/opencv_hal_neon.hpp function v_deinterleave_expand is clearly buggy, for -1 should NEVER appear in an uchar array. Anyway, just change -1 to 255 as a temporary solution, namely:

Change

1
2
3
4
5
6
constexpr int nlanes = static_cast<int>(v_uint8x16::nlanes);
uchar mask_e[nlanes] = { 0, -1, 2, -1, 4, -1, 6, -1,
8, -1, 10, -1, 12, -1, 14, -1 };

uchar mask_o[nlanes] = { 1, -1, 3, -1, 5, -1, 7, -1,
9, -1, 11, -1, 13, -1, 15, -1 };

to:

1
2
3
4
5
6
constexpr int nlanes = static_cast<int>(v_uint8x16::nlanes);
uchar mask_e[nlanes] = { 0, 255, 2, 255, 4, 255, 6, 255,
8, 255, 10, 255, 12, 255, 14, 255 };

uchar mask_o[nlanes] = { 1, 255, 3, 255, 5, 255, 7, 255,
9, 255, 11, 255, 13, 255, 15, 255 };

3.2 Two More Things After Installation

3.2.1 OpenVINO Python Wrapper Installation

Under folder /opt/intel/OpenVINO/openvino/build/wheels, there are the following 2 files:

1
2
3
4
➜  wheels git:(2022.3.2) ✗ ll
Permissions Size User Date Modified Name
.rw-r--r-- 13M lvision 30 Jul 14:09 openvino-2022.3.2-9279-cp311-cp311-manylinux_2_36_aarch64.whl
.rw-r--r-- 24k lvision 30 Jul 13:08 openvino_dev-2022.3.2-9279-py3-none-any.whl

Use key parameter --break-system-packages

1
sudo pip install --break-system-packages openvino-2022.3.2-9279-cp311-cp311-manylinux_2_36_aarch64.whl

and

1
sudo pip install --break-system-packages openvino_dev-2022.3.2-9279-py3-none-any.whl

3.2.2 Copy plugins.xml

A kind of weird that:

  • Under build/lib, there is ONLY 1 file libade.a:
1
2
3
4
➜  lib git:(2022.3.2) ✗ pwd
/opt/intel/OpenVINO/openvino/build/lib
➜ lib git:(2022.3.2) ✗ ls
libade.a
  • Everything is built under bin/aarch64/Release:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
➜  Release git:(2022.3.2) ✗ pwd                      
/opt/intel/OpenVINO/openvino/bin/aarch64/Release
➜ Release git:(2022.3.2) ✗ ll
Permissions Size User Date Modified Name
.rwxr-xr-x 723k lvision 30 Jul 14:08 benchmark_app
.rwxr-xr-x 461k lvision 30 Jul 14:08 benchmark_app_legacy
.rwxr-xr-x 264k lvision 30 Jul 14:08 classification_sample_async
.rwxr-xr-x 264k lvision 30 Jul 14:08 compile_tool
.rwxr-xr-x 199k lvision 30 Jul 14:08 hello_classification
.rwxr-xr-x 67k lvision 30 Jul 14:08 hello_classification_c
.rwxr-xr-x 199k lvision 30 Jul 14:08 hello_nv12_input_classification
.rwxr-xr-x 67k lvision 30 Jul 14:08 hello_nv12_input_classification_c
.rwxr-xr-x 68k lvision 30 Jul 14:08 hello_query_device
.rwxr-xr-x 199k lvision 30 Jul 14:08 hello_reshape_ssd
.rw-r--r-- 367k lvision 30 Jul 12:04 libcnpy.a
.rw-r--r-- 7.5M lvision 30 Jul 14:08 libfluid.a
.rwxr-xr-x 67k lvision 30 Jul 14:08 libformat_reader.so
.rw-r--r-- 3.4M lvision 30 Jul 12:08 libie_docs_snippets.a
.rw-r--r-- 166k lvision 30 Jul 12:34 libie_samples_utils.a
.rw-r--r-- 16M lvision 30 Jul 12:31 libinference_engine_legacy.a
.rw-r--r-- 2.9M lvision 30 Jul 12:36 libinference_engine_snippets.a
.rw-r--r-- 16M lvision 30 Jul 12:56 libinterpreter_backend.a
.rw-r--r-- 3.0k lvision 30 Jul 12:04 libitt.a
.rw-r--r-- 296k lvision 30 Jul 12:05 libmvnc.a
.rw-r--r-- 1.6M lvision 30 Jul 12:04 libngraph_builders.a
.rw-r--r-- 1.6M lvision 30 Jul 14:07 libngraph_reference.a
.rw-r--r-- 2.3M lvision 30 Jul 12:33 liboffline_transformations.a
.rw-r--r-- 11M lvision 30 Jul 12:17 libonnx.a
.rw-r--r-- 54k lvision 30 Jul 12:31 libonnx_common.a
.rw-r--r-- 676k lvision 30 Jul 12:06 libonnx_proto.a
.rwxr-xr-x 67k lvision 30 Jul 12:04 libopencv_c_wrapper.so
lrwxrwxrwx 19 lvision 30 Jul 14:08 libopenvino.so -> libopenvino.so.2232
.rwxr-xr-x 17M lvision 30 Jul 14:08 libopenvino.so.2022.3.2
lrwxrwxrwx 23 lvision 30 Jul 14:08 libopenvino.so.2232 -> libopenvino.so.2022.3.2
.rwxr-xr-x 330k lvision 30 Jul 14:08 libopenvino_auto_batch_plugin.so
.rwxr-xr-x 592k lvision 30 Jul 14:08 libopenvino_auto_plugin.so
lrwxrwxrwx 21 lvision 30 Jul 14:08 libopenvino_c.so -> libopenvino_c.so.2232
.rwxr-xr-x 461k lvision 30 Jul 14:08 libopenvino_c.so.2022.3.2
lrwxrwxrwx 25 lvision 30 Jul 14:08 libopenvino_c.so.2232 -> libopenvino_c.so.2022.3.2
.rwxr-xr-x 1.2M lvision 30 Jul 14:08 libopenvino_gapi_preproc.so
.rwxr-xr-x 395k lvision 30 Jul 14:08 libopenvino_hetero_plugin.so
.rwxr-xr-x 9.3M lvision 30 Jul 14:08 libopenvino_intel_myriad_plugin.so
lrwxrwxrwx 31 lvision 30 Jul 14:08 libopenvino_ir_frontend.so -> libopenvino_ir_frontend.so.2232
.rwxr-xr-x 528k lvision 30 Jul 14:08 libopenvino_ir_frontend.so.2022.3.2
lrwxrwxrwx 35 lvision 30 Jul 14:08 libopenvino_ir_frontend.so.2232 -> libopenvino_ir_frontend.so.2022.3.2
lrwxrwxrwx 33 lvision 30 Jul 14:08 libopenvino_onnx_frontend.so -> libopenvino_onnx_frontend.so.2232
.rwxr-xr-x 4.7M lvision 30 Jul 14:08 libopenvino_onnx_frontend.so.2022.3.2
lrwxrwxrwx 37 lvision 30 Jul 14:08 libopenvino_onnx_frontend.so.2232 -> libopenvino_onnx_frontend.so.2022.3.2
lrwxrwxrwx 35 lvision 30 Jul 14:08 libopenvino_paddle_frontend.so -> libopenvino_paddle_frontend.so.2232
.rwxr-xr-x 1.9M lvision 30 Jul 14:08 libopenvino_paddle_frontend.so.2022.3.2
lrwxrwxrwx 39 lvision 30 Jul 14:08 libopenvino_paddle_frontend.so.2232 -> libopenvino_paddle_frontend.so.2022.3.2
.rwxr-xr-x 68k lvision 30 Jul 14:08 libopenvino_template_extension.so
.rwxr-xr-x 8.0M lvision 30 Jul 14:08 libopenvino_template_plugin.so
lrwxrwxrwx 39 lvision 30 Jul 14:08 libopenvino_tensorflow_frontend.so -> libopenvino_tensorflow_frontend.so.2232
.rwxr-xr-x 3.5M lvision 30 Jul 14:08 libopenvino_tensorflow_frontend.so.2022.3.2
lrwxrwxrwx 43 lvision 30 Jul 14:08 libopenvino_tensorflow_frontend.so.2232 -> libopenvino_tensorflow_frontend.so.2022.3.2
.rw-r--r-- 3.2k lvision 30 Jul 12:04 libov_protobuf_shutdown.a
.rw-r--r-- 1.2k lvision 30 Jul 12:02 libov_shape_inference.a
.rw-r--r-- 1.8M lvision 29 Jul 20:03 libprotobuf-lite.a
.rw-r--r-- 7.8M lvision 29 Jul 20:04 libprotobuf.a
.rw-r--r-- 8.3M lvision 29 Jul 21:07 libprotoc.a
.rw-r--r-- 419k lvision 29 Jul 20:01 libpugixml.a
.rwxr-xr-x 133k lvision 30 Jul 14:08 libtemplate_extension.so
.rw-r--r-- 165k lvision 30 Jul 12:03 libutil.a
.rw-r--r-- 9.9M lvision 30 Jul 12:52 libvpu_common_lib.a
.rw-r--r-- 30M lvision 30 Jul 13:07 libvpu_graph_transformer.a
.rw-r--r-- 296k lvision 30 Jul 12:04 libXLink.a
.rwxr-xr-x 265k lvision 30 Jul 14:08 model_creation_sample
.rwxr-xr-x 68k lvision 30 Jul 14:08 ov_integration_snippet
.rwxr-xr-x 67k lvision 30 Jul 14:08 ov_integration_snippet_c
.rw-r--r-- 2.1M lvision 29 Jul 20:02 pcie-ma2x8x.mvcmd
.rw-r--r-- 486 lvision 30 Jul 14:08 plugins.xml
lrwxrwxrwx 15 lvision 30 Jul 13:56 protoc -> /usr/bin/protoc
.rwxr-xr-x 3.7M lvision 29 Jul 21:07 protoc-3.20.3.0.bak
drwxr-xr-x - lvision 30 Jul 14:08 python_api
.rwxr-xr-x 462k lvision 30 Jul 14:08 speech_sample
.rwxr-xr-x 68k lvision 30 Jul 14:08 sync_benchmark
.rwxr-xr-x 68k lvision 30 Jul 14:08 throughput_benchmark
.rw-r--r-- 2.4M lvision 29 Jul 20:02 usb-ma2x8x.mvcmd

Under which, there is a file plugins.xml. VERY IMPORTANT: put plugins.xml under /usr/local/lib.

3.3 Demonstration

3.3.1 hello_query_device.py

Go to /opt/intel/OpenVINO/openvino/samples/python/hello_query_device, and run Python script hello_query_device.py:

OpenVINO Sample hello_query_device.py

3.3.2 AI Samples

Please refer to my NEXT blog Raspberry Pi 5 + Intel NCS2 + OpenVINO In 2024: