Intel NCS2 vs. Intel NCS1

Posted on Edited on In ,

As of year 2024, it looks NCSDK is NOT maintained. To clarify the relationships among OpenVINO, NCSDK master and NCSDK ncsdk2 is a kind of complicated.

1. OpenVINO Runtime NO Longer Supports Movidius VPU

1.1 OpenVINO Toolkit Release Notes 2023.0

OpenVINO NOT Support NCS2

And it looks Long Term Support (LTS) version of OpenVINO ending in December 2024 now still supports for the devices listed above.

Therefore, in order to get my old devices: 1 NCS2 and 1 NCS1 working in the year of 2024, I’ll have to resort to Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS).

1.2 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3

OpenVINO LTS Supports Movidius

1.3 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3

Ever since LTS 2023.3, OpenVINO started emphasize GenAI. On its release notes, nowhere is talking about NCS2 or NCS1 or anything related to Movidius VPU. According to OpenVINO Toolkit Release Notes 2023.0 in the above, I think ever since 2023, OpenVINO turns its direction to GenAI, and NEVER support for Movidius VPU any more. Therefore, most probably, including this Long-Term Support version, Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3 does NOT have any support to Movidius VPU devices.

OpenVINO GenAI Since 2023

1.4 OpenVINO 2024.2 @ Intel® Distribution of OpenVINO™ Toolkit Release Notes 2024.2

Not to mention the MOST UP-TO-DATE version 2024.2, which seems to be NOT a LTS. Of course, it does NOT support Movidius VPU devices.

1.5 Conclusion

Go for:

2. NCSDK Does NOT Support AArch64

2.1 get_mvcmd.sh is the key.

2.2 Firmware Is Actually Downloaded But Without AArch64 Support

1
2
3
4
5
6
7
8
➜  ncsdk ls NCSDK-2.10.01.01
ncsdk-armv7l  ncsdk-x86_64  version.txt
➜  ncsdk ll NCSDK-2.10.01.01/ncsdk-x86_64/fw                   
Permissions Size User Date Modified Name
.rwxrwxrwx  1.1M root 26 Jan  2019  MvNCAPI-ma2450.mvcmd
➜  ncsdk ll NCSDK-2.10.01.01/ncsdk-armv7l/fw 
Permissions Size User Date Modified Name
.rwxrwxrwx  1.1M root 26 Jan  2019  MvNCAPI-ma2450.mvcmd
1
2
3
4
5
6
7
8
➜  ncsdk ls NCSDK-1.12.01.01
install-ncsdk.sh  ncsdk-armv7l  ncsdk-x86_64  ncsdk.conf  requirements.txt  requirements_apt.txt  tests  uninstall-ncsdk.sh  version.txt
➜  ncsdk ll NCSDK-1.12.01.01/ncsdk-x86_64/fw 
Permissions Size User Date Modified Name
.rwxrwxrwx  866k root  5 Jan  2018  MvNCAPI.mvcmd
➜  ncsdk ll NCSDK-1.12.01.01/ncsdk-armv7l/fw 
Permissions Size User Date Modified Name
.rwxrwxrwx  866k root  5 Jan  2018  MvNCAPI.mvcmd

2.3 ncsdk-aarch64 for NCS1

Fortunately, I found ncsdk-aarch64. However, it only provides ncsdk-aarch64’s firmware from NCSDK-1.12.00.01, which seems to be also okay for NCSDK-1.12.01.01.

Let’s try it out.

2.3.1 Find the Right Version

You will see a folder named NCSDK-1.12.01.01 (or NCSDK-1.12.00.01), under which there is the key folder ncsdk-aarch64, which actually contains the Aarch64 firmware for NCS1.

1
2
3
4
5
➜  ncsdk-aarch64 git:(master) ✗ ls
api  docs  examples  install-opencv.sh  install.sh  LICENSE  Makefile  NCSDK-1.12.01.01  ncsdk.conf  README.md  uninstall-opencv.sh  uninstall.sh
➜  ncsdk-aarch64 git:(master) ✗ cd NCSDK-1.12.01.01 
➜  NCSDK-1.12.01.01 git:(master) ✗ ls
install-ncsdk.sh  ncsdk-aarch64  ncsdk-armv7l  ncsdk-x86_64  ncsdk.conf  requirements.txt  requirements_apt.txt  tests  uninstall-ncsdk.sh  version.txt

2.3.2 Installation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
➜  ncsdk-aarch64 git:(master) ✗ cd api/src
➜  src git:(master) ✗ sudo make basicinstall 
mkdir -p /usr/local/include/
mkdir -p /usr/local/lib/
cp obj-aarch64/libmvnc.so.0 /usr/local/lib/
ln -fs libmvnc.so.0 /usr/local/lib/libmvnc.so
cp ../include/*.h /usr/local/include/
mkdir -p /usr/local/lib/mvnc
cp mvnc/MvNCAPI.mvcmd /usr/local/lib/mvnc/
mkdir -p /etc/udev/rules.d/
cp 97-usbboot.rules /etc/udev/rules.d/
➜  src git:(master) ✗ sudo make pythoninstall
mkdir -p /usr/local/lib/python3.11/dist-packages
cp -r ../python/mvnc /usr/local/lib/python3.11/dist-packages/
➜  src git:(master) ✗ sudo make postinstall
udevadm control --reload-rules
udevadm trigger
ldconfig

2.3.3 Demonstrate Examples on My Raspberry Pi 5

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
➜  examples git:(master) ✗ make
make -C apps/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps'
make -C hello_ncs_cpp/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_cpp'

making hello_ncs_cpp
g++ cpp/hello_ncs.cpp -o cpp/hello_ncs_cpp -lmvnc
Created cpp/hello_ncs_cpp executable
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_cpp'
make -C hello_ncs_py/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_py'
nothing to make, use 'make run' to run.
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_py'
make -C multistick_cpp/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/multistick_cpp'

making googlenet
(cd ../../caffe/GoogLeNet; make compile; cd ../../apps/multistick_cpp; cp ../../caffe/GoogLeNet/graph ./googlenet.graph;)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[4]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[4]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'

making prototxt
Prototxt file already exists

making caffemodel
caffemodel file already exists

making compile
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making squeezenet
(cd ../../caffe/SqueezeNet; make compile; cd ../../apps/multistick_cpp; cp ../../caffe/SqueezeNet/graph ./squeezenet.graph;)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[4]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[4]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
LICENSE file already exists

making prototxt
Prototxt file already exists

making caffemodel
caffemodel file already exists

making compile
mvNCCompile -w squeezenet_v1.0.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making multistick_cpp
cp googlenet.graph cpp/googlenet.graph;
cp squeezenet.graph cpp/squeezenet.graph;
g++ cpp/multistick.cpp cpp/fp16.c -o cpp/multistick_cpp -lmvnc
cpp/multistick.cpp: In function ‘bool DoInferenceOnImageFile(void*, const char*, int, float*)’:
cpp/multistick.cpp:307:1: warning: control reaches end of non-void function [-Wreturn-type]
  307 | }
      | ^
Created cpp/multistick_cpp executable
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/multistick_cpp'
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps'
make -C caffe/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe'
make -C AlexNet/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/AlexNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
chmod +x run.py

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
Time to Execute :  107.91  ms
USB: Myriad Execution Finished
Time to Execute :  103.26  ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
                                                                                                                                                                                                                      Bandwidth   time
#    Name                                                                                                                                                                                                       MFLOPs  (MB/s)    (ms)
======================================================================================================================================================================================================================================
0    data                                                                                                                                                                                                          0.0126457.6   0.002
1    conv1                                                                                                                                                                                                       210.8  7434.1   4.811
2    norm1                                                                                                                                                                                                         0.0   721.2   0.768
3    pool1                                                                                                                                                                                                         0.6  1253.9   0.442
4    pool1_s0                                                                                                                                                                                                      0.0  1372.8   0.049
5    conv2_p0                                                                                                                                                                                                    223.9   596.2   6.099
6    pool1_s1                                                                                                                                                                                                      0.0  1455.7   0.046
7    conv2_p1                                                                                                                                                                                                    223.9   585.2   6.213
8    norm2                                                                                                                                                                                                         0.0   697.8   0.510
9    pool2                                                                                                                                                                                                         0.4  1308.0   0.272
10   conv3                                                                                                                                                                                                       299.0   302.0   8.048
11   conv3_s0                                                                                                                                                                                                      0.0  1582.4   0.039
12   conv4_p0                                                                                                                                                                                                    112.1   543.7   3.219
13   conv3_s1                                                                                                                                                                                                      0.0  1492.2   0.041
14   conv4_p1                                                                                                                                                                                                    112.1   560.1   3.125
15   conv4_p0_s0                                                                                                                                                                                                   0.0  1314.0   0.047
16   conv5_p0                                                                                                                                                                                                     74.8   578.5   2.659
17   conv4_p0_s1                                                                                                                                                                                                   0.0  1317.5   0.047
18   conv5_p1                                                                                                                                                                                                     74.8   580.0   2.652
19   pool5                                                                                                                                                                                                         0.1   918.4   0.090
20   fc6                                                                                                                                                                                                          75.5  2144.1  33.589
21   fc7                                                                                                                                                                                                          33.6  2132.6  15.008
22   fc8                                                                                                                                                                                                           8.2  2651.8   2.949
23   prob                                                                                                                                                                                                          0.0     7.6   0.253
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                                                                          Total inference time                   90.98
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w bvlc_alexnet.caffemodel -i ../../data/images/cat.jpg -s 12 -id 281 deploy.prototxt -M 110 -S 255 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result:  (1000,)
1) 281 0.512
Expected:  (1000,)
1) 281 0.522
------------------------------------------------------------
 Obtained values 
------------------------------------------------------------
 Obtained Min Pixel Accuracy: 2.385406941175461% (max allowed=2%), Fail
 Obtained Average Pixel Accuracy: 0.0052463514293776825% (max allowed=1%), Pass
 Obtained Percentage of wrong values: 0.1% (max allowed=0%), Fail
 Obtained Pixel-wise L2 error: 0.09619102055778048% (max allowed=1%), Pass
 Obtained Global Sum Difference: 0.027384519577026367
------------------------------------------------------------

making compile
mvNCCompile -w bvlc_alexnet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/AlexNet'
make -C GoogLeNet/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
Time to Execute :  130.42  ms
USB: Myriad Execution Finished
Time to Execute :  108.35  ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
                                                                                                                                                                                                                      Bandwidth   time
#    Name                                                                                                                                                                                                       MFLOPs  (MB/s)    (ms)
======================================================================================================================================================================================================================================
0    data                                                                                                                                                                                                          0.0 46160.0   0.006
1    conv1/7x7_s2                                                                                                                                                                                                236.0  2466.8   5.713
2    pool1/3x3_s2                                                                                                                                                                                                  1.8  1340.9   1.142
3    pool1/norm1                                                                                                                                                                                                   0.0   701.9   0.546
4    conv2/3x3_reduce                                                                                                                                                                                             25.7   478.8   0.816
5    conv2/3x3                                                                                                                                                                                                   693.6   307.6  11.891
6    conv2/norm2                                                                                                                                                                                                   0.0   787.5   1.458
7    pool2/3x3_s2                                                                                                                                                                                                  1.4  1390.2   0.826
8    inception_3a/1x1                                                                                                                                                                                             19.3   535.1   0.581
9    inception_3a/3x3_reduce                                                                                                                                                                                      28.9   459.3   0.702
10   inception_3a/3x3                                                                                                                                                                                            173.4   314.3   4.788
11   inception_3a/5x5_reduce                                                                                                                                                                                       4.8  1025.6   0.286
12   inception_3a/5x5                                                                                                                                                                                             20.1   678.0   0.921
13   inception_3a/pool                                                                                                                                                                                             1.4   578.0   0.497
14   inception_3a/pool_proj                                                                                                                                                                                        9.6   640.6   0.467
15   inception_3b/1x1                                                                                                                                                                                             51.4   444.3   1.003
16   inception_3b/3x3_reduce                                                                                                                                                                                      51.4   446.2   0.998
17   inception_3b/3x3                                                                                                                                                                                            346.8   257.6   8.339
18   inception_3b/5x5_reduce                                                                                                                                                                                      12.8   877.2   0.454
19   inception_3b/5x5                                                                                                                                                                                            120.4   522.8   2.577
20   inception_3b/pool                                                                                                                                                                                             1.8   660.6   0.580
21   inception_3b/pool_proj                                                                                                                                                                                       25.7   618.7   0.669
22   pool3/3x3_s2                                                                                                                                                                                                  0.8  1215.8   0.590
23   inception_4a/1x1                                                                                                                                                                                             36.1   358.6   0.991
24   inception_4a/3x3_reduce                                                                                                                                                                                      18.1   498.7   0.536
25   inception_4a/3x3                                                                                                                                                                                             70.4   296.2   2.259
26   inception_4a/5x5_reduce                                                                                                                                                                                       3.0   756.2   0.257
27   inception_4a/5x5                                                                                                                                                                                              7.5   412.6   0.457
28   inception_4a/pool                                                                                                                                                                                             0.8   505.0   0.355
29   inception_4a/pool_proj                                                                                                                                                                                       12.0   547.8   0.435
30   inception_4b/1x1                                                                                                                                                                                             32.1   330.5   1.053
31   inception_4b/3x3_reduce                                                                                                                                                                                      22.5   375.9   0.800
32   inception_4b/3x3                                                                                                                                                                                             88.5   285.9   2.838
33   inception_4b/5x5_reduce                                                                                                                                                                                       4.8   573.7   0.375
34   inception_4b/5x5                                                                                                                                                                                             15.1   309.1   0.973
35   inception_4b/pool                                                                                                                                                                                             0.9   532.7   0.359
36   inception_4b/pool_proj                                                                                                                                                                                       12.8   537.0   0.473
37   inception_4c/1x1                                                                                                                                                                                             25.7   402.8   0.786
38   inception_4c/3x3_reduce                                                                                                                                                                                      25.7   389.8   0.812
39   inception_4c/3x3                                                                                                                                                                                            115.6   260.4   3.831
40   inception_4c/5x5_reduce                                                                                                                                                                                       4.8   584.9   0.367
41   inception_4c/5x5                                                                                                                                                                                             15.1   308.1   0.976
42   inception_4c/pool                                                                                                                                                                                             0.9   627.2   0.305
43   inception_4c/pool_proj                                                                                                                                                                                       12.8   551.2   0.461
44   inception_4d/1x1                                                                                                                                                                                             22.5   413.7   0.728
45   inception_4d/3x3_reduce                                                                                                                                                                                      28.9   472.7   0.702
46   inception_4d/3x3                                                                                                                                                                                            146.3   399.4   3.008
47   inception_4d/5x5_reduce                                                                                                                                                                                       6.4   637.4   0.349
48   inception_4d/5x5                                                                                                                                                                                             20.1   388.8   1.028
49   inception_4d/pool                                                                                                                                                                                             0.9   610.5   0.314
50   inception_4d/pool_proj                                                                                                                                                                                       12.8   566.9   0.448
51   inception_4e/1x1                                                                                                                                                                                             53.0   297.6   1.531
52   inception_4e/3x3_reduce                                                                                                                                                                                      33.1   258.2   1.389
53   inception_4e/3x3                                                                                                                                                                                            180.6   280.8   5.067
54   inception_4e/5x5_reduce                                                                                                                                                                                       6.6   493.5   0.465
55   inception_4e/5x5                                                                                                                                                                                             40.1   356.2   1.405
56   inception_4e/pool                                                                                                                                                                                             0.9   600.5   0.329
57   inception_4e/pool_proj                                                                                                                                                                                       26.5   430.8   0.758
58   pool4/3x3_s2                                                                                                                                                                                                  0.4  1221.7   0.255
59   inception_5a/1x1                                                                                                                                                                                             20.9   614.2   0.789
60   inception_5a/3x3_reduce                                                                                                                                                                                      13.0   553.5   0.599
61   inception_5a/3x3                                                                                                                                                                                             45.2   550.4   1.851
62   inception_5a/5x5_reduce                                                                                                                                                                                       2.6   336.2   0.382
63   inception_5a/5x5                                                                                                                                                                                             10.0   427.3   0.646
64   inception_5a/pool                                                                                                                                                                                             0.4   405.1   0.192
65   inception_5a/pool_proj                                                                                                                                                                                       10.4   484.7   0.580
66   inception_5b/1x1                                                                                                                                                                                             31.3   611.9   1.124
67   inception_5b/3x3_reduce                                                                                                                                                                                      15.7   608.1   0.629
68   inception_5b/3x3                                                                                                                                                                                             65.0   569.9   2.516
69   inception_5b/5x5_reduce                                                                                                                                                                                       3.9   362.9   0.424
70   inception_5b/5x5                                                                                                                                                                                             15.1   438.7   0.937
71   inception_5b/pool                                                                                                                                                                                             0.4   483.8   0.161
72   inception_5b/pool_proj                                                                                                                                                                                       10.4   502.0   0.560
73   pool5/7x7_s1                                                                                                                                                                                                  0.1   460.8   0.208
74   loss3/classifier                                                                                                                                                                                              2.0  2497.0   0.783
75   prob                                                                                                                                                                                                          0.0     9.5   0.200
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                                                                          Total inference time                   95.91
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w bvlc_googlenet.caffemodel -i ../../data/images/nps_electric_guitar.png -s 12 -id 546  deploy.prototxt -S 255 -M 110 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result:  (1000,)
1) 546 0.995
Expected:  (1000,)
1) 546 0.9946
------------------------------------------------------------
 Obtained values 
------------------------------------------------------------
 Obtained Min Pixel Accuracy: 0.049091799883171916% (max allowed=2%), Pass
 Obtained Average Pixel Accuracy: 7.989402774910559e-05% (max allowed=1%), Pass
 Obtained Percentage of wrong values: 0.0% (max allowed=0%), Pass
 Obtained Pixel-wise L2 error: 0.0017114860971367098% (max allowed=1%), Pass
 Obtained Global Sum Difference: 0.0007946491241455078
------------------------------------------------------------

making compile
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'
make -C SqueezeNet/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
LICENSE file already exists

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
Time to Execute :  78.37  ms
USB: Myriad Execution Finished
Time to Execute :  60.05  ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
                                                                                                                                                                                                                      Bandwidth   time
#    Name                                                                                                                                                                                                       MFLOPs  (MB/s)    (ms)
======================================================================================================================================================================================================================================
0    data                                                                                                                                                                                                          0.0 96516.2   0.003
1    conv1                                                                                                                                                                                                       347.7  1634.5   8.861
2    pool1                                                                                                                                                                                                         2.6  1445.0   1.561
3    fire2/squeeze1x1                                                                                                                                                                                              9.3  1222.9   0.455
4    fire2/expand1x1                                                                                                                                                                                               6.2   152.1   0.620
5    fire2/expand3x3                                                                                                                                                                                              55.8   475.0   1.789
6    fire3/squeeze1x1                                                                                                                                                                                             12.4  1466.3   0.506
7    fire3/expand1x1                                                                                                                                                                                               6.2   158.4   0.596
8    fire3/expand3x3                                                                                                                                                                                              55.8   475.9   1.785
9    fire4/squeeze1x1                                                                                                                                                                                             24.8   976.6   0.764
10   fire4/expand1x1                                                                                                                                                                                              24.8   174.3   1.106
11   fire4/expand3x3                                                                                                                                                                                             223.0   391.4   4.430
12   pool4                                                                                                                                                                                                         1.7  1268.5   1.164
13   fire5/squeeze1x1                                                                                                                                                                                             11.9   788.7   0.471
14   fire5/expand1x1                                                                                                                                                                                               6.0   151.4   0.347
15   fire5/expand3x3                                                                                                                                                                                              53.7   376.4   1.257
16   fire6/squeeze1x1                                                                                                                                                                                             17.9   699.1   0.543
17   fire6/expand1x1                                                                                                                                                                                              13.4   159.6   0.531
18   fire6/expand3x3                                                                                                                                                                                             120.9   256.4   2.973
19   fire7/squeeze1x1                                                                                                                                                                                             26.9   827.4   0.688
20   fire7/expand1x1                                                                                                                                                                                              13.4   159.9   0.530
21   fire7/expand3x3                                                                                                                                                                                             120.9   259.7   2.935
22   fire8/squeeze1x1                                                                                                                                                                                             35.8   735.0   0.790
23   fire8/expand1x1                                                                                                                                                                                              23.9   155.7   0.775
24   fire8/expand3x3                                                                                                                                                                                             215.0   201.3   5.396
25   pool8                                                                                                                                                                                                         0.8  1265.2   0.563
26   fire9/squeeze1x1                                                                                                                                                                                             11.1   587.7   0.387
27   fire9/expand1x1                                                                                                                                                                                               5.5   153.4   0.341
28   fire9/expand3x3                                                                                                                                                                                              49.8   288.2   1.636
29   conv10                                                                                                                                                                                                      173.1   331.1   3.448
30   pool10                                                                                                                                                                                                        0.3   671.0   0.480
31   prob                                                                                                                                                                                                          0.0     9.5   0.202
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                                                                          Total inference time                   47.94
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w squeezenet_v1.0.caffemodel -i ../../data/images/cat.jpg -s 12 -id 281  deploy.prototxt -S 255 -M 120 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result:  (1000, 1, 1)
1) 281 0.6177
Expected:  (1000, 1, 1)
1) 281 0.616
------------------------------------------------------------
 Obtained values 
------------------------------------------------------------
 Obtained Min Pixel Accuracy: 0.6933439057320356% (max allowed=2%), Pass
 Obtained Average Pixel Accuracy: 0.0016120149666676298% (max allowed=1%), Pass
 Obtained Percentage of wrong values: 0.0% (max allowed=0%), Pass
 Obtained Pixel-wise L2 error: 0.029018872980330863% (max allowed=1%), Pass
 Obtained Global Sum Difference: 0.009933412075042725
------------------------------------------------------------

making compile
mvNCCompile -w squeezenet_v1.0.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe'
make -C data/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data'
Possible Make targets
  make help - shows this message
  make clean - Removes all temp files from all directories
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data'
make -C tensorflow/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/tensorflow'
make -C inception_v1/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1'
test -f output/inception-v1.meta || ((wget http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz && tar zxf inception_v1_2016_08_28.tar.gz && rm inception_v1_2016_08_28.tar.gz) && ./inception-v1.py)
--2024-07-29 13:06:28--  http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz
Resolving download.tensorflow.org (download.tensorflow.org)... 2607:f8b0:400a:803::201b, 2607:f8b0:400a:805::201b, 2607:f8b0:400a:807::201b, ...
Connecting to download.tensorflow.org (download.tensorflow.org)|2607:f8b0:400a:803::201b|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24642554 (24M) [application/x-tar]
Saving to: ‘inception_v1_2016_08_28.tar.gz’

inception_v1_2016_08_28.tar.gz                            100%[====================================================================================================================================>]  23.50M  98.9MB/s    in 0.2s    

2024-07-29 13:06:28 (98.9 MB/s) - ‘inception_v1_2016_08_28.tar.gz’ saved [24642554/24642554]

Traceback (most recent call last):
  File "/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1/./inception-v1.py", line 6, in <module>
    from tensorflow.contrib.slim.nets import inception
ModuleNotFoundError: No module named 'tensorflow.contrib'
make[2]: *** [Makefile:47: weights] Error 1
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1'
make[1]: *** [Makefile:12: inception_v1/.] Error 2
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/tensorflow'
make: *** [Makefile:12: tensorflow/.] Error 2

I'm gonna giving up the issue of Tensorflow version incompatibility, for Tensorflow 1.x has NOT been maintained for years. I mean example inception_v1 and inception_v3 are compatible with Tensorflow 1.x, rather than 2.x.

1
2
➜  examples git:(master) ✗ ls tensorflow 
inception_v1  inception_v3  Makefile  readme.md

2.4 ncsdk2-aarch64 for NCS2

2.4.1 Download the Right NCSDK-2.10.01.01.tar.gz

  • git clone https://github.com/LE0xUL/ncsdk2_aarch64.git won’t provide the correct NCSDK-2.10.01.01.tar.gz.
1
2
3
4
5
6
7
8
➜  ncsdk2_aarch64 git:(aarch64) ✗ ll
Permissions Size User    Date Modified Name
drwxr-xr-x     - lvision 29 Jul 15:34  .git
.rw-r--r--   564 lvision 29 Jul 12:16  .gitattributes
.rw-r--r--   212 lvision 29 Jul 12:16  .gitignore
......
.rw-r--r--   132 lvision 29 Jul 12:16  NCSDK-2.10.01.01.tar.gz
......

As you can see, the git cloned NCSDK-2.10.01.01.tar.gz is of size 132 bytes ONLY.

1
2
3
➜  ncsdk2_aarch64 git:(aarch64) ✗ ll NCSDK-2.10.01.01.tar.gz 
Permissions Size User    Date Modified Name
.rw-r--r--  2.4M lvision 29 Jul 16:12  NCSDK-2.10.01.01.tar.gz

After extraction, you’ll also see the most important folder named ncsdk-aarch64, which actually contains the Aarch64 firmware for NCS2.

1
2
3
4
5
6
➜  ncsdk2_aarch64 git:(aarch64) ✗ ls
api           docs      extras             install-utilities.sh  LICENSE   NCSDK-2.10.01.01         ncsdk.conf  requirements.txt      requirements_apt_raspbian.txt                  uninstall-opencv.sh  version.txt
ATTRIBUTIONS  examples  install-opencv.sh  install.sh            Makefile  NCSDK-2.10.01.01.tar.gz  README.md   requirements_apt.txt  tensorflow-1.11.0-cp35-none-linux_aarch64.whl  uninstall.sh         
➜  ncsdk2_aarch64 git:(aarch64) ✗ cd NCSDK-2.10.01.01 
➜  NCSDK-2.10.01.01 git:(aarch64) ✗ ls
ncsdk-aarch64  ncsdk-armv7l  ncsdk-x86_64  version.txt

2.4.2 Installation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
➜  ncsdk2_aarch64 git:(aarch64) ✗ cd api/src 
➜  src git:(aarch64) ✗ sudo make basicinstall
NCSDK FW successfully installed
mkdir -p /usr/local/include/
mkdir -p /usr/local/include/mvnc2
mkdir -p /usr/local/lib/
cp obj-aarch64/libmvnc.so.0 /usr/local/lib/
ln -fs libmvnc.so.0 /usr/local/lib/libmvnc.so
cp ../include/mvnc.h /usr/local/include/mvnc2
ln -fs /usr/local/include/mvnc2/mvnc.h /usr/local/include/mvnc.h 
mkdir -p /usr/local/lib/mvnc
cp mvnc/MvNCAPI-*.mvcmd /usr/local/lib/mvnc/
mkdir -p /etc/udev/rules.d/
cp 97-ncs2.rules /etc/udev/rules.d/
➜  src git:(aarch64) ✗ sudo make pythoninstall
mkdir -p /usr/local/lib/python3.11/dist-packages
cp -r ../python/mvnc /usr/local/lib/python3.11/dist-packages/
➜  src git:(aarch64) ✗ sudo make postinstall
udevadm control --reload-rules
udevadm trigger
ldconfig

2.4.3 Demonstrate Examples on My Raspberry Pi 5

Let’s just try hello_ncs.py.

1
2
3
4
5
6
7
8
9
10
➜  hello_ncs_py git:(aarch64) ✗ python hello_ncs.py 
D: [         0] ncDeviceCreate:307	ncDeviceCreate index 0

D: [         0] ncDeviceCreate:307	ncDeviceCreate index 1

D: [         0] ncDeviceOpen:523	File path /usr/local/lib/mvnc/MvNCAPI-ma2480.mvcmd

W: [         0] ncDeviceOpen:527	ncDeviceOpen() XLinkBootRemote returned error 3

Error - Could not open NCS device.

It looks I dealt with this issue in my old blog OpenVINO on Raspberry Pi 4 with Movidius Neural Compute Stick II, where I’d already provided the solution:

OpenVINO should be utilized instead of NCSDK ncsdk2.

3. Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3 for NCS2

3.1 Fix an Error

File /opt/intel/OpenVINO/openvino/thirdparty/ocv/opencv_hal_neon.hpp function v_deinterleave_expand is clearly buggy, for -1 should NEVER appear in an uchar array. Anyway, just change -1 to 255 as a temporary solution, namely:

Change

1
2
3
4
5
6
constexpr int nlanes = static_cast<int>(v_uint8x16::nlanes);
     uchar mask_e[nlanes] = { 0, -1, 2, -1, 4, -1, 6, -1,                                                                                                                                                                      
                             8, -1, 10, -1, 12, -1, 14, -1 };
 
     uchar mask_o[nlanes] = { 1, -1, 3, -1, 5, -1, 7, -1,
                             9, -1, 11, -1, 13, -1, 15, -1 };

to:

1
2
3
4
5
6
constexpr int nlanes = static_cast<int>(v_uint8x16::nlanes);
     uchar mask_e[nlanes] = { 0, 255, 2, 255, 4, 255, 6, 255,                                                                                                                                                                      
                             8, 255, 10, 255, 12, 255, 14, 255 };
 
     uchar mask_o[nlanes] = { 1, 255, 3, 255, 5, 255, 7, 255,
                             9, 255, 11, 255, 13, 255, 15, 255 };

3.2 Two More Things After Installation

3.2.1 OpenVINO Python Wrapper Installation

Under folder /opt/intel/OpenVINO/openvino/build/wheels, there are the following 2 files：

1
2
3
4
➜  wheels git:(2022.3.2) ✗ ll
Permissions Size User    Date Modified Name
.rw-r--r--   13M lvision 30 Jul 14:09  openvino-2022.3.2-9279-cp311-cp311-manylinux_2_36_aarch64.whl
.rw-r--r--   24k lvision 30 Jul 13:08  openvino_dev-2022.3.2-9279-py3-none-any.whl

Use key parameter --break-system-packages

1
sudo pip install --break-system-packages openvino-2022.3.2-9279-cp311-cp311-manylinux_2_36_aarch64.whl

and

1
sudo pip install --break-system-packages openvino_dev-2022.3.2-9279-py3-none-any.whl

3.2.2 Copy plugins.xml

A kind of weird that:

  • Under build/lib, there is ONLY 1 file libade.a:
1
2
3
4
➜  lib git:(2022.3.2) ✗ pwd
/opt/intel/OpenVINO/openvino/build/lib
➜  lib git:(2022.3.2) ✗ ls
libade.a
  • Everything is built under bin/aarch64/Release:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
➜  Release git:(2022.3.2) ✗ pwd                      
/opt/intel/OpenVINO/openvino/bin/aarch64/Release
➜  Release git:(2022.3.2) ✗ ll
Permissions Size User    Date Modified Name
.rwxr-xr-x  723k lvision 30 Jul 14:08  benchmark_app
.rwxr-xr-x  461k lvision 30 Jul 14:08  benchmark_app_legacy
.rwxr-xr-x  264k lvision 30 Jul 14:08  classification_sample_async
.rwxr-xr-x  264k lvision 30 Jul 14:08  compile_tool
.rwxr-xr-x  199k lvision 30 Jul 14:08  hello_classification
.rwxr-xr-x   67k lvision 30 Jul 14:08  hello_classification_c
.rwxr-xr-x  199k lvision 30 Jul 14:08  hello_nv12_input_classification
.rwxr-xr-x   67k lvision 30 Jul 14:08  hello_nv12_input_classification_c
.rwxr-xr-x   68k lvision 30 Jul 14:08  hello_query_device
.rwxr-xr-x  199k lvision 30 Jul 14:08  hello_reshape_ssd
.rw-r--r--  367k lvision 30 Jul 12:04  libcnpy.a
.rw-r--r--  7.5M lvision 30 Jul 14:08  libfluid.a
.rwxr-xr-x   67k lvision 30 Jul 14:08  libformat_reader.so
.rw-r--r--  3.4M lvision 30 Jul 12:08  libie_docs_snippets.a
.rw-r--r--  166k lvision 30 Jul 12:34  libie_samples_utils.a
.rw-r--r--   16M lvision 30 Jul 12:31  libinference_engine_legacy.a
.rw-r--r--  2.9M lvision 30 Jul 12:36  libinference_engine_snippets.a
.rw-r--r--   16M lvision 30 Jul 12:56  libinterpreter_backend.a
.rw-r--r--  3.0k lvision 30 Jul 12:04  libitt.a
.rw-r--r--  296k lvision 30 Jul 12:05  libmvnc.a
.rw-r--r--  1.6M lvision 30 Jul 12:04  libngraph_builders.a
.rw-r--r--  1.6M lvision 30 Jul 14:07  libngraph_reference.a
.rw-r--r--  2.3M lvision 30 Jul 12:33  liboffline_transformations.a
.rw-r--r--   11M lvision 30 Jul 12:17  libonnx.a
.rw-r--r--   54k lvision 30 Jul 12:31  libonnx_common.a
.rw-r--r--  676k lvision 30 Jul 12:06  libonnx_proto.a
.rwxr-xr-x   67k lvision 30 Jul 12:04  libopencv_c_wrapper.so
lrwxrwxrwx    19 lvision 30 Jul 14:08  libopenvino.so -> libopenvino.so.2232
.rwxr-xr-x   17M lvision 30 Jul 14:08  libopenvino.so.2022.3.2
lrwxrwxrwx    23 lvision 30 Jul 14:08  libopenvino.so.2232 -> libopenvino.so.2022.3.2
.rwxr-xr-x  330k lvision 30 Jul 14:08  libopenvino_auto_batch_plugin.so
.rwxr-xr-x  592k lvision 30 Jul 14:08  libopenvino_auto_plugin.so
lrwxrwxrwx    21 lvision 30 Jul 14:08  libopenvino_c.so -> libopenvino_c.so.2232
.rwxr-xr-x  461k lvision 30 Jul 14:08  libopenvino_c.so.2022.3.2
lrwxrwxrwx    25 lvision 30 Jul 14:08  libopenvino_c.so.2232 -> libopenvino_c.so.2022.3.2
.rwxr-xr-x  1.2M lvision 30 Jul 14:08  libopenvino_gapi_preproc.so
.rwxr-xr-x  395k lvision 30 Jul 14:08  libopenvino_hetero_plugin.so
.rwxr-xr-x  9.3M lvision 30 Jul 14:08  libopenvino_intel_myriad_plugin.so
lrwxrwxrwx    31 lvision 30 Jul 14:08  libopenvino_ir_frontend.so -> libopenvino_ir_frontend.so.2232
.rwxr-xr-x  528k lvision 30 Jul 14:08  libopenvino_ir_frontend.so.2022.3.2
lrwxrwxrwx    35 lvision 30 Jul 14:08  libopenvino_ir_frontend.so.2232 -> libopenvino_ir_frontend.so.2022.3.2
lrwxrwxrwx    33 lvision 30 Jul 14:08  libopenvino_onnx_frontend.so -> libopenvino_onnx_frontend.so.2232
.rwxr-xr-x  4.7M lvision 30 Jul 14:08  libopenvino_onnx_frontend.so.2022.3.2
lrwxrwxrwx    37 lvision 30 Jul 14:08  libopenvino_onnx_frontend.so.2232 -> libopenvino_onnx_frontend.so.2022.3.2
lrwxrwxrwx    35 lvision 30 Jul 14:08  libopenvino_paddle_frontend.so -> libopenvino_paddle_frontend.so.2232
.rwxr-xr-x  1.9M lvision 30 Jul 14:08  libopenvino_paddle_frontend.so.2022.3.2
lrwxrwxrwx    39 lvision 30 Jul 14:08  libopenvino_paddle_frontend.so.2232 -> libopenvino_paddle_frontend.so.2022.3.2
.rwxr-xr-x   68k lvision 30 Jul 14:08  libopenvino_template_extension.so
.rwxr-xr-x  8.0M lvision 30 Jul 14:08  libopenvino_template_plugin.so
lrwxrwxrwx    39 lvision 30 Jul 14:08  libopenvino_tensorflow_frontend.so -> libopenvino_tensorflow_frontend.so.2232
.rwxr-xr-x  3.5M lvision 30 Jul 14:08  libopenvino_tensorflow_frontend.so.2022.3.2
lrwxrwxrwx    43 lvision 30 Jul 14:08  libopenvino_tensorflow_frontend.so.2232 -> libopenvino_tensorflow_frontend.so.2022.3.2
.rw-r--r--  3.2k lvision 30 Jul 12:04  libov_protobuf_shutdown.a
.rw-r--r--  1.2k lvision 30 Jul 12:02  libov_shape_inference.a
.rw-r--r--  1.8M lvision 29 Jul 20:03  libprotobuf-lite.a
.rw-r--r--  7.8M lvision 29 Jul 20:04  libprotobuf.a
.rw-r--r--  8.3M lvision 29 Jul 21:07  libprotoc.a
.rw-r--r--  419k lvision 29 Jul 20:01  libpugixml.a
.rwxr-xr-x  133k lvision 30 Jul 14:08  libtemplate_extension.so
.rw-r--r--  165k lvision 30 Jul 12:03  libutil.a
.rw-r--r--  9.9M lvision 30 Jul 12:52  libvpu_common_lib.a
.rw-r--r--   30M lvision 30 Jul 13:07  libvpu_graph_transformer.a
.rw-r--r--  296k lvision 30 Jul 12:04  libXLink.a
.rwxr-xr-x  265k lvision 30 Jul 14:08  model_creation_sample
.rwxr-xr-x   68k lvision 30 Jul 14:08  ov_integration_snippet
.rwxr-xr-x   67k lvision 30 Jul 14:08  ov_integration_snippet_c
.rw-r--r--  2.1M lvision 29 Jul 20:02  pcie-ma2x8x.mvcmd
.rw-r--r--   486 lvision 30 Jul 14:08  plugins.xml
lrwxrwxrwx    15 lvision 30 Jul 13:56  protoc -> /usr/bin/protoc
.rwxr-xr-x  3.7M lvision 29 Jul 21:07  protoc-3.20.3.0.bak
drwxr-xr-x     - lvision 30 Jul 14:08  python_api
.rwxr-xr-x  462k lvision 30 Jul 14:08  speech_sample
.rwxr-xr-x   68k lvision 30 Jul 14:08  sync_benchmark
.rwxr-xr-x   68k lvision 30 Jul 14:08  throughput_benchmark
.rw-r--r--  2.4M lvision 29 Jul 20:02  usb-ma2x8x.mvcmd

Under which, there is a file plugins.xml. VERY IMPORTANT: put plugins.xml under /usr/local/lib.

3.3 Demonstration

3.3.1 hello_query_device.py

Go to /opt/intel/OpenVINO/openvino/samples/python/hello_query_device, and run Python script hello_query_device.py:

OpenVINO Sample hello_query_device.py

3.3.2 AI Samples

Please refer to my NEXT blog Raspberry Pi 5 + Intel NCS2 + OpenVINO In 2024: