Intel NCS2 vs. Intel NCS1

Posted on 2024-07-27 Edited on 2024-07-31 In AI , EdgeComputing

As of year 2024, it looks NCSDK is NOT maintained. To clarify the relationships among OpenVINO, NCSDK master and NCSDK ncsdk2 is a kind of complicated.

1. OpenVINO Runtime NO Longer Supports Movidius VPU

1.1 OpenVINO Toolkit Release Notes 2023.0

OpenVINO Runtime no longer supports Intel NCS2 and Movidius Myriad

And it looks Long Term Support (LTS) version of OpenVINO ending in December 2024 now still supports for the devices listed above.

Therefore, in order to get my old devices: 1 NCS2 and 1 NCS1 working in the year of 2024, I’ll have to resort to Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS).

1.2 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3

OpenVINO Runtime no longer supports Intel NCS2 and Movidius Myriad

1.3 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3

Ever since LTS 2023.3, OpenVINO started emphasize GenAI. On its release notes, nowhere is talking about NCS2 or NCS1 or anything related to Movidius VPU. According to OpenVINO Toolkit Release Notes 2023.0 in the above, I think ever since 2023, OpenVINO turns its direction to GenAI, and NEVER support for Movidius VPU any more. Therefore, most probably, including this Long-Term Support version, Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3 does NOT have any support to Movidius VPU devices.

OpenVINO Runtime no longer supports Intel NCS2 and Movidius Myriad

1.4 OpenVINO 2024.2 @ Intel® Distribution of OpenVINO™ Toolkit Release Notes 2024.2

Not to mention the MOST UP-TO-DATE version 2024.2, which seems to be NOT a LTS. Of course, it does NOT support Movidius VPU devices.

1.5 Conclusion

Go for:

2. NCSDK Does NOT Support AArch64

2.1 `get_mvcmd.sh` is the key.

ncsdk_download_link for NCSDK ncsdk2: Please refer to NCSDK ncsdk2‘s install-utilities.sh, it is https://downloadmirror.intel.com/28532/eng/NCSDK-2.10.01.01.tar.gz.
ncsdk_link for NCSDK master: Please refer to NCSDK master‘s api/src/get_mvcmd.sh, it is https://downloadmirror.intel.com/28192/eng/NCSDK-1.12.01.01.tar.gz.

2.2 Firmware Is Actually Downloaded But Without AArch64 Support

➜  ncsdk ls NCSDK-2.10.01.01
ncsdk-armv7l  ncsdk-x86_64  version.txt
➜  ncsdk ll NCSDK-2.10.01.01/ncsdk-x86_64/fw                   
Permissions Size User Date Modified Name
.rwxrwxrwx  1.1M root 26 Jan  2019  MvNCAPI-ma2450.mvcmd
➜  ncsdk ll NCSDK-2.10.01.01/ncsdk-armv7l/fw 
Permissions Size User Date Modified Name
.rwxrwxrwx  1.1M root 26 Jan  2019  MvNCAPI-ma2450.mvcmd

➜  ncsdk ls NCSDK-1.12.01.01
install-ncsdk.sh  ncsdk-armv7l  ncsdk-x86_64  ncsdk.conf  requirements.txt  requirements_apt.txt  tests  uninstall-ncsdk.sh  version.txt
➜  ncsdk ll NCSDK-1.12.01.01/ncsdk-x86_64/fw 
Permissions Size User Date Modified Name
.rwxrwxrwx  866k root  5 Jan  2018  MvNCAPI.mvcmd
➜  ncsdk ll NCSDK-1.12.01.01/ncsdk-armv7l/fw 
Permissions Size User Date Modified Name
.rwxrwxrwx  866k root  5 Jan  2018  MvNCAPI.mvcmd

2.3 ncsdk-aarch64 for NCS1

Fortunately, I found ncsdk-aarch64. However, it only provides ncsdk-aarch64’s firmware from NCSDK-1.12.00.01, which seems to be also okay for NCSDK-1.12.01.01.

Let’s try it out.

2.3.1 Find the Right Version

Either checkout official ncsdk-aarch64 with LongerVision pull request for ncsdk-aarch64
Or directly clone LongerVision ncsdk-aarch64

You will see a folder named NCSDK-1.12.01.01 (or NCSDK-1.12.00.01), under which there is the key folder ncsdk-aarch64, which actually contains the Aarch64 firmware for NCS1.

➜  ncsdk-aarch64 git:(master) ✗ ls
api  docs  examples  install-opencv.sh  install.sh  LICENSE  Makefile  NCSDK-1.12.01.01  ncsdk.conf  README.md  uninstall-opencv.sh  uninstall.sh
➜  ncsdk-aarch64 git:(master) ✗ cd NCSDK-1.12.01.01 
➜  NCSDK-1.12.01.01 git:(master) ✗ ls
install-ncsdk.sh  ncsdk-aarch64  ncsdk-armv7l  ncsdk-x86_64  ncsdk.conf  requirements.txt  requirements_apt.txt  tests  uninstall-ncsdk.sh  version.txt

2.3.2 Installation

➜  ncsdk-aarch64 git:(master) ✗ cd api/src
➜  src git:(master) ✗ sudo make basicinstall 
mkdir -p /usr/local/include/
mkdir -p /usr/local/lib/
cp obj-aarch64/libmvnc.so.0 /usr/local/lib/
ln -fs libmvnc.so.0 /usr/local/lib/libmvnc.so
cp ../include/*.h /usr/local/include/
mkdir -p /usr/local/lib/mvnc
cp mvnc/MvNCAPI.mvcmd /usr/local/lib/mvnc/
mkdir -p /etc/udev/rules.d/
cp 97-usbboot.rules /etc/udev/rules.d/
➜  src git:(master) ✗ sudo make pythoninstall
mkdir -p /usr/local/lib/python3.11/dist-packages
cp -r ../python/mvnc /usr/local/lib/python3.11/dist-packages/
➜  src git:(master) ✗ sudo make postinstall
udevadm control --reload-rules
udevadm trigger
ldconfig

2.3.3 Demonstrate Examples on My Raspberry Pi 5

➜  examples git:(master) ✗ make
make -C apps/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps'
make -C hello_ncs_cpp/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_cpp'

making hello_ncs_cpp
g++ cpp/hello_ncs.cpp -o cpp/hello_ncs_cpp -lmvnc
Created cpp/hello_ncs_cpp executable
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_cpp'
make -C hello_ncs_py/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_py'
nothing to make, use 'make run' to run.
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_py'
make -C multistick_cpp/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/multistick_cpp'

making googlenet
(cd ../../caffe/GoogLeNet; make compile; cd ../../apps/multistick_cpp; cp ../../caffe/GoogLeNet/graph ./googlenet.graph;)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[4]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[4]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'

making prototxt
Prototxt file already exists

making caffemodel
caffemodel file already exists

making compile
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making squeezenet
(cd ../../caffe/SqueezeNet; make compile; cd ../../apps/multistick_cpp; cp ../../caffe/SqueezeNet/graph ./squeezenet.graph;)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[4]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[4]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
LICENSE file already exists

making prototxt
Prototxt file already exists

making caffemodel
caffemodel file already exists

making compile
mvNCCompile -w squeezenet_v1.0.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making multistick_cpp
cp googlenet.graph cpp/googlenet.graph;
cp squeezenet.graph cpp/squeezenet.graph;
g++ cpp/multistick.cpp cpp/fp16.c -o cpp/multistick_cpp -lmvnc
cpp/multistick.cpp: In function ‘bool DoInferenceOnImageFile(void*, const char*, int, float*)’:
cpp/multistick.cpp:307:1: warning: control reaches end of non-void function [-Wreturn-type]
  307 | }
      | ^
Created cpp/multistick_cpp executable
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/multistick_cpp'
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps'
make -C caffe/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe'
make -C AlexNet/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/AlexNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
chmod +x run.py

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
Time to Execute :  107.91  ms
USB: Myriad Execution Finished
Time to Execute :  103.26  ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
                                                                                                                                                                                                                      Bandwidth   time
#    Name                                                                                                                                                                                                       MFLOPs  (MB/s)    (ms)
======================================================================================================================================================================================================================================
0    data                                                                                                                                                                                                          0.0126457.6   0.002
1    conv1                                                                                                                                                                                                       210.8  7434.1   4.811
2    norm1                                                                                                                                                                                                         0.0   721.2   0.768
3    pool1                                                                                                                                                                                                         0.6  1253.9   0.442
4    pool1_s0                                                                                                                                                                                                      0.0  1372.8   0.049
5    conv2_p0                                                                                                                                                                                                    223.9   596.2   6.099
6    pool1_s1                                                                                                                                                                                                      0.0  1455.7   0.046
7    conv2_p1                                                                                                                                                                                                    223.9   585.2   6.213
8    norm2                                                                                                                                                                                                         0.0   697.8   0.510
9    pool2                                                                                                                                                                                                         0.4  1308.0   0.272
10   conv3                                                                                                                                                                                                       299.0   302.0   8.048
11   conv3_s0                                                                                                                                                                                                      0.0  1582.4   0.039
12   conv4_p0                                                                                                                                                                                                    112.1   543.7   3.219
13   conv3_s1                                                                                                                                                                                                      0.0  1492.2   0.041
14   conv4_p1                                                                                                                                                                                                    112.1   560.1   3.125
15   conv4_p0_s0                                                                                                                                                                                                   0.0  1314.0   0.047
16   conv5_p0                                                                                                                                                                                                     74.8   578.5   2.659
17   conv4_p0_s1                                                                                                                                                                                                   0.0  1317.5   0.047
18   conv5_p1                                                                                                                                                                                                     74.8   580.0   2.652
19   pool5                                                                                                                                                                                                         0.1   918.4   0.090
20   fc6                                                                                                                                                                                                          75.5  2144.1  33.589
21   fc7                                                                                                                                                                                                          33.6  2132.6  15.008
22   fc8                                                                                                                                                                                                           8.2  2651.8   2.949
23   prob                                                                                                                                                                                                          0.0     7.6   0.253
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                                                                          Total inference time                   90.98
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w bvlc_alexnet.caffemodel -i ../../data/images/cat.jpg -s 12 -id 281 deploy.prototxt -M 110 -S 255 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result:  (1000,)
1) 281 0.512
Expected:  (1000,)
1) 281 0.522
------------------------------------------------------------
 Obtained values 
------------------------------------------------------------
 Obtained Min Pixel Accuracy: 2.385406941175461% (max allowed=2%), Fail
 Obtained Average Pixel Accuracy: 0.0052463514293776825% (max allowed=1%), Pass
 Obtained Percentage of wrong values: 0.1% (max allowed=0%), Fail
 Obtained Pixel-wise L2 error: 0.09619102055778048% (max allowed=1%), Pass
 Obtained Global Sum Difference: 0.027384519577026367
------------------------------------------------------------

making compile
mvNCCompile -w bvlc_alexnet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/AlexNet'
make -C GoogLeNet/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
Time to Execute :  130.42  ms
USB: Myriad Execution Finished
Time to Execute :  108.35  ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
                                                                                                                                                                                                                      Bandwidth   time
#    Name                                                                                                                                                                                                       MFLOPs  (MB/s)    (ms)
======================================================================================================================================================================================================================================
0    data                                                                                                                                                                                                          0.0 46160.0   0.006
1    conv1/7x7_s2                                                                                                                                                                                                236.0  2466.8   5.713
2    pool1/3x3_s2                                                                                                                                                                                                  1.8  1340.9   1.142
3    pool1/norm1                                                                                                                                                                                                   0.0   701.9   0.546
4    conv2/3x3_reduce                                                                                                                                                                                             25.7   478.8   0.816
5    conv2/3x3                                                                                                                                                                                                   693.6   307.6  11.891
6    conv2/norm2                                                                                                                                                                                                   0.0   787.5   1.458
7    pool2/3x3_s2                                                                                                                                                                                                  1.4  1390.2   0.826
8    inception_3a/1x1                                                                                                                                                                                             19.3   535.1   0.581
9    inception_3a/3x3_reduce                                                                                                                                                                                      28.9   459.3   0.702
10   inception_3a/3x3                                                                                                                                                                                            173.4   314.3   4.788
11   inception_3a/5x5_reduce                                                                                                                                                                                       4.8  1025.6   0.286
12   inception_3a/5x5                                                                                                                                                                                             20.1   678.0   0.921
13   inception_3a/pool                                                                                                                                                                                             1.4   578.0   0.497
14   inception_3a/pool_proj                                                                                                                                                                                        9.6   640.6   0.467
15   inception_3b/1x1                                                                                                                                                                                             51.4   444.3   1.003
16   inception_3b/3x3_reduce                                                                                                                                                                                      51.4   446.2   0.998
17   inception_3b/3x3                                                                                                                                                                                            346.8   257.6   8.339
18   inception_3b/5x5_reduce                                                                                                                                                                                      12.8   877.2   0.454
19   inception_3b/5x5                                                                                                                                                                                            120.4   522.8   2.577
20   inception_3b/pool                                                                                                                                                                                             1.8   660.6   0.580
21   inception_3b/pool_proj                                                                                                                                                                                       25.7   618.7   0.669
22   pool3/3x3_s2                                                                                                                                                                                                  0.8  1215.8   0.590
23   inception_4a/1x1                                                                                                                                                                                             36.1   358.6   0.991
24   inception_4a/3x3_reduce                                                                                                                                                                                      18.1   498.7   0.536
25   inception_4a/3x3                                                                                                                                                                                             70.4   296.2   2.259
26   inception_4a/5x5_reduce                                                                                                                                                                                       3.0   756.2   0.257
27   inception_4a/5x5                                                                                                                                                                                              7.5   412.6   0.457
28   inception_4a/pool                                                                                                                                                                                             0.8   505.0   0.355
29   inception_4a/pool_proj                                                                                                                                                                                       12.0   547.8   0.435
30   inception_4b/1x1                                                                                                                                                                                             32.1   330.5   1.053
31   inception_4b/3x3_reduce                                                                                                                                                                                      22.5   375.9   0.800
32   inception_4b/3x3                                                                                                                                                                                             88.5   285.9   2.838
33   inception_4b/5x5_reduce                                                                                                                                                                                       4.8   573.7   0.375
34   inception_4b/5x5                                                                                                                                                                                             15.1   309.1   0.973
35   inception_4b/pool                                                                                                                                                                                             0.9   532.7   0.359
36   inception_4b/pool_proj                                                                                                                                                                                       12.8   537.0   0.473
37   inception_4c/1x1                                                                                                                                                                                             25.7   402.8   0.786
38   inception_4c/3x3_reduce                                                                                                                                                                                      25.7   389.8   0.812
39   inception_4c/3x3                                                                                                                                                                                            115.6   260.4   3.831
40   inception_4c/5x5_reduce                                                                                                                                                                                       4.8   584.9   0.367
41   inception_4c/5x5                                                                                                                                                                                             15.1   308.1   0.976
42   inception_4c/pool                                                                                                                                                                                             0.9   627.2   0.305
43   inception_4c/pool_proj                                                                                                                                                                                       12.8   551.2   0.461
44   inception_4d/1x1                                                                                                                                                                                             22.5   413.7   0.728
45   inception_4d/3x3_reduce                                                                                                                                                                                      28.9   472.7   0.702
46   inception_4d/3x3                                                                                                                                                                                            146.3   399.4   3.008
47   inception_4d/5x5_reduce                                                                                                                                                                                       6.4   637.4   0.349
48   inception_4d/5x5                                                                                                                                                                                             20.1   388.8   1.028
49   inception_4d/pool                                                                                                                                                                                             0.9   610.5   0.314
50   inception_4d/pool_proj                                                                                                                                                                                       12.8   566.9   0.448
51   inception_4e/1x1                                                                                                                                                                                             53.0   297.6   1.531
52   inception_4e/3x3_reduce                                                                                                                                                                                      33.1   258.2   1.389
53   inception_4e/3x3                                                                                                                                                                                            180.6   280.8   5.067
54   inception_4e/5x5_reduce                                                                                                                                                                                       6.6   493.5   0.465
55   inception_4e/5x5                                                                                                                                                                                             40.1   356.2   1.405
56   inception_4e/pool                                                                                                                                                                                             0.9   600.5   0.329
57   inception_4e/pool_proj                                                                                                                                                                                       26.5   430.8   0.758
58   pool4/3x3_s2                                                                                                                                                                                                  0.4  1221.7   0.255
59   inception_5a/1x1                                                                                                                                                                                             20.9   614.2   0.789
60   inception_5a/3x3_reduce                                                                                                                                                                                      13.0   553.5   0.599
61   inception_5a/3x3                                                                                                                                                                                             45.2   550.4   1.851
62   inception_5a/5x5_reduce                                                                                                                                                                                       2.6   336.2   0.382
63   inception_5a/5x5                                                                                                                                                                                             10.0   427.3   0.646
64   inception_5a/pool                                                                                                                                                                                             0.4   405.1   0.192
65   inception_5a/pool_proj                                                                                                                                                                                       10.4   484.7   0.580
66   inception_5b/1x1                                                                                                                                                                                             31.3   611.9   1.124
67   inception_5b/3x3_reduce                                                                                                                                                                                      15.7   608.1   0.629
68   inception_5b/3x3                                                                                                                                                                                             65.0   569.9   2.516
69   inception_5b/5x5_reduce                                                                                                                                                                                       3.9   362.9   0.424
70   inception_5b/5x5                                                                                                                                                                                             15.1   438.7   0.937
71   inception_5b/pool                                                                                                                                                                                             0.4   483.8   0.161
72   inception_5b/pool_proj                                                                                                                                                                                       10.4   502.0   0.560
73   pool5/7x7_s1                                                                                                                                                                                                  0.1   460.8   0.208
74   loss3/classifier                                                                                                                                                                                              2.0  2497.0   0.783
75   prob                                                                                                                                                                                                          0.0     9.5   0.200
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                                                                          Total inference time                   95.91
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w bvlc_googlenet.caffemodel -i ../../data/images/nps_electric_guitar.png -s 12 -id 546  deploy.prototxt -S 255 -M 110 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result:  (1000,)
1) 546 0.995
Expected:  (1000,)
1) 546 0.9946
------------------------------------------------------------
 Obtained values 
------------------------------------------------------------
 Obtained Min Pixel Accuracy: 0.049091799883171916% (max allowed=2%), Pass
 Obtained Average Pixel Accuracy: 7.989402774910559e-05% (max allowed=1%), Pass
 Obtained Percentage of wrong values: 0.0% (max allowed=0%), Pass
 Obtained Pixel-wise L2 error: 0.0017114860971367098% (max allowed=1%), Pass
 Obtained Global Sum Difference: 0.0007946491241455078
------------------------------------------------------------

making compile
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'
make -C SqueezeNet/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
LICENSE file already exists

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
Time to Execute :  78.37  ms
USB: Myriad Execution Finished
Time to Execute :  60.05  ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
                                                                                                                                                                                                                      Bandwidth   time
#    Name                                                                                                                                                                                                       MFLOPs  (MB/s)    (ms)
======================================================================================================================================================================================================================================
0    data                                                                                                                                                                                                          0.0 96516.2   0.003
1    conv1                                                                                                                                                                                                       347.7  1634.5   8.861
2    pool1                                                                                                                                                                                                         2.6  1445.0   1.561
3    fire2/squeeze1x1                                                                                                                                                                                              9.3  1222.9   0.455
4    fire2/expand1x1                                                                                                                                                                                               6.2   152.1   0.620
5    fire2/expand3x3                                                                                                                                                                                              55.8   475.0   1.789
6    fire3/squeeze1x1                                                                                                                                                                                             12.4  1466.3   0.506
7    fire3/expand1x1                                                                                                                                                                                               6.2   158.4   0.596
8    fire3/expand3x3                                                                                                                                                                                              55.8   475.9   1.785
9    fire4/squeeze1x1                                                                                                                                                                                             24.8   976.6   0.764
10   fire4/expand1x1                                                                                                                                                                                              24.8   174.3   1.106
11   fire4/expand3x3                                                                                                                                                                                             223.0   391.4   4.430
12   pool4                                                                                                                                                                                                         1.7  1268.5   1.164
13   fire5/squeeze1x1                                                                                                                                                                                             11.9   788.7   0.471
14   fire5/expand1x1                                                                                                                                                                                               6.0   151.4   0.347
15   fire5/expand3x3                                                                                                                                                                                              53.7   376.4   1.257
16   fire6/squeeze1x1                                                                                                                                                                                             17.9   699.1   0.543
17   fire6/expand1x1                                                                                                                                                                                              13.4   159.6   0.531
18   fire6/expand3x3                                                                                                                                                                                             120.9   256.4   2.973
19   fire7/squeeze1x1                                                                                                                                                                                             26.9   827.4   0.688
20   fire7/expand1x1                                                                                                                                                                                              13.4   159.9   0.530
21   fire7/expand3x3                                                                                                                                                                                             120.9   259.7   2.935
22   fire8/squeeze1x1                                                                                                                                                                                             35.8   735.0   0.790
23   fire8/expand1x1                                                                                                                                                                                              23.9   155.7   0.775
24   fire8/expand3x3                                                                                                                                                                                             215.0   201.3   5.396
25   pool8                                                                                                                                                                                                         0.8  1265.2   0.563
26   fire9/squeeze1x1                                                                                                                                                                                             11.1   587.7   0.387
27   fire9/expand1x1                                                                                                                                                                                               5.5   153.4   0.341
28   fire9/expand3x3                                                                                                                                                                                              49.8   288.2   1.636
29   conv10                                                                                                                                                                                                      173.1   331.1   3.448
30   pool10                                                                                                                                                                                                        0.3   671.0   0.480
31   prob                                                                                                                                                                                                          0.0     9.5   0.202
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                                                                          Total inference time                   47.94
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w squeezenet_v1.0.caffemodel -i ../../data/images/cat.jpg -s 12 -id 281  deploy.prototxt -S 255 -M 120 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result:  (1000, 1, 1)
1) 281 0.6177
Expected:  (1000, 1, 1)
1) 281 0.616
------------------------------------------------------------
 Obtained values 
------------------------------------------------------------
 Obtained Min Pixel Accuracy: 0.6933439057320356% (max allowed=2%), Pass
 Obtained Average Pixel Accuracy: 0.0016120149666676298% (max allowed=1%), Pass
 Obtained Percentage of wrong values: 0.0% (max allowed=0%), Pass
 Obtained Pixel-wise L2 error: 0.029018872980330863% (max allowed=1%), Pass
 Obtained Global Sum Difference: 0.009933412075042725
------------------------------------------------------------

making compile
mvNCCompile -w squeezenet_v1.0.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe'
make -C data/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data'
Possible Make targets
  make help - shows this message
  make clean - Removes all temp files from all directories
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data'
make -C tensorflow/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/tensorflow'
make -C inception_v1/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1'
test -f output/inception-v1.meta || ((wget http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz && tar zxf inception_v1_2016_08_28.tar.gz && rm inception_v1_2016_08_28.tar.gz) && ./inception-v1.py)
--2024-07-29 13:06:28--  http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz
Resolving download.tensorflow.org (download.tensorflow.org)... 2607:f8b0:400a:803::201b, 2607:f8b0:400a:805::201b, 2607:f8b0:400a:807::201b, ...
Connecting to download.tensorflow.org (download.tensorflow.org)|2607:f8b0:400a:803::201b|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24642554 (24M) [application/x-tar]
Saving to: ‘inception_v1_2016_08_28.tar.gz’

inception_v1_2016_08_28.tar.gz                            100%[====================================================================================================================================>]  23.50M  98.9MB/s    in 0.2s    

2024-07-29 13:06:28 (98.9 MB/s) - ‘inception_v1_2016_08_28.tar.gz’ saved [24642554/24642554]

Traceback (most recent call last):
  File "/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1/./inception-v1.py", line 6, in <module>
    from tensorflow.contrib.slim.nets import inception
ModuleNotFoundError: No module named 'tensorflow.contrib'
make[2]: *** [Makefile:47: weights] Error 1
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1'
make[1]: *** [Makefile:12: inception_v1/.] Error 2
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/tensorflow'
make: *** [Makefile:12: tensorflow/.] Error 2

I’m gonna giving up the issue of Tensorflow version incompatibility, for Tensorflow 1.x has NOT been maintained for years. I mean example inception_v1 and inception_v3 are compatible with Tensorflow 1.x, rather than 2.x.

1 2	➜ examples git:(master) ✗ ls tensorflow inception_v1 inception_v3 Makefile readme.md

2.4 ncsdk2-aarch64 for NCS2

2.4.1 Download the Right NCSDK-2.10.01.01.tar.gz

git clone https://github.com/LE0xUL/ncsdk2_aarch64.git won’t provide the correct NCSDK-2.10.01.01.tar.gz.

➜  ncsdk2_aarch64 git:(aarch64) ✗ ll
Permissions Size User    Date Modified Name
drwxr-xr-x     - lvision 29 Jul 15:34  .git
.rw-r--r--   564 lvision 29 Jul 12:16  .gitattributes
.rw-r--r--   212 lvision 29 Jul 12:16  .gitignore
......
.rw-r--r--   132 lvision 29 Jul 12:16  NCSDK-2.10.01.01.tar.gz
......

As you can see, the git cloned NCSDK-2.10.01.01.tar.gz is of size 132 bytes ONLY.

You have to download NCSDK-2.10.01.01.tar.gz by open up View raw at this page, which is actually of size 2.4M:

1
2
3

➜  ncsdk2_aarch64 git:(aarch64) ✗ ll NCSDK-2.10.01.01.tar.gz 
Permissions Size User    Date Modified Name
.rw-r--r--  2.4M lvision 29 Jul 16:12  NCSDK-2.10.01.01.tar.gz

After extraction, you’ll also see the most important folder named ncsdk-aarch64, which actually contains the Aarch64 firmware for NCS2.

➜  ncsdk2_aarch64 git:(aarch64) ✗ ls
api           docs      extras             install-utilities.sh  LICENSE   NCSDK-2.10.01.01         ncsdk.conf  requirements.txt      requirements_apt_raspbian.txt                  uninstall-opencv.sh  version.txt
ATTRIBUTIONS  examples  install-opencv.sh  install.sh            Makefile  NCSDK-2.10.01.01.tar.gz  README.md   requirements_apt.txt  tensorflow-1.11.0-cp35-none-linux_aarch64.whl  uninstall.sh         
➜  ncsdk2_aarch64 git:(aarch64) ✗ cd NCSDK-2.10.01.01 
➜  NCSDK-2.10.01.01 git:(aarch64) ✗ ls
ncsdk-aarch64  ncsdk-armv7l  ncsdk-x86_64  version.txt

2.4.2 Installation

➜  ncsdk2_aarch64 git:(aarch64) ✗ cd api/src 
➜  src git:(aarch64) ✗ sudo make basicinstall
NCSDK FW successfully installed
mkdir -p /usr/local/include/
mkdir -p /usr/local/include/mvnc2
mkdir -p /usr/local/lib/
cp obj-aarch64/libmvnc.so.0 /usr/local/lib/
ln -fs libmvnc.so.0 /usr/local/lib/libmvnc.so
cp ../include/mvnc.h /usr/local/include/mvnc2
ln -fs /usr/local/include/mvnc2/mvnc.h /usr/local/include/mvnc.h 
mkdir -p /usr/local/lib/mvnc
cp mvnc/MvNCAPI-*.mvcmd /usr/local/lib/mvnc/
mkdir -p /etc/udev/rules.d/
cp 97-ncs2.rules /etc/udev/rules.d/
➜  src git:(aarch64) ✗ sudo make pythoninstall
mkdir -p /usr/local/lib/python3.11/dist-packages
cp -r ../python/mvnc /usr/local/lib/python3.11/dist-packages/
➜  src git:(aarch64) ✗ sudo make postinstall
udevadm control --reload-rules
udevadm trigger
ldconfig

2.4.3 Demonstrate Examples on My Raspberry Pi 5

Let’s just try hello_ncs.py.

➜  hello_ncs_py git:(aarch64) ✗ python hello_ncs.py 
D: [         0] ncDeviceCreate:307	ncDeviceCreate index 0

D: [         0] ncDeviceCreate:307	ncDeviceCreate index 1

D: [         0] ncDeviceOpen:523	File path /usr/local/lib/mvnc/MvNCAPI-ma2480.mvcmd

W: [         0] ncDeviceOpen:527	ncDeviceOpen() XLinkBootRemote returned error 3

Error - Could not open NCS device.

It looks I dealt with this issue in my old blog OpenVINO on Raspberry Pi 4 with Movidius Neural Compute Stick II, where I’d already provided the solution:

OpenVINO should be utilized instead of NCSDK ncsdk2.

3. Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3 for NCS2

3.1 Fix an Error

File /opt/intel/OpenVINO/openvino/thirdparty/ocv/opencv_hal_neon.hpp function v_deinterleave_expand is clearly buggy, for -1 should NEVER appear in an uchar array. Anyway, just change -1 to 255 as a temporary solution, namely:

Change

constexpr int nlanes = static_cast<int>(v_uint8x16::nlanes);
     uchar mask_e[nlanes] = { 0, -1, 2, -1, 4, -1, 6, -1,                                                                                                                                                                      
                             8, -1, 10, -1, 12, -1, 14, -1 };
 
     uchar mask_o[nlanes] = { 1, -1, 3, -1, 5, -1, 7, -1,
                             9, -1, 11, -1, 13, -1, 15, -1 };

to:

constexpr int nlanes = static_cast<int>(v_uint8x16::nlanes);
     uchar mask_e[nlanes] = { 0, 255, 2, 255, 4, 255, 6, 255,                                                                                                                                                                      
                             8, 255, 10, 255, 12, 255, 14, 255 };
 
     uchar mask_o[nlanes] = { 1, 255, 3, 255, 5, 255, 7, 255,
                             9, 255, 11, 255, 13, 255, 15, 255 };

3.2 Two More Things After Installation

3.2.1 OpenVINO Python Wrapper Installation

Under folder /opt/intel/OpenVINO/openvino/build/wheels, there are the following 2 files：

➜  wheels git:(2022.3.2) ✗ ll
Permissions Size User    Date Modified Name
.rw-r--r--   13M lvision 30 Jul 14:09  openvino-2022.3.2-9279-cp311-cp311-manylinux_2_36_aarch64.whl
.rw-r--r--   24k lvision 30 Jul 13:08  openvino_dev-2022.3.2-9279-py3-none-any.whl

Use key parameter --break-system-packages

1	sudo pip install --break-system-packages openvino-2022.3.2-9279-cp311-cp311-manylinux_2_36_aarch64.whl

and

1	sudo pip install --break-system-packages openvino_dev-2022.3.2-9279-py3-none-any.whl

3.2.2 Copy `plugins.xml`

A kind of weird that

Under build/lib, there is ONLY 1 file libade.a:

➜  lib git:(2022.3.2) ✗ pwd
/opt/intel/OpenVINO/openvino/build/lib
➜  lib git:(2022.3.2) ✗ ls
libade.a

Everything is built under bin/aarch64/Release:

➜  Release git:(2022.3.2) ✗ pwd                      
/opt/intel/OpenVINO/openvino/bin/aarch64/Release
➜  Release git:(2022.3.2) ✗ ll
Permissions Size User    Date Modified Name
.rwxr-xr-x  723k lvision 30 Jul 14:08  benchmark_app
.rwxr-xr-x  461k lvision 30 Jul 14:08  benchmark_app_legacy
.rwxr-xr-x  264k lvision 30 Jul 14:08  classification_sample_async
.rwxr-xr-x  264k lvision 30 Jul 14:08  compile_tool
.rwxr-xr-x  199k lvision 30 Jul 14:08  hello_classification
.rwxr-xr-x   67k lvision 30 Jul 14:08  hello_classification_c
.rwxr-xr-x  199k lvision 30 Jul 14:08  hello_nv12_input_classification
.rwxr-xr-x   67k lvision 30 Jul 14:08  hello_nv12_input_classification_c
.rwxr-xr-x   68k lvision 30 Jul 14:08  hello_query_device
.rwxr-xr-x  199k lvision 30 Jul 14:08  hello_reshape_ssd
.rw-r--r--  367k lvision 30 Jul 12:04  libcnpy.a
.rw-r--r--  7.5M lvision 30 Jul 14:08  libfluid.a
.rwxr-xr-x   67k lvision 30 Jul 14:08  libformat_reader.so
.rw-r--r--  3.4M lvision 30 Jul 12:08  libie_docs_snippets.a
.rw-r--r--  166k lvision 30 Jul 12:34  libie_samples_utils.a
.rw-r--r--   16M lvision 30 Jul 12:31  libinference_engine_legacy.a
.rw-r--r--  2.9M lvision 30 Jul 12:36  libinference_engine_snippets.a
.rw-r--r--   16M lvision 30 Jul 12:56  libinterpreter_backend.a
.rw-r--r--  3.0k lvision 30 Jul 12:04  libitt.a
.rw-r--r--  296k lvision 30 Jul 12:05  libmvnc.a
.rw-r--r--  1.6M lvision 30 Jul 12:04  libngraph_builders.a
.rw-r--r--  1.6M lvision 30 Jul 14:07  libngraph_reference.a
.rw-r--r--  2.3M lvision 30 Jul 12:33  liboffline_transformations.a
.rw-r--r--   11M lvision 30 Jul 12:17  libonnx.a
.rw-r--r--   54k lvision 30 Jul 12:31  libonnx_common.a
.rw-r--r--  676k lvision 30 Jul 12:06  libonnx_proto.a
.rwxr-xr-x   67k lvision 30 Jul 12:04  libopencv_c_wrapper.so
lrwxrwxrwx    19 lvision 30 Jul 14:08  libopenvino.so -> libopenvino.so.2232
.rwxr-xr-x   17M lvision 30 Jul 14:08  libopenvino.so.2022.3.2
lrwxrwxrwx    23 lvision 30 Jul 14:08  libopenvino.so.2232 -> libopenvino.so.2022.3.2
.rwxr-xr-x  330k lvision 30 Jul 14:08  libopenvino_auto_batch_plugin.so
.rwxr-xr-x  592k lvision 30 Jul 14:08  libopenvino_auto_plugin.so
lrwxrwxrwx    21 lvision 30 Jul 14:08  libopenvino_c.so -> libopenvino_c.so.2232
.rwxr-xr-x  461k lvision 30 Jul 14:08  libopenvino_c.so.2022.3.2
lrwxrwxrwx    25 lvision 30 Jul 14:08  libopenvino_c.so.2232 -> libopenvino_c.so.2022.3.2
.rwxr-xr-x  1.2M lvision 30 Jul 14:08  libopenvino_gapi_preproc.so
.rwxr-xr-x  395k lvision 30 Jul 14:08  libopenvino_hetero_plugin.so
.rwxr-xr-x  9.3M lvision 30 Jul 14:08  libopenvino_intel_myriad_plugin.so
lrwxrwxrwx    31 lvision 30 Jul 14:08  libopenvino_ir_frontend.so -> libopenvino_ir_frontend.so.2232
.rwxr-xr-x  528k lvision 30 Jul 14:08  libopenvino_ir_frontend.so.2022.3.2
lrwxrwxrwx    35 lvision 30 Jul 14:08  libopenvino_ir_frontend.so.2232 -> libopenvino_ir_frontend.so.2022.3.2
lrwxrwxrwx    33 lvision 30 Jul 14:08  libopenvino_onnx_frontend.so -> libopenvino_onnx_frontend.so.2232
.rwxr-xr-x  4.7M lvision 30 Jul 14:08  libopenvino_onnx_frontend.so.2022.3.2
lrwxrwxrwx    37 lvision 30 Jul 14:08  libopenvino_onnx_frontend.so.2232 -> libopenvino_onnx_frontend.so.2022.3.2
lrwxrwxrwx    35 lvision 30 Jul 14:08  libopenvino_paddle_frontend.so -> libopenvino_paddle_frontend.so.2232
.rwxr-xr-x  1.9M lvision 30 Jul 14:08  libopenvino_paddle_frontend.so.2022.3.2
lrwxrwxrwx    39 lvision 30 Jul 14:08  libopenvino_paddle_frontend.so.2232 -> libopenvino_paddle_frontend.so.2022.3.2
.rwxr-xr-x   68k lvision 30 Jul 14:08  libopenvino_template_extension.so
.rwxr-xr-x  8.0M lvision 30 Jul 14:08  libopenvino_template_plugin.so
lrwxrwxrwx    39 lvision 30 Jul 14:08  libopenvino_tensorflow_frontend.so -> libopenvino_tensorflow_frontend.so.2232
.rwxr-xr-x  3.5M lvision 30 Jul 14:08  libopenvino_tensorflow_frontend.so.2022.3.2
lrwxrwxrwx    43 lvision 30 Jul 14:08  libopenvino_tensorflow_frontend.so.2232 -> libopenvino_tensorflow_frontend.so.2022.3.2
.rw-r--r--  3.2k lvision 30 Jul 12:04  libov_protobuf_shutdown.a
.rw-r--r--  1.2k lvision 30 Jul 12:02  libov_shape_inference.a
.rw-r--r--  1.8M lvision 29 Jul 20:03  libprotobuf-lite.a
.rw-r--r--  7.8M lvision 29 Jul 20:04  libprotobuf.a
.rw-r--r--  8.3M lvision 29 Jul 21:07  libprotoc.a
.rw-r--r--  419k lvision 29 Jul 20:01  libpugixml.a
.rwxr-xr-x  133k lvision 30 Jul 14:08  libtemplate_extension.so
.rw-r--r--  165k lvision 30 Jul 12:03  libutil.a
.rw-r--r--  9.9M lvision 30 Jul 12:52  libvpu_common_lib.a
.rw-r--r--   30M lvision 30 Jul 13:07  libvpu_graph_transformer.a
.rw-r--r--  296k lvision 30 Jul 12:04  libXLink.a
.rwxr-xr-x  265k lvision 30 Jul 14:08  model_creation_sample
.rwxr-xr-x   68k lvision 30 Jul 14:08  ov_integration_snippet
.rwxr-xr-x   67k lvision 30 Jul 14:08  ov_integration_snippet_c
.rw-r--r--  2.1M lvision 29 Jul 20:02  pcie-ma2x8x.mvcmd
.rw-r--r--   486 lvision 30 Jul 14:08  plugins.xml
lrwxrwxrwx    15 lvision 30 Jul 13:56  protoc -> /usr/bin/protoc
.rwxr-xr-x  3.7M lvision 29 Jul 21:07  protoc-3.20.3.0.bak
drwxr-xr-x     - lvision 30 Jul 14:08  python_api
.rwxr-xr-x  462k lvision 30 Jul 14:08  speech_sample
.rwxr-xr-x   68k lvision 30 Jul 14:08  sync_benchmark
.rwxr-xr-x   68k lvision 30 Jul 14:08  throughput_benchmark
.rw-r--r--  2.4M lvision 29 Jul 20:02  usb-ma2x8x.mvcmd

Under which, there is a file plugins.xml. VERY IMPORTANT: put plugins.xml under /usr/local/lib.

3.3 Demonstration

3.3.1 hello_query_device.py

Go to /opt/intel/OpenVINO/openvino/samples/python/hello_query_device, and run Python script hello_query_device.py:

OpenVINO Runtime no longer supports Intel NCS2 and Movidius Myriad

3.3.2 AI Samples

Please refer to my NEXT blog Raspberry Pi 5 + Intel NCS2 + OpenVINO In 2024:

1. OpenVINO Runtime NO Longer Supports Movidius VPU

1.1 OpenVINO Toolkit Release Notes 2023.0

1.2 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3

1.3 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3

1.4 OpenVINO 2024.2 @ Intel® Distribution of OpenVINO™ Toolkit Release Notes 2024.2

1.5 Conclusion

2. NCSDK Does NOT Support AArch64

2.1 get_mvcmd.sh is the key.

2.2 Firmware Is Actually Downloaded But Without AArch64 Support

2.3 ncsdk-aarch64 for NCS1

2.3.1 Find the Right Version

2.3.2 Installation

2.3.3 Demonstrate Examples on My Raspberry Pi 5

2.4 ncsdk2-aarch64 for NCS2

2.4.1 Download the Right NCSDK-2.10.01.01.tar.gz

2.4.2 Installation

2.4.3 Demonstrate Examples on My Raspberry Pi 5

3. Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3 for NCS2

3.1 Fix an Error

3.2 Two More Things After Installation

3.2.1 OpenVINO Python Wrapper Installation

3.2.2 Copy plugins.xml

3.3 Demonstration

3.3.1 hello_query_device.py

3.3.2 AI Samples

2.1 `get_mvcmd.sh` is the key.

3.2.2 Copy `plugins.xml`