Intel NCS2 vs. Intel NCS1

Posted on 2024-07-28 Edited on 2024-09-30 In AI , Edge Computing

As of year 2024, it looks NCSDK is NOT maintained. To clarify the relationships among OpenVINO, NCSDK master and NCSDK ncsdk2 is a kind of complicated.

1. OpenVINO Runtime NO Longer Supports Movidius VPU

1.1 OpenVINO Toolkit Release Notes 2023.0

And it looks Long Term Support (LTS) version of OpenVINO ending in December 2024 now still supports for the devices listed above.

Therefore, in order to get my old devices: 1 NCS2 and 1 NCS1 working in the year of 2024, I’ll have to resort to Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS).

1.2 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3

1.3 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3

Ever since LTS 2023.3, OpenVINO started emphasize GenAI. On its release notes, nowhere is talking about NCS2 or NCS1 or anything related to Movidius VPU. According to OpenVINO Toolkit Release Notes 2023.0 in the above, I think ever since 2023, OpenVINO turns its direction to GenAI, and NEVER support for Movidius VPU any more. Therefore, most probably, including this Long-Term Support version, Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3 does NOT have any support to Movidius VPU devices.

1.4 OpenVINO 2024.2 @ Intel® Distribution of OpenVINO™ Toolkit Release Notes 2024.2

Not to mention the MOST UP-TO-DATE version 2024.2, which seems to be NOT a LTS. Of course, it does NOT support Movidius VPU devices.

1.5 Conclusion

Go for:

2. NCSDK Does NOT Support AArch64

2.1 `get_mvcmd.sh` is the key.

ncsdk_download_link for NCSDK ncsdk2: Please refer to NCSDK ncsdk2's install-utilities.sh, it is https://downloadmirror.intel.com/28532/eng/NCSDK-2.10.01.01.tar.gz.
ncsdk_link for NCSDK master: Please refer to NCSDK master's api/src/get_mvcmd.sh, it is https://downloadmirror.intel.com/28192/eng/NCSDK-1.12.01.01.tar.gz.

2.2 Firmware Is Actually Downloaded But Without AArch64 Support

➜  ncsdk ls NCSDK-2.10.01.01
ncsdk-armv7l  ncsdk-x86_64  version.txt
➜  ncsdk ll NCSDK-2.10.01.01/ncsdk-x86_64/fw                   
Permissions Size User Date Modified Name
.rwxrwxrwx  1.1M root 26 Jan  2019  MvNCAPI-ma2450.mvcmd
➜  ncsdk ll NCSDK-2.10.01.01/ncsdk-armv7l/fw 
Permissions Size User Date Modified Name
.rwxrwxrwx  1.1M root 26 Jan  2019  MvNCAPI-ma2450.mvcmd

➜  ncsdk ls NCSDK-1.12.01.01
install-ncsdk.sh  ncsdk-armv7l  ncsdk-x86_64  ncsdk.conf  requirements.txt  requirements_apt.txt  tests  uninstall-ncsdk.sh  version.txt
➜  ncsdk ll NCSDK-1.12.01.01/ncsdk-x86_64/fw 
Permissions Size User Date Modified Name
.rwxrwxrwx  866k root  5 Jan  2018  MvNCAPI.mvcmd
➜  ncsdk ll NCSDK-1.12.01.01/ncsdk-armv7l/fw 
Permissions Size User Date Modified Name
.rwxrwxrwx  866k root  5 Jan  2018  MvNCAPI.mvcmd

2.3 ncsdk-aarch64 for NCS1

Fortunately, I found ncsdk-aarch64. However, it only provides ncsdk-aarch64’s firmware from NCSDK-1.12.00.01, which seems to be also okay for NCSDK-1.12.01.01.

Let’s try it out.

2.3.1 Find the Right Version

Either checkout official ncsdk-aarch64 with LongerVision pull request for ncsdk-aarch64
Or directly clone LongerVision ncsdk-aarch64

You will see a folder named NCSDK-1.12.01.01 (or NCSDK-1.12.00.01), under which there is the key folder ncsdk-aarch64, which actually contains the Aarch64 firmware for NCS1.

➜  ncsdk-aarch64 git:(master) ✗ ls
api  docs  examples  install-opencv.sh  install.sh  LICENSE  Makefile  NCSDK-1.12.01.01  ncsdk.conf  README.md  uninstall-opencv.sh  uninstall.sh
➜  ncsdk-aarch64 git:(master) ✗ cd NCSDK-1.12.01.01 
➜  NCSDK-1.12.01.01 git:(master) ✗ ls
install-ncsdk.sh  ncsdk-aarch64  ncsdk-armv7l  ncsdk-x86_64  ncsdk.conf  requirements.txt  requirements_apt.txt  tests  uninstall-ncsdk.sh  version.txt

2.3.2 Installation

➜  ncsdk-aarch64 git:(master) ✗ cd api/src
➜  src git:(master) ✗ sudo make basicinstall 
mkdir -p /usr/local/include/
mkdir -p /usr/local/lib/
cp obj-aarch64/libmvnc.so.0 /usr/local/lib/
ln -fs libmvnc.so.0 /usr/local/lib/libmvnc.so
cp ../include/*.h /usr/local/include/
mkdir -p /usr/local/lib/mvnc
cp mvnc/MvNCAPI.mvcmd /usr/local/lib/mvnc/
mkdir -p /etc/udev/rules.d/
cp 97-usbboot.rules /etc/udev/rules.d/
➜  src git:(master) ✗ sudo make pythoninstall
mkdir -p /usr/local/lib/python3.11/dist-packages
cp -r ../python/mvnc /usr/local/lib/python3.11/dist-packages/
➜  src git:(master) ✗ sudo make postinstall
udevadm control --reload-rules
udevadm trigger
ldconfig

2.3.3 Demonstrate Examples on My Raspberry Pi 5

➜  examples git:(master) ✗ make
make -C apps/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps'
make -C hello_ncs_cpp/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_cpp'

making hello_ncs_cpp
g++ cpp/hello_ncs.cpp -o cpp/hello_ncs_cpp -lmvnc
Created cpp/hello_ncs_cpp executable
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_cpp'
make -C hello_ncs_py/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_py'
nothing to make, use 'make run' to run.
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/hello_ncs_py'
make -C multistick_cpp/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/apps/multistick_cpp'

making googlenet
(cd ../../caffe/GoogLeNet; make compile; cd ../../apps/multistick_cpp; cp ../../caffe/GoogLeNet/graph ./googlenet.graph;)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[4]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[4]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'

making prototxt
Prototxt file already exists

making caffemodel
caffemodel file already exists

making compile
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making squeezenet
(cd ../../caffe/SqueezeNet; make compile; cd ../../apps/multistick_cpp; cp ../../caffe/SqueezeNet/graph ./squeezenet.graph;)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[4]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[4]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
LICENSE file already exists

making prototxt
Prototxt file already exists

making caffemodel
caffemodel file already exists

making compile
mvNCCompile -w squeezenet_v1.0.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making multistick_cpp
cp googlenet.graph cpp/googlenet.graph;
cp squeezenet.graph cpp/squeezenet.graph;
g++ cpp/multistick.cpp cpp/fp16.c -o cpp/multistick_cpp -lmvnc
cpp/multistick.cpp: In function ‘bool DoInferenceOnImageFile(void*, const char*, int, float*)’:
cpp/multistick.cpp:307:1: warning: control reaches end of non-void function [-Wreturn-type]
  307 | }
      | ^
Created cpp/multistick_cpp executable
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps/multistick_cpp'
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/apps'
make -C caffe/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe'
make -C AlexNet/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/AlexNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
chmod +x run.py

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
Time to Execute :  107.91  ms
USB: Myriad Execution Finished
Time to Execute :  103.26  ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
                                                                                                                                                                                                                      Bandwidth   time
#    Name                                                                                                                                                                                                       MFLOPs  (MB/s)    (ms)
======================================================================================================================================================================================================================================
0    data                                                                                                                                                                                                          0.0126457.6   0.002
1    conv1                                                                                                                                                                                                       210.8  7434.1   4.811
2    norm1                                                                                                                                                                                                         0.0   721.2   0.768
3    pool1                                                                                                                                                                                                         0.6  1253.9   0.442
4    pool1_s0                                                                                                                                                                                                      0.0  1372.8   0.049
5    conv2_p0                                                                                                                                                                                                    223.9   596.2   6.099
6    pool1_s1                                                                                                                                                                                                      0.0  1455.7   0.046
7    conv2_p1                                                                                                                                                                                                    223.9   585.2   6.213
8    norm2                                                                                                                                                                                                         0.0   697.8   0.510
9    pool2                                                                                                                                                                                                         0.4  1308.0   0.272
10   conv3                                                                                                                                                                                                       299.0   302.0   8.048
11   conv3_s0                                                                                                                                                                                                      0.0  1582.4   0.039
12   conv4_p0                                                                                                                                                                                                    112.1   543.7   3.219
13   conv3_s1                                                                                                                                                                                                      0.0  1492.2   0.041
14   conv4_p1                                                                                                                                                                                                    112.1   560.1   3.125
15   conv4_p0_s0                                                                                                                                                                                                   0.0  1314.0   0.047
16   conv5_p0                                                                                                                                                                                                     74.8   578.5   2.659
17   conv4_p0_s1                                                                                                                                                                                                   0.0  1317.5   0.047
18   conv5_p1                                                                                                                                                                                                     74.8   580.0   2.652
19   pool5                                                                                                                                                                                                         0.1   918.4   0.090
20   fc6                                                                                                                                                                                                          75.5  2144.1  33.589
21   fc7                                                                                                                                                                                                          33.6  2132.6  15.008
22   fc8                                                                                                                                                                                                           8.2  2651.8   2.949
23   prob                                                                                                                                                                                                          0.0     7.6   0.253
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                                                                          Total inference time                   90.98
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w bvlc_alexnet.caffemodel -i ../../data/images/cat.jpg -s 12 -id 281 deploy.prototxt -M 110 -S 255 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result:  (1000,)
1) 281 0.512
Expected:  (1000,)
1) 281 0.522
------------------------------------------------------------
 Obtained values 
------------------------------------------------------------
 Obtained Min Pixel Accuracy: 2.385406941175461% (max allowed=2%), Fail
 Obtained Average Pixel Accuracy: 0.0052463514293776825% (max allowed=1%), Pass
 Obtained Percentage of wrong values: 0.1% (max allowed=0%), Fail
 Obtained Pixel-wise L2 error: 0.09619102055778048% (max allowed=1%), Pass
 Obtained Global Sum Difference: 0.027384519577026367
------------------------------------------------------------

making compile
mvNCCompile -w bvlc_alexnet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/AlexNet'
make -C GoogLeNet/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
Time to Execute :  130.42  ms
USB: Myriad Execution Finished
Time to Execute :  108.35  ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
                                                                                                                                                                                                                      Bandwidth   time
#    Name                                                                                                                                                                                                       MFLOPs  (MB/s)    (ms)
======================================================================================================================================================================================================================================
0    data                                                                                                                                                                                                          0.0 46160.0   0.006
1    conv1/7x7_s2                                                                                                                                                                                                236.0  2466.8   5.713
2    pool1/3x3_s2                                                                                                                                                                                                  1.8  1340.9   1.142
3    pool1/norm1                                                                                                                                                                                                   0.0   701.9   0.546
4    conv2/3x3_reduce                                                                                                                                                                                             25.7   478.8   0.816
5    conv2/3x3                                                                                                                                                                                                   693.6   307.6  11.891
6    conv2/norm2                                                                                                                                                                                                   0.0   787.5   1.458
7    pool2/3x3_s2                                                                                                                                                                                                  1.4  1390.2   0.826
8    inception_3a/1x1                                                                                                                                                                                             19.3   535.1   0.581
9    inception_3a/3x3_reduce                                                                                                                                                                                      28.9   459.3   0.702
10   inception_3a/3x3                                                                                                                                                                                            173.4   314.3   4.788
11   inception_3a/5x5_reduce                                                                                                                                                                                       4.8  1025.6   0.286
12   inception_3a/5x5                                                                                                                                                                                             20.1   678.0   0.921
13   inception_3a/pool                                                                                                                                                                                             1.4   578.0   0.497
14   inception_3a/pool_proj                                                                                                                                                                                        9.6   640.6   0.467
15   inception_3b/1x1                                                                                                                                                                                             51.4   444.3   1.003
16   inception_3b/3x3_reduce                                                                                                                                                                                      51.4   446.2   0.998
17   inception_3b/3x3                                                                                                                                                                                            346.8   257.6   8.339
18   inception_3b/5x5_reduce                                                                                                                                                                                      12.8   877.2   0.454
19   inception_3b/5x5                                                                                                                                                                                            120.4   522.8   2.577
20   inception_3b/pool                                                                                                                                                                                             1.8   660.6   0.580
21   inception_3b/pool_proj                                                                                                                                                                                       25.7   618.7   0.669
22   pool3/3x3_s2                                                                                                                                                                                                  0.8  1215.8   0.590
23   inception_4a/1x1                                                                                                                                                                                             36.1   358.6   0.991
24   inception_4a/3x3_reduce                                                                                                                                                                                      18.1   498.7   0.536
25   inception_4a/3x3                                                                                                                                                                                             70.4   296.2   2.259
26   inception_4a/5x5_reduce                                                                                                                                                                                       3.0   756.2   0.257
27   inception_4a/5x5                                                                                                                                                                                              7.5   412.6   0.457
28   inception_4a/pool                                                                                                                                                                                             0.8   505.0   0.355
29   inception_4a/pool_proj                                                                                                                                                                                       12.0   547.8   0.435
30   inception_4b/1x1                                                                                                                                                                                             32.1   330.5   1.053
31   inception_4b/3x3_reduce                                                                                                                                                                                      22.5   375.9   0.800
32   inception_4b/3x3                                                                                                                                                                                             88.5   285.9   2.838
33   inception_4b/5x5_reduce                                                                                                                                                                                       4.8   573.7   0.375
34   inception_4b/5x5                                                                                                                                                                                             15.1   309.1   0.973
35   inception_4b/pool                                                                                                                                                                                             0.9   532.7   0.359
36   inception_4b/pool_proj                                                                                                                                                                                       12.8   537.0   0.473
37   inception_4c/1x1                                                                                                                                                                                             25.7   402.8   0.786
38   inception_4c/3x3_reduce                                                                                                                                                                                      25.7   389.8   0.812
39   inception_4c/3x3                                                                                                                                                                                            115.6   260.4   3.831
40   inception_4c/5x5_reduce                                                                                                                                                                                       4.8   584.9   0.367
41   inception_4c/5x5                                                                                                                                                                                             15.1   308.1   0.976
42   inception_4c/pool                                                                                                                                                                                             0.9   627.2   0.305
43   inception_4c/pool_proj                                                                                                                                                                                       12.8   551.2   0.461
44   inception_4d/1x1                                                                                                                                                                                             22.5   413.7   0.728
45   inception_4d/3x3_reduce                                                                                                                                                                                      28.9   472.7   0.702
46   inception_4d/3x3                                                                                                                                                                                            146.3   399.4   3.008
47   inception_4d/5x5_reduce                                                                                                                                                                                       6.4   637.4   0.349
48   inception_4d/5x5                                                                                                                                                                                             20.1   388.8   1.028
49   inception_4d/pool                                                                                                                                                                                             0.9   610.5   0.314
50   inception_4d/pool_proj                                                                                                                                                                                       12.8   566.9   0.448
51   inception_4e/1x1                                                                                                                                                                                             53.0   297.6   1.531
52   inception_4e/3x3_reduce                                                                                                                                                                                      33.1   258.2   1.389
53   inception_4e/3x3                                                                                                                                                                                            180.6   280.8   5.067
54   inception_4e/5x5_reduce                                                                                                                                                                                       6.6   493.5   0.465
55   inception_4e/5x5                                                                                                                                                                                             40.1   356.2   1.405
56   inception_4e/pool                                                                                                                                                                                             0.9   600.5   0.329
57   inception_4e/pool_proj                                                                                                                                                                                       26.5   430.8   0.758
58   pool4/3x3_s2                                                                                                                                                                                                  0.4  1221.7   0.255
59   inception_5a/1x1                                                                                                                                                                                             20.9   614.2   0.789
60   inception_5a/3x3_reduce                                                                                                                                                                                      13.0   553.5   0.599
61   inception_5a/3x3                                                                                                                                                                                             45.2   550.4   1.851
62   inception_5a/5x5_reduce                                                                                                                                                                                       2.6   336.2   0.382
63   inception_5a/5x5                                                                                                                                                                                             10.0   427.3   0.646
64   inception_5a/pool                                                                                                                                                                                             0.4   405.1   0.192
65   inception_5a/pool_proj                                                                                                                                                                                       10.4   484.7   0.580
66   inception_5b/1x1                                                                                                                                                                                             31.3   611.9   1.124
67   inception_5b/3x3_reduce                                                                                                                                                                                      15.7   608.1   0.629
68   inception_5b/3x3                                                                                                                                                                                             65.0   569.9   2.516
69   inception_5b/5x5_reduce                                                                                                                                                                                       3.9   362.9   0.424
70   inception_5b/5x5                                                                                                                                                                                             15.1   438.7   0.937
71   inception_5b/pool                                                                                                                                                                                             0.4   483.8   0.161
72   inception_5b/pool_proj                                                                                                                                                                                       10.4   502.0   0.560
73   pool5/7x7_s1                                                                                                                                                                                                  0.1   460.8   0.208
74   loss3/classifier                                                                                                                                                                                              2.0  2497.0   0.783
75   prob                                                                                                                                                                                                          0.0     9.5   0.200
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                                                                          Total inference time                   95.91
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w bvlc_googlenet.caffemodel -i ../../data/images/nps_electric_guitar.png -s 12 -id 546  deploy.prototxt -S 255 -M 110 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result:  (1000,)
1) 546 0.995
Expected:  (1000,)
1) 546 0.9946
------------------------------------------------------------
 Obtained values 
------------------------------------------------------------
 Obtained Min Pixel Accuracy: 0.049091799883171916% (max allowed=2%), Pass
 Obtained Average Pixel Accuracy: 7.989402774910559e-05% (max allowed=1%), Pass
 Obtained Percentage of wrong values: 0.0% (max allowed=0%), Pass
 Obtained Pixel-wise L2 error: 0.0017114860971367098% (max allowed=1%), Pass
 Obtained Global Sum Difference: 0.0007946491241455078
------------------------------------------------------------

making compile
mvNCCompile -w bvlc_googlenet.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

Layer  inception_3b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_3b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4b/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4c/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4d/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_4e/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5a/pool_proj  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/1x1  forced to im2col_v2, because its output is used in concat
Layer  inception_5b/pool_proj  forced to im2col_v2, because its output is used in concat
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/GoogLeNet'
make -C SqueezeNet/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'

making prereqs
(cd ../../data/ilsvrc12; make)
make[3]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
make[3]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data/ilsvrc12'
LICENSE file already exists

making prototxt
Prototxt file already exists

making profile
mvNCProfile deploy.prototxt -s 12
mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

****** WARNING: using empty weights ******
/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
Time to Execute :  78.37  ms
USB: Myriad Execution Finished
Time to Execute :  60.05  ms
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.
Network Summary

Detailed Per Layer Profile
                                                                                                                                                                                                                      Bandwidth   time
#    Name                                                                                                                                                                                                       MFLOPs  (MB/s)    (ms)
======================================================================================================================================================================================================================================
0    data                                                                                                                                                                                                          0.0 96516.2   0.003
1    conv1                                                                                                                                                                                                       347.7  1634.5   8.861
2    pool1                                                                                                                                                                                                         2.6  1445.0   1.561
3    fire2/squeeze1x1                                                                                                                                                                                              9.3  1222.9   0.455
4    fire2/expand1x1                                                                                                                                                                                               6.2   152.1   0.620
5    fire2/expand3x3                                                                                                                                                                                              55.8   475.0   1.789
6    fire3/squeeze1x1                                                                                                                                                                                             12.4  1466.3   0.506
7    fire3/expand1x1                                                                                                                                                                                               6.2   158.4   0.596
8    fire3/expand3x3                                                                                                                                                                                              55.8   475.9   1.785
9    fire4/squeeze1x1                                                                                                                                                                                             24.8   976.6   0.764
10   fire4/expand1x1                                                                                                                                                                                              24.8   174.3   1.106
11   fire4/expand3x3                                                                                                                                                                                             223.0   391.4   4.430
12   pool4                                                                                                                                                                                                         1.7  1268.5   1.164
13   fire5/squeeze1x1                                                                                                                                                                                             11.9   788.7   0.471
14   fire5/expand1x1                                                                                                                                                                                               6.0   151.4   0.347
15   fire5/expand3x3                                                                                                                                                                                              53.7   376.4   1.257
16   fire6/squeeze1x1                                                                                                                                                                                             17.9   699.1   0.543
17   fire6/expand1x1                                                                                                                                                                                              13.4   159.6   0.531
18   fire6/expand3x3                                                                                                                                                                                             120.9   256.4   2.973
19   fire7/squeeze1x1                                                                                                                                                                                             26.9   827.4   0.688
20   fire7/expand1x1                                                                                                                                                                                              13.4   159.9   0.530
21   fire7/expand3x3                                                                                                                                                                                             120.9   259.7   2.935
22   fire8/squeeze1x1                                                                                                                                                                                             35.8   735.0   0.790
23   fire8/expand1x1                                                                                                                                                                                              23.9   155.7   0.775
24   fire8/expand3x3                                                                                                                                                                                             215.0   201.3   5.396
25   pool8                                                                                                                                                                                                         0.8  1265.2   0.563
26   fire9/squeeze1x1                                                                                                                                                                                             11.1   587.7   0.387
27   fire9/expand1x1                                                                                                                                                                                               5.5   153.4   0.341
28   fire9/expand3x3                                                                                                                                                                                              49.8   288.2   1.636
29   conv10                                                                                                                                                                                                      173.1   331.1   3.448
30   pool10                                                                                                                                                                                                        0.3   671.0   0.480
31   prob                                                                                                                                                                                                          0.0     9.5   0.202
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                                                                                          Total inference time                   47.94
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Generating Profile Report 'output_report.html'...

making caffemodel
caffemodel file already exists

making check
mvNCCheck -w squeezenet_v1.0.caffemodel -i ../../data/images/cat.jpg -s 12 -id 281  deploy.prototxt -S 255 -M 120 -metric top1
mvNCCheck v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(
USB: Transferring Data...
/usr/local/lib/python3.11/dist-packages/mvnc/mvncapi.py:244: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
  tensor = tensor.tostring()
USB: Myriad Execution Finished
USB: Myriad Connection Closing.
USB: Myriad Connection Closed.

Result: Validation Pass

Result:  (1000, 1, 1)
1) 281 0.6177
Expected:  (1000, 1, 1)
1) 281 0.616
------------------------------------------------------------
 Obtained values 
------------------------------------------------------------
 Obtained Min Pixel Accuracy: 0.6933439057320356% (max allowed=2%), Pass
 Obtained Average Pixel Accuracy: 0.0016120149666676298% (max allowed=1%), Pass
 Obtained Percentage of wrong values: 0.0% (max allowed=0%), Pass
 Obtained Pixel-wise L2 error: 0.029018872980330863% (max allowed=1%), Pass
 Obtained Global Sum Difference: 0.009933412075042725
------------------------------------------------------------

making compile
mvNCCompile -w squeezenet_v1.0.caffemodel -s 12 deploy.prototxt
mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/bin/ncsdk/Controllers/FileIO.py:50: UserWarning: You are using a large type. Consider reducing your data sizes for best performance
  warnings.warn(

making cpp
g++ cpp/run.cpp cpp/fp16.c -o cpp/run_cpp -lmvnc
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe/SqueezeNet'
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/caffe'
make -C data/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/data'
Possible Make targets
  make help - shows this message
  make clean - Removes all temp files from all directories
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/data'
make -C tensorflow/. 
make[1]: Entering directory '/opt/intel/ncsdk-aarch64/examples/tensorflow'
make -C inception_v1/. 
make[2]: Entering directory '/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1'
test -f output/inception-v1.meta || ((wget http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz && tar zxf inception_v1_2016_08_28.tar.gz && rm inception_v1_2016_08_28.tar.gz) && ./inception-v1.py)
--2024-07-29 13:06:28--  http://download.tensorflow.org/models/inception_v1_2016_08_28.tar.gz
Resolving download.tensorflow.org (download.tensorflow.org)... 2607:f8b0:400a:803::201b, 2607:f8b0:400a:805::201b, 2607:f8b0:400a:807::201b, ...
Connecting to download.tensorflow.org (download.tensorflow.org)|2607:f8b0:400a:803::201b|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24642554 (24M) [application/x-tar]
Saving to: ‘inception_v1_2016_08_28.tar.gz’

inception_v1_2016_08_28.tar.gz                            100%[====================================================================================================================================>]  23.50M  98.9MB/s    in 0.2s    

2024-07-29 13:06:28 (98.9 MB/s) - ‘inception_v1_2016_08_28.tar.gz’ saved [24642554/24642554]

Traceback (most recent call last):
  File "/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1/./inception-v1.py", line 6, in <module>
    from tensorflow.contrib.slim.nets import inception
ModuleNotFoundError: No module named 'tensorflow.contrib'
make[2]: *** [Makefile:47: weights] Error 1
make[2]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/tensorflow/inception_v1'
make[1]: *** [Makefile:12: inception_v1/.] Error 2
make[1]: Leaving directory '/opt/intel/ncsdk-aarch64/examples/tensorflow'
make: *** [Makefile:12: tensorflow/.] Error 2

I'm gonna giving up the issue of Tensorflow version incompatibility, for Tensorflow 1.x has NOT been maintained for years. I mean example inception_v1 and inception_v3 are compatible with Tensorflow 1.x, rather than 2.x.

1 2	➜ examples git:(master) ✗ ls tensorflow inception_v1 inception_v3 Makefile readme.md

2.4 ncsdk2-aarch64 for NCS2

2.4.1 Download the Right NCSDK-2.10.01.01.tar.gz

git clone https://github.com/LE0xUL/ncsdk2_aarch64.git won’t provide the correct NCSDK-2.10.01.01.tar.gz.

➜  ncsdk2_aarch64 git:(aarch64) ✗ ll
Permissions Size User    Date Modified Name
drwxr-xr-x     - lvision 29 Jul 15:34  .git
.rw-r--r--   564 lvision 29 Jul 12:16  .gitattributes
.rw-r--r--   212 lvision 29 Jul 12:16  .gitignore
......
.rw-r--r--   132 lvision 29 Jul 12:16  NCSDK-2.10.01.01.tar.gz
......

As you can see, the git cloned NCSDK-2.10.01.01.tar.gz is of size 132 bytes ONLY.

You have to download NCSDK-2.10.01.01.tar.gz by open up View raw at this page, which is actually of size 2.4M:

1
2
3

➜  ncsdk2_aarch64 git:(aarch64) ✗ ll NCSDK-2.10.01.01.tar.gz 
Permissions Size User    Date Modified Name
.rw-r--r--  2.4M lvision 29 Jul 16:12  NCSDK-2.10.01.01.tar.gz

After extraction, you’ll also see the most important folder named ncsdk-aarch64, which actually contains the Aarch64 firmware for NCS2.

➜  ncsdk2_aarch64 git:(aarch64) ✗ ls
api           docs      extras             install-utilities.sh  LICENSE   NCSDK-2.10.01.01         ncsdk.conf  requirements.txt      requirements_apt_raspbian.txt                  uninstall-opencv.sh  version.txt
ATTRIBUTIONS  examples  install-opencv.sh  install.sh            Makefile  NCSDK-2.10.01.01.tar.gz  README.md   requirements_apt.txt  tensorflow-1.11.0-cp35-none-linux_aarch64.whl  uninstall.sh         
➜  ncsdk2_aarch64 git:(aarch64) ✗ cd NCSDK-2.10.01.01 
➜  NCSDK-2.10.01.01 git:(aarch64) ✗ ls
ncsdk-aarch64  ncsdk-armv7l  ncsdk-x86_64  version.txt

2.4.2 Installation

➜  ncsdk2_aarch64 git:(aarch64) ✗ cd api/src 
➜  src git:(aarch64) ✗ sudo make basicinstall
NCSDK FW successfully installed
mkdir -p /usr/local/include/
mkdir -p /usr/local/include/mvnc2
mkdir -p /usr/local/lib/
cp obj-aarch64/libmvnc.so.0 /usr/local/lib/
ln -fs libmvnc.so.0 /usr/local/lib/libmvnc.so
cp ../include/mvnc.h /usr/local/include/mvnc2
ln -fs /usr/local/include/mvnc2/mvnc.h /usr/local/include/mvnc.h 
mkdir -p /usr/local/lib/mvnc
cp mvnc/MvNCAPI-*.mvcmd /usr/local/lib/mvnc/
mkdir -p /etc/udev/rules.d/
cp 97-ncs2.rules /etc/udev/rules.d/
➜  src git:(aarch64) ✗ sudo make pythoninstall
mkdir -p /usr/local/lib/python3.11/dist-packages
cp -r ../python/mvnc /usr/local/lib/python3.11/dist-packages/
➜  src git:(aarch64) ✗ sudo make postinstall
udevadm control --reload-rules
udevadm trigger
ldconfig

2.4.3 Demonstrate Examples on My Raspberry Pi 5

Let’s just try hello_ncs.py.

➜  hello_ncs_py git:(aarch64) ✗ python hello_ncs.py 
D: [         0] ncDeviceCreate:307	ncDeviceCreate index 0

D: [         0] ncDeviceCreate:307	ncDeviceCreate index 1

D: [         0] ncDeviceOpen:523	File path /usr/local/lib/mvnc/MvNCAPI-ma2480.mvcmd

W: [         0] ncDeviceOpen:527	ncDeviceOpen() XLinkBootRemote returned error 3

Error - Could not open NCS device.

It looks I dealt with this issue in my old blog OpenVINO on Raspberry Pi 4 with Movidius Neural Compute Stick II, where I’d already provided the solution:

OpenVINO should be utilized instead of NCSDK ncsdk2.

3. Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3 for NCS2

3.1 Fix an Error

File /opt/intel/OpenVINO/openvino/thirdparty/ocv/opencv_hal_neon.hpp function v_deinterleave_expand is clearly buggy, for -1 should NEVER appear in an uchar array. Anyway, just change -1 to 255 as a temporary solution, namely:

Change

constexpr int nlanes = static_cast<int>(v_uint8x16::nlanes);
     uchar mask_e[nlanes] = { 0, -1, 2, -1, 4, -1, 6, -1,                                                                                                                                                                      
                             8, -1, 10, -1, 12, -1, 14, -1 };
 
     uchar mask_o[nlanes] = { 1, -1, 3, -1, 5, -1, 7, -1,
                             9, -1, 11, -1, 13, -1, 15, -1 };

to:

constexpr int nlanes = static_cast<int>(v_uint8x16::nlanes);
     uchar mask_e[nlanes] = { 0, 255, 2, 255, 4, 255, 6, 255,                                                                                                                                                                      
                             8, 255, 10, 255, 12, 255, 14, 255 };
 
     uchar mask_o[nlanes] = { 1, 255, 3, 255, 5, 255, 7, 255,
                             9, 255, 11, 255, 13, 255, 15, 255 };

3.2 Two More Things After Installation

3.2.1 OpenVINO Python Wrapper Installation

Under folder /opt/intel/OpenVINO/openvino/build/wheels, there are the following 2 files：

➜  wheels git:(2022.3.2) ✗ ll
Permissions Size User    Date Modified Name
.rw-r--r--   13M lvision 30 Jul 14:09  openvino-2022.3.2-9279-cp311-cp311-manylinux_2_36_aarch64.whl
.rw-r--r--   24k lvision 30 Jul 13:08  openvino_dev-2022.3.2-9279-py3-none-any.whl

Use key parameter --break-system-packages

1	sudo pip install --break-system-packages openvino-2022.3.2-9279-cp311-cp311-manylinux_2_36_aarch64.whl

and

1	sudo pip install --break-system-packages openvino_dev-2022.3.2-9279-py3-none-any.whl

3.2.2 Copy `plugins.xml`

A kind of weird that:

Under build/lib, there is ONLY 1 file libade.a:

➜  lib git:(2022.3.2) ✗ pwd
/opt/intel/OpenVINO/openvino/build/lib
➜  lib git:(2022.3.2) ✗ ls
libade.a

Everything is built under bin/aarch64/Release:

➜  Release git:(2022.3.2) ✗ pwd                      
/opt/intel/OpenVINO/openvino/bin/aarch64/Release
➜  Release git:(2022.3.2) ✗ ll
Permissions Size User    Date Modified Name
.rwxr-xr-x  723k lvision 30 Jul 14:08  benchmark_app
.rwxr-xr-x  461k lvision 30 Jul 14:08  benchmark_app_legacy
.rwxr-xr-x  264k lvision 30 Jul 14:08  classification_sample_async
.rwxr-xr-x  264k lvision 30 Jul 14:08  compile_tool
.rwxr-xr-x  199k lvision 30 Jul 14:08  hello_classification
.rwxr-xr-x   67k lvision 30 Jul 14:08  hello_classification_c
.rwxr-xr-x  199k lvision 30 Jul 14:08  hello_nv12_input_classification
.rwxr-xr-x   67k lvision 30 Jul 14:08  hello_nv12_input_classification_c
.rwxr-xr-x   68k lvision 30 Jul 14:08  hello_query_device
.rwxr-xr-x  199k lvision 30 Jul 14:08  hello_reshape_ssd
.rw-r--r--  367k lvision 30 Jul 12:04  libcnpy.a
.rw-r--r--  7.5M lvision 30 Jul 14:08  libfluid.a
.rwxr-xr-x   67k lvision 30 Jul 14:08  libformat_reader.so
.rw-r--r--  3.4M lvision 30 Jul 12:08  libie_docs_snippets.a
.rw-r--r--  166k lvision 30 Jul 12:34  libie_samples_utils.a
.rw-r--r--   16M lvision 30 Jul 12:31  libinference_engine_legacy.a
.rw-r--r--  2.9M lvision 30 Jul 12:36  libinference_engine_snippets.a
.rw-r--r--   16M lvision 30 Jul 12:56  libinterpreter_backend.a
.rw-r--r--  3.0k lvision 30 Jul 12:04  libitt.a
.rw-r--r--  296k lvision 30 Jul 12:05  libmvnc.a
.rw-r--r--  1.6M lvision 30 Jul 12:04  libngraph_builders.a
.rw-r--r--  1.6M lvision 30 Jul 14:07  libngraph_reference.a
.rw-r--r--  2.3M lvision 30 Jul 12:33  liboffline_transformations.a
.rw-r--r--   11M lvision 30 Jul 12:17  libonnx.a
.rw-r--r--   54k lvision 30 Jul 12:31  libonnx_common.a
.rw-r--r--  676k lvision 30 Jul 12:06  libonnx_proto.a
.rwxr-xr-x   67k lvision 30 Jul 12:04  libopencv_c_wrapper.so
lrwxrwxrwx    19 lvision 30 Jul 14:08  libopenvino.so -> libopenvino.so.2232
.rwxr-xr-x   17M lvision 30 Jul 14:08  libopenvino.so.2022.3.2
lrwxrwxrwx    23 lvision 30 Jul 14:08  libopenvino.so.2232 -> libopenvino.so.2022.3.2
.rwxr-xr-x  330k lvision 30 Jul 14:08  libopenvino_auto_batch_plugin.so
.rwxr-xr-x  592k lvision 30 Jul 14:08  libopenvino_auto_plugin.so
lrwxrwxrwx    21 lvision 30 Jul 14:08  libopenvino_c.so -> libopenvino_c.so.2232
.rwxr-xr-x  461k lvision 30 Jul 14:08  libopenvino_c.so.2022.3.2
lrwxrwxrwx    25 lvision 30 Jul 14:08  libopenvino_c.so.2232 -> libopenvino_c.so.2022.3.2
.rwxr-xr-x  1.2M lvision 30 Jul 14:08  libopenvino_gapi_preproc.so
.rwxr-xr-x  395k lvision 30 Jul 14:08  libopenvino_hetero_plugin.so
.rwxr-xr-x  9.3M lvision 30 Jul 14:08  libopenvino_intel_myriad_plugin.so
lrwxrwxrwx    31 lvision 30 Jul 14:08  libopenvino_ir_frontend.so -> libopenvino_ir_frontend.so.2232
.rwxr-xr-x  528k lvision 30 Jul 14:08  libopenvino_ir_frontend.so.2022.3.2
lrwxrwxrwx    35 lvision 30 Jul 14:08  libopenvino_ir_frontend.so.2232 -> libopenvino_ir_frontend.so.2022.3.2
lrwxrwxrwx    33 lvision 30 Jul 14:08  libopenvino_onnx_frontend.so -> libopenvino_onnx_frontend.so.2232
.rwxr-xr-x  4.7M lvision 30 Jul 14:08  libopenvino_onnx_frontend.so.2022.3.2
lrwxrwxrwx    37 lvision 30 Jul 14:08  libopenvino_onnx_frontend.so.2232 -> libopenvino_onnx_frontend.so.2022.3.2
lrwxrwxrwx    35 lvision 30 Jul 14:08  libopenvino_paddle_frontend.so -> libopenvino_paddle_frontend.so.2232
.rwxr-xr-x  1.9M lvision 30 Jul 14:08  libopenvino_paddle_frontend.so.2022.3.2
lrwxrwxrwx    39 lvision 30 Jul 14:08  libopenvino_paddle_frontend.so.2232 -> libopenvino_paddle_frontend.so.2022.3.2
.rwxr-xr-x   68k lvision 30 Jul 14:08  libopenvino_template_extension.so
.rwxr-xr-x  8.0M lvision 30 Jul 14:08  libopenvino_template_plugin.so
lrwxrwxrwx    39 lvision 30 Jul 14:08  libopenvino_tensorflow_frontend.so -> libopenvino_tensorflow_frontend.so.2232
.rwxr-xr-x  3.5M lvision 30 Jul 14:08  libopenvino_tensorflow_frontend.so.2022.3.2
lrwxrwxrwx    43 lvision 30 Jul 14:08  libopenvino_tensorflow_frontend.so.2232 -> libopenvino_tensorflow_frontend.so.2022.3.2
.rw-r--r--  3.2k lvision 30 Jul 12:04  libov_protobuf_shutdown.a
.rw-r--r--  1.2k lvision 30 Jul 12:02  libov_shape_inference.a
.rw-r--r--  1.8M lvision 29 Jul 20:03  libprotobuf-lite.a
.rw-r--r--  7.8M lvision 29 Jul 20:04  libprotobuf.a
.rw-r--r--  8.3M lvision 29 Jul 21:07  libprotoc.a
.rw-r--r--  419k lvision 29 Jul 20:01  libpugixml.a
.rwxr-xr-x  133k lvision 30 Jul 14:08  libtemplate_extension.so
.rw-r--r--  165k lvision 30 Jul 12:03  libutil.a
.rw-r--r--  9.9M lvision 30 Jul 12:52  libvpu_common_lib.a
.rw-r--r--   30M lvision 30 Jul 13:07  libvpu_graph_transformer.a
.rw-r--r--  296k lvision 30 Jul 12:04  libXLink.a
.rwxr-xr-x  265k lvision 30 Jul 14:08  model_creation_sample
.rwxr-xr-x   68k lvision 30 Jul 14:08  ov_integration_snippet
.rwxr-xr-x   67k lvision 30 Jul 14:08  ov_integration_snippet_c
.rw-r--r--  2.1M lvision 29 Jul 20:02  pcie-ma2x8x.mvcmd
.rw-r--r--   486 lvision 30 Jul 14:08  plugins.xml
lrwxrwxrwx    15 lvision 30 Jul 13:56  protoc -> /usr/bin/protoc
.rwxr-xr-x  3.7M lvision 29 Jul 21:07  protoc-3.20.3.0.bak
drwxr-xr-x     - lvision 30 Jul 14:08  python_api
.rwxr-xr-x  462k lvision 30 Jul 14:08  speech_sample
.rwxr-xr-x   68k lvision 30 Jul 14:08  sync_benchmark
.rwxr-xr-x   68k lvision 30 Jul 14:08  throughput_benchmark
.rw-r--r--  2.4M lvision 29 Jul 20:02  usb-ma2x8x.mvcmd

Under which, there is a file plugins.xml. VERY IMPORTANT: put plugins.xml under /usr/local/lib.

3.3 Demonstration

3.3.1 hello_query_device.py

Go to /opt/intel/OpenVINO/openvino/samples/python/hello_query_device, and run Python script hello_query_device.py:

3.3.2 AI Samples

Please refer to my NEXT blog Raspberry Pi 5 + Intel NCS2 + OpenVINO In 2024:

libreELEC on A Raspberry Pi

Posted on 2024-07-27 Edited on 2024-10-29 In Operating System , Raspberry Pi

Try a media center on a Raspberry Pi.

1. Overview

1.1 What is LibreELEC? What is Kodi?

What is LibreELEC? Directly cited from LibreELEC Wiki: LibreELEC is a minimalist 'Just enough OS' Linux distribution for running Kodi.
What is Kodi? Directly cited from About Kodi: Kodi is an award-winning free and open source (GPL) software media player and entertainment hub that can be installed on Linux, OSX, Windows, iOS, tvOS and Android. It is designed around a "10-foot user interface" for use with televisions and remote controls.

1.2 Flash LibreELEC Onto Raspberry Pi

$ sudo dd if=./LibreELEC-RPi4.aarch64-12.0.0.img of=/dev/sdb bs=1M status=progress conv=fsync
[sudo] password for lvision: 
549+0 records in
549+0 records out
575668224 bytes (576 MB, 549 MiB) copied, 68.3918 s, 8.4 MB/s

2. Demonstration

2.1 First View

You’ll be automatically led to the following two pages after direct reboot. The ONLY thing you need to do is to connect your Raspberry Pi (already installed with LibreELEC) with HDMI cable and connect to your Smart TV.

LibreELEC Welcome	LibreELEC Kodi

2.2 Enable SSH and Allow HTTP

Same as OctoPi discussed in my previous blog OctoPrint on A Raspberry Pi, LibreELEC's Wi-Fi is NOT started by default. Therefore, we’ll have to access LibreELEC via Wired connection for the FIRST time.

In fact, you’ll even have to enable SSH from within LibreELEC smart TV configuration before you’re able to SSH into LibreELEC on the Raspberry Pi. Better allow HTTP at the same time. As follows:

LibreELEC Enable SSH	LibreELEC Allow HTTP

2.3 `ssh` Into LibreELEC via Wired Connection

$ ssh root@192.168.1.75
root@192.168.1.75's password: 
##############################################
#                 LibreELEC                  #
#            https://libreelec.tv            #
##############################################

LibreELEC (official): 12.0.0 (RPi4.aarch64)

Now, you’re able to SSH into LibreELEC through Wired Connection.

2.4 View LibreELEC Over HTTP

And, you’re able to browse content of LibreELEC via HTTP remotely.

2.5 Configure Wi-Fi

You’ll have to use the command connmanctl.

connmanctl How?	connmanctl Outcome

2.5.1 Passphrase is Just Your Wifi Password!!

connmanctl> connect wifi_<my_network_service_id>
Agent RequestInput wifi_<my_network_service_id>
  Passphrase = [ Type=psk, Requirement=mandatory ]
Passphrase?

When you meet the above question (Passphrase?), please input your Wifi password.

2.5.2 Important Commands

You may use the following commands from time to time during Wifi configuration.

systemctl status connman
systemctl restart connman
journalctl -u connman
connmanctl state
connmanctl services
etc.

2.5.3 LibreELEC Configuration

And, you should be able to doublecheck your Wifi configuration via LibreELEC Configuration as:

LibreELEC Configuration	Connections Wifi

Now, you’re good to go. You can put your videos, pictures, etc. under respective folder of LibreELEC as a real media center.

3. Upgrade LibreELEC

3.1 Check Current Version

LibreELEC:~ # cat /etc/os-release
NAME="LibreELEC"
VERSION="12.0.0"
ID="libreelec"
VERSION_ID="12.0"
PRETTY_NAME="LibreELEC (official): 12.0.0"
HOME_URL="https://libreelec.tv"
BUG_REPORT_URL="https://github.com/LibreELEC/LibreELEC.tv"
BUILD_ID="451387894ba86fd65079fcba21f5b30392f6435d"
LIBREELEC_ARCH="RPi4.aarch64"
LIBREELEC_BUILD="official"
LIBREELEC_PROJECT="RPi"
LIBREELEC_DEVICE="RPi4"

3.2 LibreELEC For Raspberry Pi 4

3.2.1 Two Source of LibreELEC Builds

I'll upgrade my current LibreELEC from 12.0.0 to 12.0.1.

3.2.2 Upgrade

The ONLY thing you need to do is to put LibreELEC-RPi4.aarch64-12.0.1.img.gz under folder /storage/.update. And then, reboot.

➜  ~ ssh root@192.168.1.74
root@192.168.1.74's password: 
##############################################
#                 LibreELEC                  #
#            https://libreelec.tv            #
##############################################

LibreELEC (official): 12.0.1 (RPi4.aarch64)

and

LibreELEC:~ # cat /etc/os-release 
NAME="LibreELEC"
VERSION="12.0.1"
ID="libreelec"
VERSION_ID="12.0"
PRETTY_NAME="LibreELEC (official): 12.0.1"
HOME_URL="https://libreelec.tv"
BUG_REPORT_URL="https://github.com/LibreELEC/LibreELEC.tv"
BUILD_ID="7e0b15d49ae7beb086bd2fd00fbf5d107aa9f251"
LIBREELEC_ARCH="RPi4.aarch64"
LIBREELEC_BUILD="official"
LIBREELEC_PROJECT="RPi"
LIBREELEC_DEVICE="RPi4"

OctoPrint on A Raspberry Pi

Posted on 2024-07-25 Edited on 2024-10-29 In Operating System , Raspberry Pi

Finally, I got some time for OctoPrint, another specific operating system for Raspberry Pi, as Home Assistant talked in my previous blog Home Assistant on A Raspberry Pi.

1. Overview

1.1 What is OctoPrint?

From its official website, clearly, OctoPrint is an operating system for 3D Printing. Its capabilities can be extended to other manufacturing fields. Specifically, OctoPi will work on a Raspberry Pi.

1.2 Flash OctoPi Onto Raspberry Pi

$ sudo dd if=./octopi-1.0.0-1.10.2-20240618101629.img of=/dev/sdb bs=1M status=progress conv=fsync 
[sudo] password for lvision: 
2357198848 bytes (2.4 GB, 2.2 GiB) copied, 2 s, 1.2 GB/s2906653696 bytes (2.9 GB, 2.7 GiB) copied, 2.4707 s, 1.2 GB/s

2772+1 records in
2772+1 records out
2906653696 bytes (2.9 GB, 2.7 GiB) copied, 248.633 s, 11.7 MB/s

2. `ssh` Into OctoPi

You got to login OctoPi locally for the FIRST time, instead of remotely, for the NetworkManager service is NOT started by default. Namely, there is NO Wi-Fi by default. Well, you can ALWAYS use the wired connection of course.

2.1.1 Start NetworkManager Whenever Reboot

Use the following commands:

sudo systemctl status NetworkManager
sudo systemctl enable NetworkManager
sudo systemctl start NetworkManager
sudo systemctl status NetworkManager for double check
sudo reboot reboot to ensure NetworkManager is started by default
sudo systemctl status NetworkManager check again

2.1.2 nmcli to Set Up Wi-Fi

Use the following commands:

nmcli dev wifi list
nmcli dev wifi connect NetworkName password WiFiPassword (You don't need to quote or double-quote NetworkName or WiFiPassword)

Now, you’re good to go.

2.2 `ssh` Into OctoPi Via Wi-Fi Connection

$ ssh lvision@192.168.1.76
lvision@192.168.1.76's password: 
Linux octopi 6.1.21-v7+ #1642 SMP Mon Apr  3 17:20:52 BST 2023 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Jul 25 20:23:51 2024 from 192.168.1.1
➜  ~ neofetch
  `.::///+:/-.        --///+//-:``    lvision@octopi 
 `+oooooooooooo:   `+oooooooooooo:    -------------- 
  /oooo++//ooooo:  ooooo+//+ooooo.    OS: Raspbian GNU/Linux 11 (bullseye) armv7l 
  `+ooooooo:-:oo-  +o+::/ooooooo:     Host: Raspberry Pi 3 Model B Rev 1.2 
   `:oooooooo+``    `.oooooooo+-      Kernel: 6.1.21-v7+ 
     `:++ooo/.        :+ooo+/.`       Uptime: 5 mins 
        ...`  `.----.` ``..           Packages: 816 (dpkg) 
     .::::-``:::::::::.`-:::-`        Shell: zsh 5.8 
    -:::-`   .:::::::-`  `-:::-       Terminal: /dev/pts/0 
   `::.  `.--.`  `` `.---.``.::`      CPU: BCM2835 (4) @ 1.200GHz 
       .::::::::`  -::::::::` `       Memory: 111MiB / 870MiB 
 .::` .:::::::::- `::::::::::``::.
-:::` ::::::::::.  ::::::::::.`:::-                           
::::  -::::::::.   `-::::::::  ::::                           
-::-   .-:::-.``....``.-::-.   -::-
 .. ``       .::::::::.     `..`..
   -:::-`   -::::::::::`  .:::::`
   :::::::` -::::::::::` :::::::.
   .:::::::  -::::::::. ::::::::
    `-:::::`   ..--.`   ::::::.
      `...`  `...--..`  `...`
            .::::::::::
             `.-::::-`

3. OctoPrint on OctoPi

octopi.local Setup Wizard	OctoPrint 3D Printer

OctoPrint Server Reload	OctoPrint Webcam

4. LaserGRBL and LightBurn

Kinect 2 on Jetson AGX Xavier

Posted on 2024-07-08 Edited on 2024-10-20 In Machine Vision , RGBD

Today, I wanna try out the old Kinect 2 on my old Jetson AGX Xavier.

1. My Working Environment

For how I set up my old Jetson AGX Xavier, please refer to my previous blog NVidia Jetson Xavier AGX.

2. Kinect 2

2.1 Overview

2.2 Hardware Specification

You can find specs of Kinect v2 in Evaluating the Accuracy of the Azure Kinect and Kinect v2.

3. `lsusb` and `add udev rules`

3.1 `lsusb`

$ lsusb | grep Microsoft
Bus 009 Device 015: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor      
Bus 010 Device 005: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor      
Bus 010 Device 006: ID 045e:02c4 Microsoft Corp. Xbox NUI Sensor

3.2 `lsusb` More Details

➜  ~ lsusb -v -d 045e:02c4

Bus 002 Device 015: ID 045e:02c4 Microsoft Corp. 
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               3.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0         9
  idVendor           0x045e Microsoft Corp.
  idProduct          0x02c4 
  bcdDevice            1.00
  iManufacturer           1 Microsoft
  iProduct                2 Xbox NUI Sensor
  iSerial                 4 178602434347
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x00d6
    bNumInterfaces          4
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower              144mA
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         0
      bInterfaceCount         2
      bFunctionClass        255 Vendor Specific Class
      bFunctionSubClass     255 Vendor Specific Subclass
      bFunctionProtocol       0 
      iFunction               0 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           4
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst               7
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst               7
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst               7
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes           19
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Feedback
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval              14
        bMaxBurst               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x84  EP 4 IN
        bmAttributes            1
          Transfer Type            Isochronous
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               1
        bMaxBurst              10
        Mult                    2
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         2
      bInterfaceCount         2
      bFunctionClass          1 Audio
      bFunctionSubClass       2 Streaming
      bFunctionProtocol       0 
      iFunction               0 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      1 Control Device
      bInterfaceProtocol      0 
      iInterface              0 
      AudioControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdADC               1.00
        wTotalLength       0x001e
        bInCollection           1
        baInterfaceNr(0)        3
      AudioControl Interface Descriptor:
        bLength                12
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0205 Microphone Array
        bAssocTerminal          0
        bNrChannels             4
        wChannelConfig     0x0000
        iChannelNames           0 
        iTerminal               0 
      AudioControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID             3
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bSourceID               1
        iTerminal               0 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        3
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol      0 
      iInterface              0 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        3
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol      0 
      iInterface              0 
      AudioStreaming Interface Descriptor:
        bLength                 7
        bDescriptorType        36
        bDescriptorSubtype      1 (AS_GENERAL)
        bTerminalLink           3
        bDelay                  1 frames
        wFormatTag         0x0001 PCM
      AudioStreaming Interface Descriptor:
        bLength                11
        bDescriptorType        36
        bDescriptorSubtype      2 (FORMAT_TYPE)
        bFormatType             1 (FORMAT_TYPE_I)
        bNrChannels             4
        bSubframeSize           4
        bBitResolution         32
        bSamFreqType            1 Discrete
        tSamFreq[ 0]        16000
      Endpoint Descriptor:
        bLength                 9
        bDescriptorType         5
        bEndpointAddress     0x85  EP 5 IN
        bmAttributes            1
          Transfer Type            Isochronous
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0100  1x 256 bytes
        bInterval               4
        bRefresh                0
        bSynchAddress           0
        bMaxBurst               0
        AudioStreaming Endpoint Descriptor:
          bLength                 7
          bDescriptorType        37
          bDescriptorSubtype      1 (EP_GENERAL)
          bmAttributes         0x00
          bLockDelayUnits         0 Undefined
          wLockDelay         0x0000
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength       0x0016
  bNumDeviceCaps          2
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x000c
      Device can operate at High Speed (480Mbps)
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   2
      Lowest fully-functional device speed is High Speed (480Mbps)
    bU1DevExitLat          10 micro seconds
    bU2DevExitLat        2047 micro seconds
  USB 2.0 Extension Device Capability:
    bLength                 7
    bDescriptorType        16
    bDevCapabilityType      2
    bmAttributes   0x00000002
      HIRD Link Power Management (LPM) Supported
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x0001
  Self Powered

➜  ~ lsusb -v -d 045e:02d9 

Bus 002 Device 014: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor      
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               3.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 
  bDeviceProtocol         3 
  bMaxPacketSize0         9
  idVendor           0x045e Microsoft Corp.
  idProduct          0x02d9 
  bcdDevice            0.73
  iManufacturer           1 Microsoft Corporation  
  iProduct                2 NuiSensor Adaptor      
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x001f
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes           19
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Feedback
        wMaxPacketSize     0x0002  1x 2 bytes
        bInterval               8
        bMaxBurst               0
Hub Descriptor:
  bLength              12
  bDescriptorType      42
  nNbrPorts             1
  wHubCharacteristic 0x000d
    Per-port power switching
    Compound device
    Per-port overcurrent protection
  bPwrOn2PwrGood      100 * 2 milli seconds
  bHubContrCurrent      0 milli Ampere
  bHubDecLat          0.4 micro seconds
  wHubDelay          4004 nano seconds
  DeviceRemovable    0x00
 Hub Port Status:
   Port 1: 0000.0203 5Gbps power U0 enable connect
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength       0x002a
  bNumDeviceCaps          3
  USB 2.0 Extension Device Capability:
    bLength                 7
    bDescriptorType        16
    bDevCapabilityType      2
    bmAttributes   0x00000002
      HIRD Link Power Management (LPM) Supported
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x000e
      Device can operate at Full Speed (12Mbps)
      Device can operate at High Speed (480Mbps)
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   1
      Lowest fully-functional device speed is Full Speed (12Mbps)
    bU1DevExitLat           4 micro seconds
    bU2DevExitLat         231 micro seconds
  Container ID Device Capability:
    bLength                20
    bDescriptorType        16
    bDevCapabilityType      4
    bReserved               0
    ContainerID             {30eef35c-07d5-2549-b001-802d79434c30}
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x000d
  Self Powered
  U1 Enabled
  U2 Enabled

Bus 001 Device 008: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor      
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.10
  bDeviceClass            9 Hub
  bDeviceSubClass         0 
  bDeviceProtocol         1 Single TT
  bMaxPacketSize0        64
  idVendor           0x045e Microsoft Corp.
  idProduct          0x02d9 
  bcdDevice            0.70
  iManufacturer           1 Microsoft Corporation  
  iProduct                2 NuiSensor Adaptor      
  iSerial                 0 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0019
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 Full speed (or root) hub
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             1
  wHubCharacteristic 0x00ed
    Per-port power switching
    Compound device
    Per-port overcurrent protection
    TT think time 32 FS bits
    Port indicators
  bPwrOn2PwrGood       50 * 2 milli seconds
  bHubContrCurrent    100 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0100 power
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength       0x002a
  bNumDeviceCaps          3
  USB 2.0 Extension Device Capability:
    bLength                 7
    bDescriptorType        16
    bDevCapabilityType      2
    bmAttributes   0x00000002
      HIRD Link Power Management (LPM) Supported
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x000e
      Device can operate at Full Speed (12Mbps)
      Device can operate at High Speed (480Mbps)
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   1
      Lowest fully-functional device speed is Full Speed (12Mbps)
    bU1DevExitLat           4 micro seconds
    bU2DevExitLat         231 micro seconds
  Container ID Device Capability:
    bLength                20
    bDescriptorType        16
    bDevCapabilityType      4
    bReserved               0
    ContainerID             {30eef35c-07d5-2549-b001-802d79434c30}
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x0001
  Self Powered

3.3 Add `udev rules` Under `/etc/udev/rules.d`

Take a look at OpenKinect libfreenect2 90-kinect2.rules:

$ cat platform/linux/udev/90-kinect2.rules            
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: platform/linux/udev/90-kinect2.rules
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ # this file belongs in /etc/udev/rules.d/
   2   │ # ATTR{product}=="Kinect2"
   3   │ SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02c4", MODE="0666"
   4   │ SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02d8", MODE="0666"
   5   │ SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02d9", MODE="0666"
───────┴───────────────────────────────────────────────────────────────────────────────────

We’ll do the following 2 steps:

step 1:

1	$ sudo cp platform/linux/udev/90-kinect2.rules /etc/udev/rules.d

step 2:

1 2	$ sudo udevadm control --reload-rules $ sudo udevadm trigger

3.4 Don’t Use Default OpenNI2 from Default Repository

3.4.1 Incorrect Rules

Take a look at OpenNI2 Kinect2 primesense-usb.rules:

$ cat Packaging/Linux/primesense-usb.rules               
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: Packaging/Linux/primesense-usb.rules
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ # Make primesense device mount with writing permissions (default is read only for unknown devices)
   2   │ SUBSYSTEM=="usb", ATTR{idProduct}=="0200", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
   3   │ SUBSYSTEM=="usb", ATTR{idProduct}=="0300", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
   4   │ SUBSYSTEM=="usb", ATTR{idProduct}=="0401", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
   5   │ SUBSYSTEM=="usb", ATTR{idProduct}=="0500", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
   6   │ SUBSYSTEM=="usb", ATTR{idProduct}=="0600", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
   7   │ SUBSYSTEM=="usb", ATTR{idProduct}=="0601", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
   8   │ SUBSYSTEM=="usb", ATTR{idProduct}=="0609", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
   9   │ SUBSYSTEM=="usb", ATTR{idProduct}=="1250", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
  10   │ SUBSYSTEM=="usb", ATTR{idProduct}=="1260", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
  11   │ SUBSYSTEM=="usb", ATTR{idProduct}=="1270", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
  12   │ SUBSYSTEM=="usb", ATTR{idProduct}=="1280", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
  13   │ SUBSYSTEM=="usb", ATTR{idProduct}=="1290", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
  14   │ SUBSYSTEM=="usb", ATTR{idProduct}=="f9db", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"

!Note: by installing libopenni2-dev and openni2-utils from default repo, you’re going to have the above rule primesense-usb.rules installed under /usr/lib/udev/rules.d, instead of /etc/udev/rules.d.

3.4.2 Incorrect NiViewer

Even if package openni2-utils installed an executable NiViewer2 under /usr/bin, it fails to even detect the device Kinect v2.

$ NiViewer2
openDevice failed:
	DeviceOpen using default: no devices found

Press any key to continue . . .

Due to the above Incorrect Rules, this NiViewer2, of course, should NOT detect the device.

Feel so disappointed …

😞

What made me feel worse, even desperate???

The above failure happens on both of my Jetson AGX Xavier running Ubuntu 20.04.6 LTS aarch64, and my desktop running Ubuntu 24.04.

4. For Windows

Please strictly follow Kinect for Windows SDK 2.0. For my Kinect 2, I got the following outputs on my laptop running Windows 11.

5. For Linux

5.1 Demonstration FIRST

OpenNI2 NiViewer

libfreenect2 Protonect CPU

libfreenect2 Protonect -GPU=0

5.2 Build OpenNI2 Github from Source

5.2.1 Key Components

Components	Description
PSCommon	The third-party library directly from PrimeSense
OniFile	Must be dealing with OpenNI's `.oni` files
PS1080	Clearly related to PrimeSense
PSLink	Clearly related to PrimeSense
Kinect	I believe this is the driver for original Kinect v1, namely, Kinect for Xbox 360.
Kinect2	I believe this is the driver for Kinect2, based on Kinect for Windows SDK 2.0

To build Kinect or Kinect2, I think a Windows environment is a MUST, which is NOT for me.

5.2.2 Tool NiViewer

Driver Kinect2 in the above is a MUST for OpenNI2 Github's application NiViewer.

5.2.3 Any Driver to Drive Kinect2 and Can Work With OpenNI2 Github?

Now, stucked. Under Linux, how to build a driver working for my Kinect v2?

Solution: Build libfreenect2 from source, with BUILD_OPENNI2_DRIVER ON.

5.3 Build libfreenect2 from Source

Download OpenNI2 and save it, and built libfreenect2 with BUILD_OPENNI2_DRIVER set ON.

By doing so, you will have the following 2 variables set by default as:

1 2	OpenNI2_INCLUDE_DIRS /opt/OpenNI/Include OpenNI2_LIBRARY /opt/OpenNI/Redist/libOpenNI2.so

and you will have the following two libraries built out:

1 2	lib/libfreenect2-openni2.so lib/libfreenect2.so

And, if needed, after the installation of libfreenect2, manually copy libfreenect2-openni2.so under folder /usr/local/lib/OpenNI2/Drivers. Now, ALL set. You’re good to go NiViewer.

5.4 Protonect or ProtonectSR from libfreenect2

5.4.1 `./bin/Protonect cuda`

I got NO idea why ./bin/Protonect cuda failed to run ALL the time. Then, I created a simple Python script verify_cuda.py and ALWAYS meet the following error:

$ python verify_cuda.py   
[Info] [CudaDepthPacketProcessorImpl] device 0: NVIDIA GeForce RTX 3090 @ 1725MHz Memory 24219MB
[Info] [CudaDepthPacketProcessorImpl] selected device 0
[Error] [CudaDepthPacketProcessorImpl] cudaGetLastError(): the provided PTX was compiled with an unsupported toolchain.
CudaPacketPipeline is available.

Finally, I found the reason: Cuda version is NOT synced with NVidia driver version. More specifically, in my case:

NVida Driver Version: NVidia Official Driver 550.100, which is built and compatible with Cuda 12.4
Cuda Version: cuda_12.5.1_555.42.06_linux.run. Clearly: Cuda 12.5.1 comes with a newer NVidia driver version 555.42

To solve this problem, install NVidia driver during the installation of Cuda. Namely: when run cuda_12.5.1_555.42.06_linux.run, tick

1 2	- [X] Driver │ │ [X] 555.42.06

You may have to reboot into text mode and have all previously-installed NVidia driver unloaded. For this issue, please refer to How to unload kernel module ‘nvidia-drm’?
The key: sudo systemctl isolate multi-user.target

5.4.2 `./bin/Protonect -gpu=0`

$ ./bin/Protonect -gpu=0
Version: 0.2.0
Environment variables: LOGFILE=<protonect.log>
Usage: ./bin/Protonect [-gpu=<id>] [gl | cl | clkde | cuda | cudakde | cpu] [<device serial>]
        [-noviewer] [-norgb | -nodepth] [-help] [-version]
        [-frames <number of frames to process>]
To pause and unpause: pkill -USR1 Protonect
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 23 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @10:5 with serial 178602434347
[Info] [Freenect2Impl] found 1 devices
libva info: VA-API version 1.20.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nvidia_drv_video.so
libva info: va_openDriver() returns -1
[Error] [VaapiRgbPacketProcessorImpl] vaInitialize(display, &major_ver, &minor_ver): unknown libva error
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 60*8*33792
[Info] [Freenect2DeviceImpl] opened
[Info] [Freenect2DeviceImpl] starting...
[Debug] [Freenect2DeviceImpl] status 0x090000: 9855
[Debug] [Freenect2DeviceImpl] status 0x090000: 9855
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
device serial: 178602434347
device firmware: 4.0.3916.0
[Debug] [DepthPacketStreamParser] not all subsequences received 0
[Debug] [DepthPacketStreamParser] not all subsequences received 1021
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [DepthPacketStreamParser] 2 packets were lost
[Info] [DepthPacketStreamParser] 7 packets were lost
[Info] [OpenGLDepthPacketProcessor] avg. time: 5.12134ms -> ~195.261Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 4.06057ms -> ~246.271Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 19.003ms -> ~52.6232Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 4.01312ms -> ~249.182Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 4.20707ms -> ~237.695Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 17.376ms -> ~57.5507Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 3.95256ms -> ~253.001Hz
[2]  - 2620271 killed     ./bin/ProtonectSR cuda
[Info] [OpenGLDepthPacketProcessor] avg. time: 4.40171ms -> ~227.184Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 17.8467ms -> ~56.0326Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 6.23296ms -> ~160.437Hz
[3]  + 2628690 killed     ./bin/Protonect
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [OpenGLDepthPacketProcessor] avg. time: 9.88223ms -> ~101.192Hz
[Info] [DepthPacketStreamParser] 2 packets were lost
[Info] [TurboJpegRgbPacketProcessor] avg. time: 19.2099ms -> ~52.0565Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 5.8084ms -> ~172.164Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 8.7243ms -> ~114.622Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 18.2581ms -> ~54.7703Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 9.24396ms -> ~108.179Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 10.1021ms -> ~98.9893Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 18.3488ms -> ~54.4995Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 8.35109ms -> ~119.745Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 5.61051ms -> ~178.237Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 17.9773ms -> ~55.6256Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 6.80606ms -> ~146.928Hz
[Info] [Freenect2DeviceImpl] stopping...
[Info] [Freenect2DeviceImpl] canceling rgb transfers...
[Info] [Freenect2DeviceImpl] canceling depth transfers...
[Info] [Freenect2DeviceImpl] stopped
[Info] [Freenect2DeviceImpl] closing...
[Info] [Freenect2DeviceImpl] releasing usb interfaces...
[Info] [Freenect2DeviceImpl] deallocating usb transfer pools...
[Info] [Freenect2DeviceImpl] closing usb device...
[Info] [Freenect2DeviceImpl] closed
[Info] [Freenect2DeviceImpl] closing...
[Info] [Freenect2DeviceImpl] already closed, doing nothing
[TurboJpegRgbPacketProcessor] 9.8382 10.0887 18.9733 20.2907 26.1092 mean=18.2873 std=2.86968 n=749
[OpenGLDepthPacketProcessor] 2.90518 3.87679 5.69045 12.5144 95.8873 mean=6.46404 std=4.29096 n=1528

5.4.3 `./bin/Protonect cpu`

$ ./bin/Protonect cpu
Version: 0.2.0
Environment variables: LOGFILE=<protonect.log>
Usage: ./bin/Protonect [-gpu=<id>] [gl | cl | clkde | cuda | cudakde | cpu] [<device serial>]
        [-noviewer] [-norgb | -nodepth] [-help] [-version]
        [-frames <number of frames to process>]
To pause and unpause: pkill -USR1 Protonect
libva info: VA-API version 1.20.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nvidia_drv_video.so
libva info: va_openDriver() returns -1
[Error] [VaapiRgbPacketProcessorImpl] vaInitialize(display, &major_ver, &minor_ver): unknown libva error
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 23 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @10:5 with serial 178602434347
[Info] [Freenect2Impl] found 1 devices
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 60*8*33792
[Info] [Freenect2DeviceImpl] opened
[Info] [Freenect2DeviceImpl] starting...
[Debug] [Freenect2DeviceImpl] status 0x090000: 9729
[Debug] [Freenect2DeviceImpl] status 0x090000: 9731
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
device serial: 178602434347
device firmware: 4.0.3916.0
[Debug] [DepthPacketStreamParser] not all subsequences received 0
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [DepthPacketStreamParser] 30 packets were lost
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [DepthPacketStreamParser] 30 packets were lost
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [Freenect2DeviceImpl] stopping...
[Info] [Freenect2DeviceImpl] canceling rgb transfers...
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [Freenect2DeviceImpl] canceling depth transfers...
[Info] [Freenect2DeviceImpl] stopped
[Info] [Freenect2DeviceImpl] closing...
[Info] [Freenect2DeviceImpl] releasing usb interfaces...
[Info] [Freenect2DeviceImpl] deallocating usb transfer pools...
[Info] [Freenect2DeviceImpl] closing usb device...
[Info] [Freenect2DeviceImpl] closed
[Info] [Freenect2DeviceImpl] closing...
[Info] [Freenect2DeviceImpl] already closed, doing nothing
[TurboJpegRgbPacketProcessor] 10.0384 10.1629 18.9524 20.9346 27.5923 mean=18.5922 std=3.23163 n=44
[CpuDepthPacketProcessor] 101.371 101.485 102.303 109.367 111.332 mean=102.829 std=2.25233 n=27

5.4.4 Conclusive Comparisons

Let’s do a simple comparison of the speed using 3 different parameters: cuda, -cpu=0, cpu.

Commands	RGB Mean Time (ms)	Depth Mean Time (ms)
`./bin/Protonect cuda`	18.8951	0.521472
`./bin/Protonect -gpu=0`	18.2873	6.46404
`./bin/Protonect cpu`	18.5922	102.829

The conclusion is quite obvious:

for depth: CudaDepthPacketProcessor > OpenGLDepthPacketProcessor > CpuDepthPacketProcessor.
for RGB: Always TurboJpegRgbPacketProcessor

5.5 pylibfreenect2 - libfreenect2's Python Wrapper

So far, in my personal point of view, pylibfreenect2 is the BEST Python wrapper of libfreenect2. But, according to the official documentation of libfreenect2, it’s clearly written:

Audio. Raw audio is accessible via Linux USB audio. There is no support for the calibrated directional audio.

5.6 Kinect2's Audio pyalsaaudio

5.6.1 `lsusb` Again

➜  ~ lsusb | grep Microsoft 
Bus 002 Device 003: ID 045e:02c4 Microsoft Corp. 
Bus 002 Device 002: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor      
Bus 001 Device 002: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor

5.6.2 `arecord -l`

1 2	➜ ~ arecord -l \| grep NUI card 1: Sensor [Xbox NUI Sensor], device 0: USB Audio [USB Audio]

Card number: 1
Device number: 0

5.6.3 List Details of `hw:1,0`

➜  arecord -D hw:1,0 --dump-hw-params
Recording WAVE 'stdin' : Unsigned 8 bit, Rate 8000 Hz, Mono
HW Params of device "hw:1,0":
--------------------
ACCESS:  MMAP_INTERLEAVED RW_INTERLEAVED
FORMAT:  S32_LE
SUBFORMAT:  STD
SAMPLE_BITS: 32
FRAME_BITS: 128
CHANNELS: 4
RATE: 16000
PERIOD_TIME: [1000 2048000]
PERIOD_SIZE: [16 32768]
PERIOD_BYTES: [256 524288]
PERIODS: [2 1024]
BUFFER_TIME: [2000 4096000]
BUFFER_SIZE: [32 65536]
BUFFER_BYTES: [512 1048576]
TICK_TIME: ALL
--------------------
arecord: set_params:1368: Sample format non available
Available formats:
- S32_LE

A couple of concerns:

I personally think FRAME_BITS is a typo. It should be FRAME_SIZE.
What the sample rate is 16000, but NOT 44100? 😀

5.6.4 Demonstration

import alsaaudio
import wave
import time

# Open the audio device
inp = alsaaudio.PCM(alsaaudio.PCM_CAPTURE, alsaaudio.PCM_NORMAL, "plughw:Sensor")

# Set audio parameters
inp.setformat(alsaaudio.PCM_FORMAT_S32_LE)
inp.setchannels(4)
inp.setrate(16000)
inp.setperiodsize(128)

# Prepare a file to write the audio data
with wave.open("test.wav", "wb") as f:
    f.setnchannels(4)
    f.setsampwidth(4)
    f.setframerate(16000)

    # Record audio for 10 seconds
    timeout = 10  # seconds
    start_time = time.time()
    while time.time() - start_time < timeout:
        num_bytes, buffer = inp.read()
        f.writeframes(buffer)

# Close the audio device
inp.close()

print("Recording finished")

6. For Jetson AGX Xavier

Now, we’re done on my desktop. Let’s try Kinect2 on Jetson AGX Xavier.

6.1 libfreenect2 - Do NOT Enable `ENABLE_TEGRAJPEG`

By default, libfreenect2 has ENABLE_TEGRAJPEG set ON, which builds fine. However, the followig issue has been reported for over 8 years.

Solution for Jetson AGX Xavier:

NEVER turn on ENABLE_TEGRAJPEG on Jetson AGX Xavier
Jetson AGX Xavier is working fine with TurboJPEG

libfreenect2 Protonect Cuda

libfreenect2 Protonect Cuda Speed

From the above output, we can easily come up with the following table:

Attribute	RGB	Depth
Resolution	1920x1080	512x424
Number of Channels	3 (Red, Green, Blue)	1 (Depth)
Bytes per Pixel	3 bytes	2 bytes
Average Processing Time	~22.4048 ms	~7.18678 ms
FPS	~44.6163 FPS	~110.134 FPS
Data per Frame (Bytes)	1920 * 1080 * 3 = 6,220,800 bytes	512 * 424 * 2 = 434,176 bytes
Data per Frame (MB)	~6.22 MB	~0.414 MB
Data per Second (MB/s)	6.22 MB * 44.6163 ≈ 277.57 MB/s	0.414 MB * 110.134 ≈ 45.60 MB/s

Total Data per Second: 277.57 MB + 45.60 MB ≈ 323.17 MB

6.2 Integrate pylibfreenect2 and pyalsaaudio with GStreamer for 3 Streams from Kinect2: `depth`, `rgb`, and `audio`

Sorry my friends. I’m NOT going to disclose my code, in Python.

6.2.1 Server Side

Kinect2 RTSP Server

Kinect2 RTSP Server Stream Responses

6.2.2 Client Side

RGB Stream

Depth Stream

Audio Stream

NVidia Jetson Xavier AGX

Posted on 2024-07-06 Edited on 2024-10-22 In Single Board Computers , ARM

I got one Jetson AGX Xavier unused for quite long. There are a couple of reasons prohibiting me from using it.

The price. I couldn’t even come up with JUST ONE application that has to use Jetson Embedded Systems without cheaper but qualified replacement.
The host Ubuntu that NVidia SDK Manager supports is ALWAYS one version behind. Which means: I can NEVER run NVidia SDK Manager on my host desktop.

1. My Working Environment

➜  ~ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 24.04.1 LTS
Release:	24.04
Codename:	noble

➜  ~ uname -r
6.8.0-47-generic

2. Jetson Embedded Products

2.1 GPU Compute Capability

Jetson Model	Compute Capability
Jetson Orin Series including: Jetson AGX Orin, Jetson Orin NX, Jetson Orin Nano	8.7
Jetson Xavier Series including: Jetson AGX Xavier, Jetson Xavier NX	7.2
Jetson TX2	6.2
Jetson Nano including: default Jetson Nano 4GB, Jetson Nano 2GB	5.3

2.2 JetPack Version Compatability

Please just refer to JetPack Archive.

2.2.1 Jetson Nano Series

Specification	Jetson Nano 4GB	Jetson Nano 2GB
JetPack Versions	4.2 ~ 4.6.5	4.4 ~ 4.6.5
CUDA Cores	128	128
Compute Capability	5.3	5.3
GPU Architecture	Maxwell	Maxwell
Memory	4GB LPDDR4	2GB LPDDR4
AI Performance	472 GFLOPs	472 GFLOPs

2.2.2 Jetson Xavier Series

Specification	Jetson AGX Xavier 32GB	Jetson AGX Xavier 8GB
JetPack Versions	4.2 ~ 5.1.4	4.4 ~ 5.1.4
CUDA Cores	512	512
Compute Capability	7.2	7.2
GPU Architecture	Volta	Volta
Memory	32GB LPDDR4x	8GB LPDDR4x
AI Performance	11 TFLOPs (FP16)	11 TFLOPs (FP16)

2.2.3 Jetson Orin Series

Specification	Jetson AGX Orin 64GB	Jetson AGX Orin 32GB	Jetson Orin NX 16GB	Jetson Orin NX 8GB	Jetson Orin Nano 8GB	Jetson Orin Nano 4GB
JetPack Versions	5.0 Developer Preview - 6.1	5.0 Developer Preview - 6.1	5.0 Developer Preview - 6.1	5.0 Developer Preview - 6.1	5.0 Developer Preview - 6.1	5.0 Developer Preview - 6.1
CUDA Cores	2048	2048	1024	1024	512	512
Compute Capability	8.7	8.7	8.7	8.7	8.7	8.7
GPU Architecture	Ampere	Ampere	Ampere	Ampere	Ampere	Ampere
Memory	64GB LPDDR5	32GB LPDDR5	16GB LPDDR5	8GB LPDDR5	8GB LPDDR5	4GB LPDDR5
AI Performance	up to 200 TOPs	up to 200 TOPs	up to 70 TOPs	up to 70 TOPs	up to 40 TOPs	up to 40 TOPs

2.3 JetPack Archive

2.4 Jetson Download Center

A variety of software can be downloaded from Jetson Download Center.

3. My Jetson Embedded Boards

I actually own 4 Jetson embedded boards and am trying 1 more:

one Jetson Nano 2GB: which is purchased with JetBot
one Jetson Nano: provided by ShenZhen Longer Vision Technology, which now comes with 16G memory and 2 Wired ports.
one Jetson Xavier NX: provided by ShenZhen Longer Vision Technology
one Jetson AGX Xavier: for pure R&D only, but sooooo expensive
one Jetson Orin Nano: borrowed from one of my students. Jetson Orin Series was FIRSTLY on the market in August 2022 (Please refer to: NVIDIA® Jetson Orin™: A Breakthrough in Industrial Edge AI). It is toooo new and toooo expensive to be purchased. Ever since that time, a variety of single board computers with made-in-China NPUs are much much much cheaper than Jetson Orin Series. Yeah, 10 times cheaper guaranteed.

3.1 Jetson Nano 2GB for JetBot

➜  ~ cat /proc/cpuinfo
processor	: 0
model name	: ARMv8 Processor rev 1 (v8l)
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd07
CPU revision	: 1

processor	: 1
model name	: ARMv8 Processor rev 1 (v8l)
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd07
CPU revision	: 1

processor	: 2
model name	: ARMv8 Processor rev 1 (v8l)
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd07
CPU revision	: 1

processor	: 3
model name	: ARMv8 Processor rev 1 (v8l)
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd07
CPU revision	: 1

➜  ~ cat /proc/meminfo
MemTotal:        2027240 kB
MemFree:          304868 kB
MemAvailable:    1619252 kB
Buffers:          301640 kB
Cached:           978212 kB
SwapCached:            0 kB
Active:           591384 kB
Inactive:         838300 kB
Active(anon):      77428 kB
Inactive(anon):   112776 kB
Active(file):     513956 kB
Inactive(file):   725524 kB
Unevictable:         828 kB
Mlocked:               0 kB
SwapTotal:       5207916 kB
SwapFree:        5207796 kB
Dirty:               272 kB
Writeback:             0 kB
AnonPages:        143560 kB
Mapped:           168900 kB
Shmem:             39548 kB
Slab:             217244 kB
SReclaimable:     162968 kB
SUnreclaim:        54276 kB
KernelStack:        4384 kB
PageTables:         3520 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     6221536 kB
Committed_AS:    1029676 kB
VmallocTotal:   263061440 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:     40960 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
NvMapMemFree:       2048 kB
NvMapMemUsed:      21324 kB
CmaTotal:          65536 kB
CmaFree:           13236 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

➜  ~ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   58G   29G   28G  51% /
none            951M     0  951M   0% /dev
tmpfs           990M  4.0K  990M   1% /dev/shm
tmpfs           990M   38M  953M   4% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           990M     0  990M   0% /sys/fs/cgroup
tmpfs           198M  4.0K  198M   1% /run/user/121
tmpfs           198M     0  198M   0% /run/user/1000

3.2 Jetson Nano from ShenZhen Longer Vision Technology

➜  ~ cat /proc/cpuinfo
processor	: 0
model name	: ARMv8 Processor rev 1 (v8l)
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd07
CPU revision	: 1

processor	: 1
model name	: ARMv8 Processor rev 1 (v8l)
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd07
CPU revision	: 1

processor	: 2
model name	: ARMv8 Processor rev 1 (v8l)
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd07
CPU revision	: 1

processor	: 3
model name	: ARMv8 Processor rev 1 (v8l)
BogoMIPS	: 38.40
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer	: 0x41
CPU architecture: 8
CPU variant	: 0x1
CPU part	: 0xd07
CPU revision	: 1

➜  ~ cat /proc/meminfo
MemTotal:        4059260 kB
MemFree:         1409344 kB
MemAvailable:    2538768 kB
Buffers:           30120 kB
Cached:          1254004 kB
SwapCached:            0 kB
Active:          1017736 kB
Inactive:        1170840 kB
Active(anon):     905764 kB
Inactive(anon):    45076 kB
Active(file):     111972 kB
Inactive(file):  1125764 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       2029616 kB
SwapFree:        2029616 kB
Dirty:               152 kB
Writeback:             0 kB
AnonPages:        898652 kB
Mapped:           560276 kB
Shmem:             46388 kB
Slab:             102792 kB
SReclaimable:      46832 kB
SUnreclaim:        55960 kB
KernelStack:        6160 kB
PageTables:         7752 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     4059244 kB
Committed_AS:    2500320 kB
VmallocTotal:   263061440 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:    131072 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
NvMapMemFree:          0 kB
NvMapMemUsed:     258692 kB
CmaTotal:         475136 kB
CmaFree:          441280 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

➜  ~ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   14G   12G  1.2G  92% /
none            1.8G     0  1.8G   0% /dev
tmpfs           2.0G   19M  2.0G   1% /dev/shm
tmpfs           2.0G   28M  2.0G   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs           397M   12K  397M   1% /run/user/120
tmpfs           397M     0  397M   0% /run/user/1000

3.3 Jetson Xavier NX from ShenZhen Longer Vision Technology

➜  ~ cat /proc/cpuinfo
processor	: 0
model name	: ARMv8 Processor rev 0 (v8l)
BogoMIPS	: 62.50
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer	: 0x4e
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x004
CPU revision	: 0
MTS version	: 50168445

processor	: 1
model name	: ARMv8 Processor rev 0 (v8l)
BogoMIPS	: 62.50
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer	: 0x4e
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x004
CPU revision	: 0
MTS version	: 50168445

processor	: 2
model name	: ARMv8 Processor rev 0 (v8l)
BogoMIPS	: 62.50
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer	: 0x4e
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x004
CPU revision	: 0
MTS version	: 50168445

processor	: 3
model name	: ARMv8 Processor rev 0 (v8l)
BogoMIPS	: 62.50
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer	: 0x4e
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x004
CPU revision	: 0
MTS version	: 50168445

processor	: 4
model name	: ARMv8 Processor rev 0 (v8l)
BogoMIPS	: 62.50
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer	: 0x4e
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x004
CPU revision	: 0
MTS version	: 50168445

processor	: 5
model name	: ARMv8 Processor rev 0 (v8l)
BogoMIPS	: 62.50
Features	: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer	: 0x4e
CPU architecture: 8
CPU variant	: 0x0
CPU part	: 0x004
CPU revision	: 0
MTS version	: 50168445

➜  ~ cat /proc/meminfo
MemTotal:        7958064 kB
MemFree:         6376676 kB
MemAvailable:    6968608 kB
Buffers:           31648 kB
Cached:           706620 kB
SwapCached:            0 kB
Active:           422156 kB
Inactive:         581804 kB
Active(anon):     267236 kB
Inactive(anon):    27908 kB
Active(file):     154920 kB
Inactive(file):   553896 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       3979008 kB
SwapFree:        3979008 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:        254048 kB
Mapped:           187980 kB
Shmem:             29456 kB
Slab:             132884 kB
SReclaimable:      60592 kB
SUnreclaim:        72292 kB
KernelStack:        6640 kB
PageTables:         5892 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     7958040 kB
Committed_AS:    1977692 kB
VmallocTotal:   263061440 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
AnonHugePages:     90112 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
NvMapMemFree:        704 kB
NvMapMemUsed:      27852 kB
CmaTotal:         753664 kB
CmaFree:          704152 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

➜  ~ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   14G   13G  253M  99% /
none            3.5G     0  3.5G   0% /dev
tmpfs           3.8G  4.0K  3.8G   1% /dev/shm
tmpfs           3.8G   29M  3.8G   1% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           3.8G     0  3.8G   0% /sys/fs/cgroup
tmpfs           778M   12K  778M   1% /run/user/120
tmpfs           778M     0  778M   0% /run/user/1000

3.4 Jetson AGX Xavier

3.4.1 Entering Recovery Mode

After recovery mode is entered, you'll see:

1 2	➜ ~ lsusb \| grep NVIDIA Bus 009 Device 031: ID 0955:7019 NVIDIA Corp. APX

3.4.2 Failed to Flash via NVidia SDK Manager

Select Jetson AGX Xavier	No available releases for host OS: Ubuntu 24.04

3.4.3 Try SD Car Image

Download JP514-xnx-sd-card-image_b11.zip.

$ sudo dd if=./sd-blob.img of=/dev/sdb bs=1M status=progress conv=fsync 
[sudo] password for lvision: 
17746100224 bytes (18 GB, 17 GiB) copied, 26 s, 683 MB/s18153996288 bytes (18 GB, 17 GiB) copied, 26.5595 s, 684 MB/s

17313+0 records in
17313+0 records out
18153996288 bytes (18 GB, 17 GiB) copied, 1617.17 s, 11.2 MB/s

However, Jetson AGX Xavier is NOT booting, but ONLY black screen.

The answer is at NVidia Forum Issue 189348 - Black Screen: Do you know that you are talking about AGX Xavier but not NX? Only NX and Nano have and support sdcard image. Other platforms do not. It’s also clearly clarified in the table on NVidia’s official page: NVidia SDK Manager.

3.4.4 Negotiated to Use Ubuntu 20.04

Still problemabic. Two main issues:

That IP: 192.168.55.1 won't let me flash successfully
CUDA is not able to be installed successfully

3.4.4.2 Again: Jetson Linux Developer Guide Quick Start

1	sudo ./flash.sh jetson-agx-xavier-devkit internal

works for me.

3.4.4.3 `ssh` and `neofetch`

4. Demonstrate GPU On Jetson AGX Xavier

4.1 `tegrastats`, `jetson_release`, `jetson_clocks`

4.2 `jtop` from jetson-stats

5. YOLOv11 On Jetson AGX Xavier

Similar difficulties happened on this board Jetson AGX Xavier, which is clearly elaborated in one of my blogs NVidia-Jetson-Orin-Nano-8G-Dev-Kit.md.

Solution:

pip install ultralytics
Overwrite torch installed with ultralytics by PyTorch for JetPack 5.1.2
Build torchvision from source

YOLOv11 Initialization	YOLOv11 GPU

One of my blogs YOLOv11.md didn't talk about which model to use. And today, let's try them out.

Intel RealSense LiDAR Camera D435/D430

Posted on 2024-07-05 Edited on 2024-10-17 In Machine Vision , RGBD

Today, we’re playing Intel RealSense Depth Camera D435 and Intel RealSense Depth Modules and Processors.

1. My Working Environment

$ lsb_release -a             
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 24.04 LTS
Release:	24.04
Codename:	noble

$ uname -r                             
6.8.0-36-generic

2. Intel RealSense

2.1 Overview

2.2 Hardware Specification

Refer to Tech Specs pdf.

3. `lsusb` and Demonstration

3.1 `lsusb`

Respectively, I got:

1 2	$ lsusb \| grep -i RealSense Bus 010 Device 012: ID 8086:0b07 Intel Corp. RealSense D435

1 2	$ lsusb \| grep -i RealSense Bus 010 Device 004: ID 8086:0ad4 Intel Corp. Intel(R) RealSense(TM) Depth Camera 430

3.2 `realsense-viewer`

3.2.1 Intel RealSense Depth Camera D435

$ realsense-viewer
 05/07 15:16:42,021 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video0
 05/07 15:16:42,099 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video1
 05/07 15:16:42,101 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video2
 05/07 15:16:42,103 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video3
 05/07 15:16:42,104 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video4 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.3/video4linux/video4
 05/07 15:16:42,106 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video5 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.3/video4linux/video5
 05/07 15:16:42,108 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video6 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
 05/07 15:16:42,211 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video7 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video7
 05/07 15:16:42,498 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video0
 05/07 15:16:42,499 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video1
 05/07 15:16:42,500 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video2
 05/07 15:16:42,501 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video3
 05/07 15:16:42,502 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video4 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.3/video4linux/video4
 05/07 15:16:42,503 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video5 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.3/video4linux/video5
 05/07 15:16:42,504 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video6 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
 05/07 15:16:42,505 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video7 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video7
 05/07 15:16:42,506 INFO [135660990636032] (context.cpp:116) ... /sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video0
 05/07 15:16:42,506 INFO [135660990636032] (context.cpp:116) ... /sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
 05/07 15:16:42,506 INFO [135660990636032] (context.cpp:128) Found 2 RealSense devices (0xff requested & 0xff from device-mask in settings)
...

`realsense-viewer` GUI	Firmware Upgrading from within `realsense-viewer` GUI

Viewing Depth Image in 2D	Viewing Depth Image in 3D

3.2.2 Intel RealSense Depth Modules and Processors D430

$ realsense-viewer  
 05/07 17:00:55,958 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video0
 05/07 17:00:56,036 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video1
 05/07 17:00:56,039 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video2
 05/07 17:00:56,041 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video3
 05/07 17:00:56,043 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video6 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
 05/07 17:00:56,146 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video7 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video7
 05/07 17:00:56,387 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video0
 05/07 17:00:56,389 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video1
 05/07 17:00:56,390 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video2
 05/07 17:00:56,391 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video3
 05/07 17:00:56,392 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video6 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
 05/07 17:00:56,393 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video7 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video7
 05/07 17:00:56,394 INFO [133348205735936] (context.cpp:116) ... /sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video0
 05/07 17:00:56,394 INFO [133348205735936] (context.cpp:116) ... /sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
 05/07 17:00:56,394 INFO [133348205735936] (context.cpp:128) Found 2 RealSense devices (0xff requested & 0xff from device-mask in settings)

D430 Depth Camera 1 - 2D IR Sparse Pattern	D430 Depth Camera 1 - 3D

D430 Depth Camera 2 - 2D IR Sparse Pattern	D430 Depth Camera 2 - 3D

Firmware To Upgrade	Firmware Upgraded Successfully

3.3 `rs-hello-realsense`

Azure Kinect DK

Posted on 2024-07-01 Edited on 2024-10-17 In Machine Vision , RGBD

I tried Azure Kinect DK 2 years ago, and rencently I’d love to some fun with it again.

1. My Working Environment

$ lsb_release -a             
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 24.04 LTS
Release:	24.04
Codename:	noble

$ uname -r                             
6.8.0-36-generic

2. Azure Kinect DK

2.1 Overview

2.2 Hardware Specification

Refer to Azure Kinect DK hardware specifications.

3. `lsusb` and Demonstration

[!NOTE:]
Azure Kinect DK is quite picky at USB cables.

3.1 Nothing Detected

For some USB cables, lsusb shows nothing related to Azure Kinect DK.

1	$ lsusb \| grep Azure

Please also refer to: cannot connect via Thunderbolt 3.

3.2 Detected

For other USB cables, both lsusb and AzureKinectFirmwareTool are able to detect Azure Kinect DK.

$ AzureKinectFirmwareTool -q
 == Azure Kinect DK Firmware Tool == 
Device Serial Number: 000593414512
Current Firmware Versions:
  RGB camera firmware:      1.6.110
  Depth camera firmware:    1.6.80
  Depth config file:        6109.7
  Audio firmware:           1.6.14
  Build Config:             Production
  Certificate Type:         Microsoft

3.2.1 k4aviewer No Available Devices

However, when we run k4aviewer, it still shows No Available Devices.

$ lsusb | grep Azure
Bus 009 Device 007: ID 045e:097b Microsoft Corp. Generic Hub [Azure Kinect]
Bus 009 Device 008: ID 045e:097d Microsoft Corp. Azure Kinect 4K Camera
Bus 009 Device 009: ID 045e:097c Microsoft Corp. Azure Kinect Depth Camera
Bus 009 Device 010: ID 045e:097e Microsoft Corp. Azure Kinect Microphone Array

No Available Devices	No Available Devices, Errors

3.2.2 Detect Okay And Run Successfully

Fortunately, I got one cable working partially perperly, but lsusb failed to detect Azure Kinect DK's Microphone Array.

$ lsusb | grep -i Azure 
Bus 010 Device 002: ID 045e:097a Microsoft Corp. Generic Superspeed Hub [Azure Kinect]
Bus 010 Device 003: ID 045e:097c Microsoft Corp. Azure Kinect Depth Camera
Bus 010 Device 004: ID 045e:097d Microsoft Corp. Azure Kinect 4K Camera

3.3 I Believe and I Found It

Luckily, I found a cable, which is able to detect ALL of the following:

$ lsusb | grep Azure   
Bus 001 Device 010: ID 045e:097b Microsoft Corp. Generic Hub [Azure Kinect]
Bus 001 Device 011: ID 045e:097e Microsoft Corp. Azure Kinect Microphone Array
Bus 002 Device 014: ID 045e:097a Microsoft Corp. Generic Superspeed Hub [Azure Kinect]
Bus 002 Device 015: ID 045e:097c Microsoft Corp. Azure Kinect Depth Camera
Bus 002 Device 016: ID 045e:097d Microsoft Corp. Azure Kinect 4K Camera

3.4 4K Camera Is Working Fine Like An Independent UVC Camera

$ v4l2-ctl --list-devices

Azure Kinect 4K Camera (usb-0000:05:00.3-2.1):
	/dev/video1
	/dev/video2
	/dev/media0

Cannot open device /dev/video0, exiting.

It can be simpilied decmonstrated that GUVCView is able to successfully detect Bus 002 Device 016: ID 045e:097d Microsoft Corp. Azure Kinect 4K Camera as a UVC Camera and carry out the real-time streaming.

4. `libdepthengine.so`

Currently, Azure Kinect DK supports Ubuntu 18.04 ONLY. However, the libdepthengine.so.2.0 is working fine under my Ubuntu 24.04, which can be extracted directly from the package libk4a1.4_1.4.2_amd64.deb. Save libdepthengine.so.2.0 under any $PATH, and create a symbolic link if you want to.

5. Build Azure-Kinect-Sensor-SDK Longer Vision From Source

Don’t use the original Azure-Kinect-Sensor-SDK. Use Azure-Kinect-Sensor-SDK Longer Vision instead.

6. Python Wrappers for Azure-Kinect-Sensor-SDK

Wrappers	Description
k4a Python wrapper	the official Python Wrapper with Azure-Kinect-Sensor-SDK
pyk4a	a third-party Python Wrapper

7. The Bug: `replace_sample(). capturesync_drop, releasing capture early due to full queue TS`

Please refer to the following 2 Github issues:

Home Assistant on A Raspberry Pi

Posted on 2024-05-30 Edited on 2024-10-29 In Operating System , Raspberry Pi

Again, amazing China. This time, Nanjing. What a city!!!!!

Ancient Qinhuai @ Qinhuai River, Nanjing	Ancient Qinhuai @ Qinhuai River, Nanjing

Raspberry Pi has already well-established a comprehensive community support world wide. Among ALL the operating systems supporting Raspberry Pi, Home Assistant stands out, shining brightly as an open-source home automation operating system in the field of IoT.

1. Overview

1.1 What is Home Assistant?

Home Assistant is the ``open source home automation that puts local control and privacy first. Powered by a worldwide community of tinkerers and DIY enthusiasts. Perfect to run on a Raspberry Pi or a local server.'' (cited from first page of the official website of Home Assistant.)

Home Assistant OS is a minimal operating system optimized for running Home Assistant, and it does not use traditional package management systems like apt. Instead, it relies on Docker containers to manage and run services, including Home Assistant and its add-ons.

1.2 Web UI

After simply flashed Home Assistant OS 12.3 onto a 16GB TF card and insert it into my old Raspberry Pi 3B V1.2, we can now take a brief look at Home Assistant web UI at http://homeassistant.local:8123/.

Home Assistant Overview	Home Assistant Dashboard

2. `ssh` Into Home Assistant OS

2.1 Configuration

2.1.1 Installation of Home Assistant Add-on Terminal & SSH

Please have the add-on Home Assistant Add-on Terminal & SSH installed from Home Assistant -> Settings -> Add-ons.

2.1.2 Edit in YAML

Please do remember to add these 2 lines:

1
2
3

ssh:
  enable: true
  port: 22

at the end of the YAML configuration:

authorized_keys: []
password: []
apks: []
server:
  tcp_forwarding: false
ssh:
  enable: true
  port: 22

2.2 `ssh` Into Home Assistant OS From Remote Desktop

➜  ~ ssh root@192.168.1.83
The authenticity of host '192.168.1.83 (192.168.1.83)' can't be established.
ED25519 key fingerprint is SHA256:MZDkXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXJmxE.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.1.83' (ED25519) to the list of known hosts.

       ▄██▄           _   _                                    
     ▄██████▄        | | | | ___  _ __ ___   ___               
   ▄████▀▀████▄      | |_| |/ _ \| '_ ` _ \ / _ \              
 ▄█████    █████▄    |  _  | (_) | | | | | |  __/              
▄██████▄  ▄██████▄   |_| |_|\___/|_| |_| |_|\___|          _   
████████  ██▀  ▀██      / \   ___ ___(_)___| |_ __ _ _ __ | |_ 
███▀▀███  ██   ▄██     / _ \ / __/ __| / __| __/ _` | '_ \| __|
██    ██  ▀ ▄█████    / ___ \\__ \__ \ \__ \ || (_| | | | | |_ 
███▄▄ ▀█  ▄███████   /_/   \_\___/___/_|___/\__\__,_|_| |_|\__|
▀█████▄   ███████▀

Welcome to the Home Assistant command line.

System information
  IPv4 addresses for enu1u1: 192.168.1.83/24
  IPv6 addresses for enu1u1: 2001:569:5993:5900:1b4a:e1e6:140c:aac0/64, fe80::ab1f:8f98:b81:a479/64
  IPv4 addresses for wlan0: 
  IPv4 addresses for wlu1u4: 

  OS Version:               Home Assistant OS 12.3
  Home Assistant Core:      2024.5.5

  Home Assistant URL:       http://homeassistant.local:8123
  Observer URL:             http://homeassistant.local:4357
[core-ssh ~]$

3. ESPHome with Home Assistant

One of my very old blogs LVT ESP32-Cam has talked about ESP32 Camera. Today, I found that blog posted 4 years ago before COVID-19, and I’m trying to integrate some of my [ESP32 Cameras] into Home Assistant. Namely, use Home Assistant running on the Raspberry Pi to manage a couple of my ESP32 Cameras.

3.1 ESP32 Boards Management

Please ensure the following 2 boards support have been installed successfully.

3.2 CameraWebServer

Current ESP32 Examples provide an webcam server example: ESP32 Example CameraWebServer. Or, we can simply open up the example from within Arduino IDE: File->Examples->ESP32->Camera->CameraWebServer.

3.2.1 Four ESP32 Cameras

2 Ai Thinker ESP32 Cameras
2 DIY More ESP32 Cameras

Both are compatible with the configuration CAMERA_MODEL_AI_THINKER in ESP32 Example CameraWebServer. Therefore, three places are to be correctly setup:

#define CAMERA_MODEL_AI_THINKER // Has PSRAM
const char *ssid = "XXXXXXXXX";
const char *password = "YYYYYYYYY";

3.2.2 Verify and Upload

3.2.3 Demonstration

However, as you can see, the captured image is not quite stable. And, I did find the WiFi connection is easy to get disconnected. What to do???

[!NOTE:] What to do??? Please continue reading.

3.3 RTSP with MJPEG

As of today, June 4, 2024, there is NO H.264 or H.265 streaming server support. Therefore, I’m going to follow the following 2 blogs and build up RTSP server with MJPEG video streams.

3.4 Integrate ESPHome Into Home Assistant

Getting Started with ESPHome and Home Assistant

3.5 Demonstration

ESPHome	ESPHome

LicheeRV Nano

Posted on 2024-05-02 Edited on 2024-09-29 In Single Board Computers , Risc-V

I got a LicheeRV Nano before I left China for Canada a couple of weeks ago. Nowadays China is amazing. Let’s take a look at my hometown Wuhan of year 2024.

Sunset @ Yangtze River 2nd Bridge	Sunset @ Yangtze River TianXingZhou Bridge

Now, let’s start our today’s topic: LicheeRV Nano.

1. Introduction

1.1 First Look

LicheeRV Nano	LicheeRV Nano with N2425D

1.2 Official Documentation

Actually, ALL you need can be found at:

LicheeRV Nano Official Documentation

1.3 LicheeRV-Nano-WE

According to the LicheeRV Nano Version Comparison, there are 4 versions totally.

LicheeRV-Nano-B
LicheeRV-Nano-E
LicheeRV-Nano-W
LicheeRV-Nano-WE

Clearly, the one I purchased is a LicheeRV-Nano-WE.

2. `lsusb`

➜  ~ lsusb
......
Bus 009 Device 010: ID 3346:1009 sipeed licheervnano
......

3. WiFi Setup

After LicheeRV Nano is connected to your host computer via USB Type-C, enter the mounted rootfs and add the following new file:

➜  rootfs sudo vim ./etc/network/wpa_supplicant.conf
network={
        ssid="XXXXX"
        psk="YYYYY"
}

Then, reboot. Now, you should be able to find out your LicheeRV Nano by sudo nmap -sN 192.168.1.254/24.

4. LicheeRV Nano Overview

4.1 `ssh` Into LicheeRV Nano

Both the default username and password of LicheeRV Nano are root.

➜  ~ ssh root@192.168.1.78
root@192.168.1.78's password:

# uname -a
Linux licheervnano-8c93 5.10.4-tag- #1 PREEMPT Mon Apr 22 11:13:44 HKT 2024 riscv64 GNU/Linux

# hostname
licheervnano-8c93

4.2 CPU and Memory

Clearly, LicheeRV Nano is armed with only 1 single Risc-V 64bit CPU, and about 160M flash memory.

# cat /proc/cpuinfo
processor	: 0
hart		: 0
isa		: rv64imafdvcsu
mmu		: sv39

# cat /proc/meminfo
MemTotal:         162672 kB
MemFree:           98276 kB
MemAvailable:     128976 kB
Buffers:            3416 kB
Cached:            28776 kB
SwapCached:            0 kB
Active:            11792 kB
Inactive:          26892 kB
Active(anon):        144 kB
Inactive(anon):     6592 kB
Active(file):      11648 kB
Inactive(file):    20300 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 4 kB
Writeback:             0 kB
AnonPages:          6508 kB
Mapped:            10092 kB
Shmem:               248 kB
KReclaimable:       5164 kB
Slab:              13988 kB
SReclaimable:       5164 kB
SUnreclaim:         8824 kB
KernelStack:        1248 kB
PageTables:          524 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:       81336 kB
Committed_AS:       9868 kB
VmallocTotal:   67108863 kB
VmallocUsed:        5468 kB
VmallocChunk:          0 kB
Percpu:               64 kB
CmaTotal:              0 kB
CmaFree:               0 kB

4.3 Python

By default, Python 3.11.6 has been installed on LicheeRV Nano.

1 2	# python --version Python 3.11.6

5. LicheeRV-Nano-Build

The default LicheeRV-Nano-Build README.md seems to be problematic. I did it in my way:

5.1 Clone LicheeRV-Nano-Build

1
2
3

➜  git clone https://github.com/sipeed/LicheeRV-Nano-Build --depth=1
➜  cd LicheeRV-Nano-Build
➜  LicheeRV-Nano-Build git:(main)

5.2 Setup

➜  LicheeRV-Nano-Build git:(main) ✗ source build/cvisetup.sh
/proc/self/fd/19:7: cv1800b_board_sel: assignment to invalid subscript range
  -------------------------------------------------------------------------------------------------------
    Usage:
    (1) menuconfig - Use menu to configure your board.
        ex: $ menuconfig

    (2) defconfig $CHIP_ARCH - List EVB boards($BOARD) by CHIP_ARCH.
       ** sg200x ** -> ['sg2000', 'sg2002']
       ** cv181x ** -> ['cv1812cp', 'cv1812h', 'cv1813h']
       ** cv180x ** -> ['cv1800b']
        ex: $ defconfig cv181x

    (3) defconfig $BOARD - Choose EVB board settings.
        ex: $ defconfig cv1813h_wevb_0007a_spinor
        ex: $ defconfig cv1812cp_wevb_0006a_spinor
  -------------------------------------------------------------------------------------------------------

5.3 GUI-based `menuconfig`

I personally prefer the GUI-based configuration.

5.3.1 Copy `menuconfig.py`

1
2
3

➜  LicheeRV-Nano-Build git:(main) ✗ cd build/scripts 
➜  scripts git:(main) ✗ cp menuconfig.py .py
➜  scripts git:(main) ✗

5.3.2 Modify `./build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig`

diff --git a/build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig b/build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig
index 4138f665b..58a5ea02d 100644
--- a/build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig
+++ b/build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig
@@ -2,12 +2,12 @@ CONFIG_CHIP_sg2002=y
 CONFIG_BOARD_licheervnano_sd=y
 CONFIG_DDR_CFG_ddr3_1866_x16=y
 CONFIG_ARCH="riscv"
-CONFIG_CROSS_COMPILE="riscv64-unknown-linux-musl-"
+CONFIG_CROSS_COMPILE="riscv64-linux-gnu-"
 CONFIG_CC_OPTIMIZE_FOR_SIZE=y
 CONFIG_KERNEL_ENTRY_HACK=y
 CONFIG_KERNEL_ENTRY_HACK_ADDR="0x80200000"
 CONFIG_TOOLCHAIN_MUSL_RISCV64=y
-CONFIG_FLASH_SIZE_SHRINK=y
+# CONFIG_FLASH_SIZE_SHRINK=y
 CONFIG_BOOT_IMAGE_SINGLE_DTB=y
 # CONFIG_FLASH_SIZE_SHRINK is not set
 CONFIG_CP_EXT_WIRELESS=y
@@ -16,7 +16,7 @@ CONFIG_MIPI_PANEL_ZCT2133V1=y
 CONFIG_SENSOR_GCORE_GC4653=y
 CONFIG_SENSOR_OV_OS04A10=y
 CONFIG_UBOOT_2021_10=y
-CONFIG_KERNEL_SRC_5.10=y
+CONFIG_KERNEL_SRC="linux_5.10"
 CONFIG_KERNEL_UNCOMPRESSED=y
 # CONFIG_SKIP_RAMDISK is not set
 CONFIG_SENSOR_TUNING_PARAM_cv181x_src_gcore_gc4653=y

By the way, even until now, I’ve got NO idea what the MIPI_PANEL should be?

5.3.3 Run `menuconfig`

1	➜ LicheeRV-Nano-Build git:(main) ✗ menuconfig

5.4 Build

5.4.1 Exports

1
2
3

➜  LicheeRV-Nano-Build git:(main) ✗ export IMGTOOL_PATH=tools/common/image_tool                      
➜  LicheeRV-Nano-Build git:(main) ✗ export COMMON_TOOLS_PATH=tools/common                                 
➜  LicheeRV-Nano-Build git:(main) ✗ export FLASH_PARTITION_XML=boards/default/partition/partition_none.xml

Milk-V Duo 256M - 02

Posted on 2024-02-09 Edited on 2024-09-28 In Single Board Computers , MCU , Risc-V

Happy Chinese Dragon Year !!! Today, let's continue this SBC: Milk-V Duo 256M.

1. Some Suggestions

Auto resizing in order to use the full TF card). So far, I'll have to do it manually. Anyway, now I can use my full TF card as:

[root@milkv-duo]~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                28.5G    146.0M     27.2G   1% /
devtmpfs                 83.1M         0     83.1M   0% /dev
tmpfs                    83.1M         0     83.1M   0% /dev/shm
tmpfs                    83.1M     48.0K     83.1M   0% /tmp
tmpfs                    83.1M     32.0K     83.1M   0% /run

Without resizing, as in my previous blog Milk-V Duo 256M - 01

[root@milkv-duo]~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root               752.0M    152.6M    557.0M  22% /
devtmpfs                 83.1M         0     83.1M   0% /dev
tmpfs                    83.1M         0     83.1M   0% /dev/shm
tmpfs                    83.1M     48.0K     83.1M   0% /tmp
tmpfs                    83.1M     32.0K     83.1M   0% /run

Fix eth0's MAC address. For the wired connection on this particular small SBC Milk-V Duo 256M, I'd strongly recommend to use the physical MAC address for eth0, rather than a dynamic one. Whenever rebooting, Milk-V Duo 256M will change its MAC address, as well as the IP address, which is seriously NOT what I expected. Therefore, what we need to do is to modify file /etc/network/interfaces as suggested by StackExchange issue Change Mac Address permanently inside /etc/network/interfaces:
1
2
3
4
5
6
auto eth0
iface eth0 inet static
address 192.168.1.51
netmask 255.255.255.0
gateway 192.168.1.1
hwaddress ether 00:11:22:33:44:55

2. Yolo5 Example Based on TPU

1. OpenVINO Runtime NO Longer Supports Movidius VPU

1.1 OpenVINO Toolkit Release Notes 2023.0

1.2 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3

1.3 Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2023.3

1.4 OpenVINO 2024.2 @ Intel® Distribution of OpenVINO™ Toolkit Release Notes 2024.2

1.5 Conclusion

2. NCSDK Does NOT Support AArch64

2.1 get_mvcmd.sh is the key.

2.2 Firmware Is Actually Downloaded But Without AArch64 Support

2.3 ncsdk-aarch64 for NCS1

2.3.1 Find the Right Version

2.3.2 Installation

2.3.3 Demonstrate Examples on My Raspberry Pi 5

2.4 ncsdk2-aarch64 for NCS2

2.4.1 Download the Right NCSDK-2.10.01.01.tar.gz

2.4.2 Installation

2.4.3 Demonstrate Examples on My Raspberry Pi 5

3. Intel® Distribution of OpenVINO™ Toolkit Long-Term Support (LTS) 2022.3 for NCS2

3.1 Fix an Error

3.2 Two More Things After Installation

3.2.1 OpenVINO Python Wrapper Installation

3.2.2 Copy plugins.xml

3.3 Demonstration

3.3.1 hello_query_device.py

3.3.2 AI Samples

1. Overview

1.1 What is LibreELEC? What is Kodi?

1.2 Flash LibreELEC Onto Raspberry Pi

2. Demonstration

2.1 First View

2.2 Enable SSH and Allow HTTP

2.3 ssh Into LibreELEC via Wired Connection

2.4 View LibreELEC Over HTTP

2.5 Configure Wi-Fi

2.5.1 Passphrase is Just Your Wifi Password!!

2.5.2 Important Commands

2.5.3 LibreELEC Configuration

3. Upgrade LibreELEC

3.1 Check Current Version

3.2 LibreELEC For Raspberry Pi 4

3.2.1 Two Source of LibreELEC Builds

3.2.2 Upgrade

1. Overview

1.1 What is OctoPrint?

1.2 Flash OctoPi Onto Raspberry Pi

2. ssh Into OctoPi

2.1 FIRST Login

2.1.1 Start NetworkManager Whenever Reboot

2.1.2 nmcli to Set Up Wi-Fi

2.2 ssh Into OctoPi Via Wi-Fi Connection

3. OctoPrint on OctoPi

4. LaserGRBL and LightBurn

1. My Working Environment

2. Kinect 2

2.1 Overview

2.2 Hardware Specification

3. lsusb and add udev rules

3.1 lsusb

3.2 lsusb More Details

3.3 Add udev rules Under /etc/udev/rules.d

3.4 Don’t Use Default OpenNI2 from Default Repository

3.4.1 Incorrect Rules

3.4.2 Incorrect NiViewer

4. For Windows

5. For Linux

5.1 Demonstration FIRST

5.2 Build OpenNI2 Github from Source

5.2.1 Key Components

5.2.2 Tool NiViewer

5.2.3 Any Driver to Drive Kinect2 and Can Work With OpenNI2 Github?

5.3 Build libfreenect2 from Source

5.4 Protonect or ProtonectSR from libfreenect2

5.4.1 ./bin/Protonect cuda

5.4.2 ./bin/Protonect -gpu=0

5.4.3 ./bin/Protonect cpu

5.4.4 Conclusive Comparisons

5.5 pylibfreenect2 - libfreenect2's Python Wrapper

5.6 Kinect2's Audio pyalsaaudio

5.6.1 lsusb Again

5.6.2 arecord -l

2.1 `get_mvcmd.sh` is the key.

3.2.2 Copy `plugins.xml`

2.3 `ssh` Into LibreELEC via Wired Connection

2. `ssh` Into OctoPi

2.2 `ssh` Into OctoPi Via Wi-Fi Connection

3. `lsusb` and `add udev rules`

3.1 `lsusb`

3.2 `lsusb` More Details

3.3 Add `udev rules` Under `/etc/udev/rules.d`

5.4.1 `./bin/Protonect cuda`

5.4.2 `./bin/Protonect -gpu=0`

5.4.3 `./bin/Protonect cpu`

5.6.1 `lsusb` Again

5.6.2 `arecord -l`

5.6.3 List Details of `hw:1,0`

6.1 libfreenect2 - Do NOT Enable `ENABLE_TEGRAJPEG`

6.2 Integrate pylibfreenect2 and pyalsaaudio with GStreamer for 3 Streams from Kinect2: `depth`, `rgb`, and `audio`

3.4.4.3 `ssh` and `neofetch`

4.1 `tegrastats`, `jetson_release`, `jetson_clocks`

4.2 `jtop` from jetson-stats

3. `lsusb` and Demonstration

3.1 `lsusb`

3.2 `realsense-viewer`

3.3 `rs-hello-realsense`

3. `lsusb` and Demonstration

4. `libdepthengine.so`

7. The Bug: `replace_sample(). capturesync_drop, releasing capture early due to full queue TS`

2. `ssh` Into Home Assistant OS