0%

Finally, I got some time for OctoPrint, another specific operating system for Raspberry Pi, as Home Assistant talked in my previous blog Home Assistant on A Raspberry Pi.

1. Overview

1.1 What is OctoPrint?

From its official website, clearly, OctoPrint is an operating system for 3D Printing. Its capabilities can be extended to other manufacturing fields. Specifically, OctoPi will work on a Raspberry Pi.

1.2 Flash OctoPi Onto Raspberry Pi

1
2
3
4
5
6
7
$ sudo dd if=./octopi-1.0.0-1.10.2-20240618101629.img of=/dev/sdb bs=1M status=progress conv=fsync 
[sudo] password for lvision:
2357198848 bytes (2.4 GB, 2.2 GiB) copied, 2 s, 1.2 GB/s2906653696 bytes (2.9 GB, 2.7 GiB) copied, 2.4707 s, 1.2 GB/s

2772+1 records in
2772+1 records out
2906653696 bytes (2.9 GB, 2.7 GiB) copied, 248.633 s, 11.7 MB/s

2. ssh Into OctoPi

2.1 FIRST Login

You got to login OctoPi locally for the FIRST time, instead of remotely, for the NetworkManager service is NOT started by default. Namely, there is NO Wi-Fi by default. Well, you can ALWAYS use the wired connection of course.

2.1.1 Start NetworkManager Whenever Reboot

Use the following commands:

  • sudo systemctl status NetworkManager
  • sudo systemctl enable NetworkManager
  • sudo systemctl start NetworkManager
  • sudo systemctl status NetworkManager for double check
  • sudo reboot reboot to ensure NetworkManager is started by default
  • sudo systemctl status NetworkManager check again

2.1.2 nmcli to Set Up Wi-Fi

Use the following commands:

  • nmcli dev wifi list
  • nmcli dev wifi connect NetworkName password WiFiPassword (You don’t need to quote or double-quote NetworkName or WiFiPassword)

Now, you’re good to go.

2.2 ssh Into OctoPi Via Wi-Fi Connection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
$ ssh lvision@192.168.1.76
lvision@192.168.1.76's password:
Linux octopi 6.1.21-v7+ #1642 SMP Mon Apr 3 17:20:52 BST 2023 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Jul 25 20:23:51 2024 from 192.168.1.1
➜ ~ neofetch
`.::///+:/-. --///+//-:`` lvision@octopi
`+oooooooooooo: `+oooooooooooo: --------------
/oooo++//ooooo: ooooo+//+ooooo. OS: Raspbian GNU/Linux 11 (bullseye) armv7l
`+ooooooo:-:oo- +o+::/ooooooo: Host: Raspberry Pi 3 Model B Rev 1.2
`:oooooooo+`` `.oooooooo+- Kernel: 6.1.21-v7+
`:++ooo/. :+ooo+/.` Uptime: 5 mins
...` `.----.` ``.. Packages: 816 (dpkg)
.::::-``:::::::::.`-:::-` Shell: zsh 5.8
-:::-` .:::::::-` `-:::- Terminal: /dev/pts/0
`::. `.--.` `` `.---.``.::` CPU: BCM2835 (4) @ 1.200GHz
.::::::::` -::::::::` ` Memory: 111MiB / 870MiB
.::` .:::::::::- `::::::::::``::.
-:::` ::::::::::. ::::::::::.`:::-
:::: -::::::::. `-:::::::: ::::
-::- .-:::-.``....``.-::-. -::-
.. `` .::::::::. `..`..
-:::-` -::::::::::` .:::::`
:::::::` -::::::::::` :::::::.
.::::::: -::::::::. ::::::::
`-:::::` ..--.` ::::::.
`...` `...--..` `...`
.::::::::::
`.-::::-`

3. OctoPrint on OctoPi

octopi.local Setup Wizard OctoPrint 3D Printer
OctoPrint OctoPi
OctoPrint Server Reload OctoPrint Webcam
OctoPrint Server Reload OctoPrint Webcam

4. LaserGRBL and LightBurn

Today, I wanna try out the old Kinect 2 on my old Jetson AGX Xavier.

1. My Working Environment

Jetson AGX Xavier 32G

For how I set up my old Jetson AGX Xavier, please refer to my previous blog NVidia Jetson Xavier AGX.

2. Kinect 2

2.1 Overview

Kinect 2 Overview

2.2 Hardware Specification

You can find specs of Kinect v2 in Evaluating the Accuracy of the Azure Kinect and Kinect v2.

3. lsusb and add udev rules

3.1 lsusb

1
2
3
4
$ lsusb | grep Microsoft
Bus 009 Device 015: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor
Bus 010 Device 005: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor
Bus 010 Device 006: ID 045e:02c4 Microsoft Corp. Xbox NUI Sensor

3.2 lsusb More Details

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
➜  ~ lsusb -v -d 045e:02c4

Bus 002 Device 015: ID 045e:02c4 Microsoft Corp.
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.00
bDeviceClass 239 Miscellaneous Device
bDeviceSubClass 2
bDeviceProtocol 1 Interface Association
bMaxPacketSize0 9
idVendor 0x045e Microsoft Corp.
idProduct 0x02c4
bcdDevice 1.00
iManufacturer 1 Microsoft
iProduct 2 Xbox NUI Sensor
iSerial 4 178602434347
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 0x00d6
bNumInterfaces 4
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xe0
Self Powered
Remote Wakeup
MaxPower 144mA
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 0
bInterfaceCount 2
bFunctionClass 255 Vendor Specific Class
bFunctionSubClass 255 Vendor Specific Subclass
bFunctionProtocol 0
iFunction 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 4
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 0
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 7
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 7
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 7
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 19
Transfer Type Interrupt
Synch Type None
Usage Type Feedback
wMaxPacketSize 0x0040 1x 64 bytes
bInterval 14
bMaxBurst 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 0
bNumEndpoints 0
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 0
bInterfaceProtocol 0
iInterface 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 1
bAlternateSetting 1
bNumEndpoints 1
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 0
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x84 EP 4 IN
bmAttributes 1
Transfer Type Isochronous
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 1
bMaxBurst 10
Mult 2
Interface Association:
bLength 8
bDescriptorType 11
bFirstInterface 2
bInterfaceCount 2
bFunctionClass 1 Audio
bFunctionSubClass 2 Streaming
bFunctionProtocol 0
iFunction 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 2
bAlternateSetting 0
bNumEndpoints 0
bInterfaceClass 1 Audio
bInterfaceSubClass 1 Control Device
bInterfaceProtocol 0
iInterface 0
AudioControl Interface Descriptor:
bLength 9
bDescriptorType 36
bDescriptorSubtype 1 (HEADER)
bcdADC 1.00
wTotalLength 0x001e
bInCollection 1
baInterfaceNr(0) 3
AudioControl Interface Descriptor:
bLength 12
bDescriptorType 36
bDescriptorSubtype 2 (INPUT_TERMINAL)
bTerminalID 1
wTerminalType 0x0205 Microphone Array
bAssocTerminal 0
bNrChannels 4
wChannelConfig 0x0000
iChannelNames 0
iTerminal 0
AudioControl Interface Descriptor:
bLength 9
bDescriptorType 36
bDescriptorSubtype 3 (OUTPUT_TERMINAL)
bTerminalID 3
wTerminalType 0x0101 USB Streaming
bAssocTerminal 0
bSourceID 1
iTerminal 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 3
bAlternateSetting 0
bNumEndpoints 0
bInterfaceClass 1 Audio
bInterfaceSubClass 2 Streaming
bInterfaceProtocol 0
iInterface 0
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 3
bAlternateSetting 1
bNumEndpoints 1
bInterfaceClass 1 Audio
bInterfaceSubClass 2 Streaming
bInterfaceProtocol 0
iInterface 0
AudioStreaming Interface Descriptor:
bLength 7
bDescriptorType 36
bDescriptorSubtype 1 (AS_GENERAL)
bTerminalLink 3
bDelay 1 frames
wFormatTag 0x0001 PCM
AudioStreaming Interface Descriptor:
bLength 11
bDescriptorType 36
bDescriptorSubtype 2 (FORMAT_TYPE)
bFormatType 1 (FORMAT_TYPE_I)
bNrChannels 4
bSubframeSize 4
bBitResolution 32
bSamFreqType 1 Discrete
tSamFreq[ 0] 16000
Endpoint Descriptor:
bLength 9
bDescriptorType 5
bEndpointAddress 0x85 EP 5 IN
bmAttributes 1
Transfer Type Isochronous
Synch Type None
Usage Type Data
wMaxPacketSize 0x0100 1x 256 bytes
bInterval 4
bRefresh 0
bSynchAddress 0
bMaxBurst 0
AudioStreaming Endpoint Descriptor:
bLength 7
bDescriptorType 37
bDescriptorSubtype 1 (EP_GENERAL)
bmAttributes 0x00
bLockDelayUnits 0 Undefined
wLockDelay 0x0000
Binary Object Store Descriptor:
bLength 5
bDescriptorType 15
wTotalLength 0x0016
bNumDeviceCaps 2
SuperSpeed USB Device Capability:
bLength 10
bDescriptorType 16
bDevCapabilityType 3
bmAttributes 0x00
wSpeedsSupported 0x000c
Device can operate at High Speed (480Mbps)
Device can operate at SuperSpeed (5Gbps)
bFunctionalitySupport 2
Lowest fully-functional device speed is High Speed (480Mbps)
bU1DevExitLat 10 micro seconds
bU2DevExitLat 2047 micro seconds
USB 2.0 Extension Device Capability:
bLength 7
bDescriptorType 16
bDevCapabilityType 2
bmAttributes 0x00000002
HIRD Link Power Management (LPM) Supported
can't get debug descriptor: Resource temporarily unavailable
Device Status: 0x0001
Self Powered
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
➜  ~ lsusb -v -d 045e:02d9 

Bus 002 Device 014: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.00
bDeviceClass 9 Hub
bDeviceSubClass 0
bDeviceProtocol 3
bMaxPacketSize0 9
idVendor 0x045e Microsoft Corp.
idProduct 0x02d9
bcdDevice 0.73
iManufacturer 1 Microsoft Corporation
iProduct 2 NuiSensor Adaptor
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 0x001f
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xe0
Self Powered
Remote Wakeup
MaxPower 0mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 9 Hub
bInterfaceSubClass 0
bInterfaceProtocol 0 Full speed (or root) hub
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 19
Transfer Type Interrupt
Synch Type None
Usage Type Feedback
wMaxPacketSize 0x0002 1x 2 bytes
bInterval 8
bMaxBurst 0
Hub Descriptor:
bLength 12
bDescriptorType 42
nNbrPorts 1
wHubCharacteristic 0x000d
Per-port power switching
Compound device
Per-port overcurrent protection
bPwrOn2PwrGood 100 * 2 milli seconds
bHubContrCurrent 0 milli Ampere
bHubDecLat 0.4 micro seconds
wHubDelay 4004 nano seconds
DeviceRemovable 0x00
Hub Port Status:
Port 1: 0000.0203 5Gbps power U0 enable connect
Binary Object Store Descriptor:
bLength 5
bDescriptorType 15
wTotalLength 0x002a
bNumDeviceCaps 3
USB 2.0 Extension Device Capability:
bLength 7
bDescriptorType 16
bDevCapabilityType 2
bmAttributes 0x00000002
HIRD Link Power Management (LPM) Supported
SuperSpeed USB Device Capability:
bLength 10
bDescriptorType 16
bDevCapabilityType 3
bmAttributes 0x00
wSpeedsSupported 0x000e
Device can operate at Full Speed (12Mbps)
Device can operate at High Speed (480Mbps)
Device can operate at SuperSpeed (5Gbps)
bFunctionalitySupport 1
Lowest fully-functional device speed is Full Speed (12Mbps)
bU1DevExitLat 4 micro seconds
bU2DevExitLat 231 micro seconds
Container ID Device Capability:
bLength 20
bDescriptorType 16
bDevCapabilityType 4
bReserved 0
ContainerID {30eef35c-07d5-2549-b001-802d79434c30}
can't get debug descriptor: Resource temporarily unavailable
Device Status: 0x000d
Self Powered
U1 Enabled
U2 Enabled

Bus 001 Device 008: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.10
bDeviceClass 9 Hub
bDeviceSubClass 0
bDeviceProtocol 1 Single TT
bMaxPacketSize0 64
idVendor 0x045e Microsoft Corp.
idProduct 0x02d9
bcdDevice 0.70
iManufacturer 1 Microsoft Corporation
iProduct 2 NuiSensor Adaptor
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 0x0019
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0xe0
Self Powered
Remote Wakeup
MaxPower 0mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 1
bInterfaceClass 9 Hub
bInterfaceSubClass 0
bInterfaceProtocol 0 Full speed (or root) hub
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 3
Transfer Type Interrupt
Synch Type None
Usage Type Data
wMaxPacketSize 0x0001 1x 1 bytes
bInterval 12
Hub Descriptor:
bLength 9
bDescriptorType 41
nNbrPorts 1
wHubCharacteristic 0x00ed
Per-port power switching
Compound device
Per-port overcurrent protection
TT think time 32 FS bits
Port indicators
bPwrOn2PwrGood 50 * 2 milli seconds
bHubContrCurrent 100 milli Ampere
DeviceRemovable 0x00
PortPwrCtrlMask 0xff
Hub Port Status:
Port 1: 0000.0100 power
Binary Object Store Descriptor:
bLength 5
bDescriptorType 15
wTotalLength 0x002a
bNumDeviceCaps 3
USB 2.0 Extension Device Capability:
bLength 7
bDescriptorType 16
bDevCapabilityType 2
bmAttributes 0x00000002
HIRD Link Power Management (LPM) Supported
SuperSpeed USB Device Capability:
bLength 10
bDescriptorType 16
bDevCapabilityType 3
bmAttributes 0x00
wSpeedsSupported 0x000e
Device can operate at Full Speed (12Mbps)
Device can operate at High Speed (480Mbps)
Device can operate at SuperSpeed (5Gbps)
bFunctionalitySupport 1
Lowest fully-functional device speed is Full Speed (12Mbps)
bU1DevExitLat 4 micro seconds
bU2DevExitLat 231 micro seconds
Container ID Device Capability:
bLength 20
bDescriptorType 16
bDevCapabilityType 4
bReserved 0
ContainerID {30eef35c-07d5-2549-b001-802d79434c30}
can't get debug descriptor: Resource temporarily unavailable
Device Status: 0x0001
Self Powered

3.3 Add udev rules Under /etc/udev/rules.d

Take a look at OpenKinect libfreenect2 90-kinect2.rules:

1
2
3
4
5
6
7
8
9
10
$ cat platform/linux/udev/90-kinect2.rules            
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ File: platform/linux/udev/90-kinect2.rules
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ # this file belongs in /etc/udev/rules.d/
2 │ # ATTR{product}=="Kinect2"
3 │ SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02c4", MODE="0666"
4 │ SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02d8", MODE="0666"
5 │ SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02d9", MODE="0666"
───────┴───────────────────────────────────────────────────────────────────────────────────

We’ll do the following 2 steps:

  • step 1:
1
$ sudo cp platform/linux/udev/90-kinect2.rules /etc/udev/rules.d
  • step 2:
1
2
$ sudo udevadm control --reload-rules
$ sudo udevadm trigger

3.4 Don’t Use Default OpenNI2 from Default Repository

3.4.1 Incorrect Rules

Take a look at OpenNI2 Kinect2 primesense-usb.rules:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ cat Packaging/Linux/primesense-usb.rules               
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
│ File: Packaging/Linux/primesense-usb.rules
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ # Make primesense device mount with writing permissions (default is read only for unknown devices)
2 │ SUBSYSTEM=="usb", ATTR{idProduct}=="0200", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
3 │ SUBSYSTEM=="usb", ATTR{idProduct}=="0300", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
4 │ SUBSYSTEM=="usb", ATTR{idProduct}=="0401", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
5 │ SUBSYSTEM=="usb", ATTR{idProduct}=="0500", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
6 │ SUBSYSTEM=="usb", ATTR{idProduct}=="0600", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
7 │ SUBSYSTEM=="usb", ATTR{idProduct}=="0601", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
8 │ SUBSYSTEM=="usb", ATTR{idProduct}=="0609", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
9 │ SUBSYSTEM=="usb", ATTR{idProduct}=="1250", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
10 │ SUBSYSTEM=="usb", ATTR{idProduct}=="1260", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
11 │ SUBSYSTEM=="usb", ATTR{idProduct}=="1270", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
12 │ SUBSYSTEM=="usb", ATTR{idProduct}=="1280", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
13 │ SUBSYSTEM=="usb", ATTR{idProduct}=="1290", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"
14 │ SUBSYSTEM=="usb", ATTR{idProduct}=="f9db", ATTR{idVendor}=="1d27", MODE:="0666", OWNER:="root", GROUP:="video"

Note:
by installing libopenni2-dev and openni2-utils from default repo, you’re going to have the above rule primesense-usb.rules installed under /usr/lib/udev/rules.d, instead of /etc/udev/rules.d.

3.4.2 Incorrect NiViewer

Even if package openni2-utils installed an executable NiViewer2 under /usr/bin, it fails to even detect the device Kinect v2.

1
2
3
4
5
$ NiViewer2
openDevice failed:
DeviceOpen using default: no devices found

Press any key to continue . . .

Due to the above Incorrect Rules, this NiViewer2, of course, should NOT detect the device.

Feel so disappointed …

😞

What made me feel worse, even desperate???

The above failure happens on both of my Jetson AGX Xavier running Ubuntu 20.04.6 LTS aarch64, and my desktop running Ubuntu 24.04.

4. For Windows

Please strictly follow Kinect for Windows SDK 2.0. For my Kinect 2, I got the following outputs on my laptop running Windows 11.

5. For Linux

5.1 Demonstration FIRST

OpenNI2 NiViewer
OpenNI2 NiViewer
libfreenect2 Protonect CPU
libfreenect2 Protonect CPU
libfreenect2 Protonect -GPU=0
libfreenect2 Protonect -GPU=0

5.2 Build OpenNI2 Github from Source

5.2.1 Key Components

Components Description
PSCommon The third-party library directly from PrimeSense
OniFile Must be dealing with OpenNI’s .oni files
PS1080 Clearly related to PrimeSense
PSLink Clearly related to PrimeSense
Kinect I believe this is the driver for original Kinect v1, namely, Kinect for Xbox 360.
Kinect2 I believe this is the driver for Kinect2, based on Kinect for Windows SDK 2.0

To build Kinect or Kinect2, I think a Windows environment is a MUST, which is NOT for me.

5.2.2 Tool NiViewer

Driver Kinect2 in the above is a MUST for OpenNI2 Github‘s application NiViewer.

5.2.3 Any Driver to Drive Kinect2 and Can Work With OpenNI2 Github?

Now, stucked. Under Linux, how to build a driver working for my Kinect v2?

Solution: Build libfreenect2 from source, with BUILD_OPENNI2_DRIVER ON.

5.3 Build libfreenect2 from Source

Download OpenNI2 and save it, and built libfreenect2 with BUILD_OPENNI2_DRIVER set ON.

By doing so, you will have the following 2 variables set by default as:

1
2
OpenNI2_INCLUDE_DIRS             /opt/OpenNI/Include
OpenNI2_LIBRARY /opt/OpenNI/Redist/libOpenNI2.so

and you will have the following two libraries built out:

1
2
lib/libfreenect2-openni2.so
lib/libfreenect2.so

And, if needed, after the installation of libfreenect2, manually copy libfreenect2-openni2.so under folder /usr/local/lib/OpenNI2/Drivers. Now, ALL set. You’re good to go NiViewer.

5.4 Protonect or ProtonectSR from libfreenect2

5.4.1 ./bin/Protonect cuda

I got NO idea why ./bin/Protonect cuda failed to run ALL the time. Then, I created a simple Python script verify_cuda.py and ALWAYS meet the following error:

1
2
3
4
5
$ python verify_cuda.py   
[Info] [CudaDepthPacketProcessorImpl] device 0: NVIDIA GeForce RTX 3090 @ 1725MHz Memory 24219MB
[Info] [CudaDepthPacketProcessorImpl] selected device 0
[Error] [CudaDepthPacketProcessorImpl] cudaGetLastError(): the provided PTX was compiled with an unsupported toolchain.
CudaPacketPipeline is available.

Finally, I found the reason: Cuda version is NOT synced with NVidia driver version. More specifically, in my case:

To solve this problem, install NVidia driver during the installation of Cuda. Namely: when run cuda_12.5.1_555.42.06_linux.run, tick

1
2
- [X] Driver                                                                 │
│ [X] 555.42.06
  • You may have to reboot into text mode and have all previously-installed NVidia driver unloaded. For this issue, please refer to How to unload kernel module ‘nvidia-drm’?
  • The key: sudo systemctl isolate multi-user.target

Protonect cuda

5.4.2 ./bin/Protonect -gpu=0

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
$ ./bin/Protonect -gpu=0
Version: 0.2.0
Environment variables: LOGFILE=<protonect.log>
Usage: ./bin/Protonect [-gpu=<id>] [gl | cl | clkde | cuda | cudakde | cpu] [<device serial>]
[-noviewer] [-norgb | -nodepth] [-help] [-version]
[-frames <number of frames to process>]
To pause and unpause: pkill -USR1 Protonect
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 23 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @10:5 with serial 178602434347
[Info] [Freenect2Impl] found 1 devices
libva info: VA-API version 1.20.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nvidia_drv_video.so
libva info: va_openDriver() returns -1
[Error] [VaapiRgbPacketProcessorImpl] vaInitialize(display, &major_ver, &minor_ver): unknown libva error
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 60*8*33792
[Info] [Freenect2DeviceImpl] opened
[Info] [Freenect2DeviceImpl] starting...
[Debug] [Freenect2DeviceImpl] status 0x090000: 9855
[Debug] [Freenect2DeviceImpl] status 0x090000: 9855
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
device serial: 178602434347
device firmware: 4.0.3916.0
[Debug] [DepthPacketStreamParser] not all subsequences received 0
[Debug] [DepthPacketStreamParser] not all subsequences received 1021
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [DepthPacketStreamParser] 2 packets were lost
[Info] [DepthPacketStreamParser] 7 packets were lost
[Info] [OpenGLDepthPacketProcessor] avg. time: 5.12134ms -> ~195.261Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 4.06057ms -> ~246.271Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 19.003ms -> ~52.6232Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 4.01312ms -> ~249.182Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 4.20707ms -> ~237.695Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 17.376ms -> ~57.5507Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 3.95256ms -> ~253.001Hz
[2] - 2620271 killed ./bin/ProtonectSR cuda
[Info] [OpenGLDepthPacketProcessor] avg. time: 4.40171ms -> ~227.184Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 17.8467ms -> ~56.0326Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 6.23296ms -> ~160.437Hz
[3] + 2628690 killed ./bin/Protonect
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [OpenGLDepthPacketProcessor] avg. time: 9.88223ms -> ~101.192Hz
[Info] [DepthPacketStreamParser] 2 packets were lost
[Info] [TurboJpegRgbPacketProcessor] avg. time: 19.2099ms -> ~52.0565Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 5.8084ms -> ~172.164Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 8.7243ms -> ~114.622Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 18.2581ms -> ~54.7703Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 9.24396ms -> ~108.179Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 10.1021ms -> ~98.9893Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 18.3488ms -> ~54.4995Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 8.35109ms -> ~119.745Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 5.61051ms -> ~178.237Hz
[Info] [TurboJpegRgbPacketProcessor] avg. time: 17.9773ms -> ~55.6256Hz
[Info] [OpenGLDepthPacketProcessor] avg. time: 6.80606ms -> ~146.928Hz
[Info] [Freenect2DeviceImpl] stopping...
[Info] [Freenect2DeviceImpl] canceling rgb transfers...
[Info] [Freenect2DeviceImpl] canceling depth transfers...
[Info] [Freenect2DeviceImpl] stopped
[Info] [Freenect2DeviceImpl] closing...
[Info] [Freenect2DeviceImpl] releasing usb interfaces...
[Info] [Freenect2DeviceImpl] deallocating usb transfer pools...
[Info] [Freenect2DeviceImpl] closing usb device...
[Info] [Freenect2DeviceImpl] closed
[Info] [Freenect2DeviceImpl] closing...
[Info] [Freenect2DeviceImpl] already closed, doing nothing
[TurboJpegRgbPacketProcessor] 9.8382 10.0887 18.9733 20.2907 26.1092 mean=18.2873 std=2.86968 n=749
[OpenGLDepthPacketProcessor] 2.90518 3.87679 5.69045 12.5144 95.8873 mean=6.46404 std=4.29096 n=1528

5.4.3 ./bin/Protonect cpu

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
$ ./bin/Protonect cpu
Version: 0.2.0
Environment variables: LOGFILE=<protonect.log>
Usage: ./bin/Protonect [-gpu=<id>] [gl | cl | clkde | cuda | cudakde | cpu] [<device serial>]
[-noviewer] [-norgb | -nodepth] [-help] [-version]
[-frames <number of frames to process>]
To pause and unpause: pkill -USR1 Protonect
libva info: VA-API version 1.20.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nvidia_drv_video.so
libva info: va_openDriver() returns -1
[Error] [VaapiRgbPacketProcessorImpl] vaInitialize(display, &major_ver, &minor_ver): unknown libva error
[Info] [Freenect2Impl] enumerating devices...
[Info] [Freenect2Impl] 23 usb devices connected
[Info] [Freenect2Impl] found valid Kinect v2 @10:5 with serial 178602434347
[Info] [Freenect2Impl] found 1 devices
[Info] [Freenect2DeviceImpl] opening...
[Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 60*8*33792
[Info] [Freenect2DeviceImpl] opened
[Info] [Freenect2DeviceImpl] starting...
[Debug] [Freenect2DeviceImpl] status 0x090000: 9729
[Debug] [Freenect2DeviceImpl] status 0x090000: 9731
[Info] [Freenect2DeviceImpl] submitting rgb transfers...
[Info] [Freenect2DeviceImpl] submitting depth transfers...
[Info] [Freenect2DeviceImpl] started
device serial: 178602434347
device firmware: 4.0.3916.0
[Debug] [DepthPacketStreamParser] not all subsequences received 0
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [DepthPacketStreamParser] 30 packets were lost
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [DepthPacketStreamParser] 30 packets were lost
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [Freenect2DeviceImpl] stopping...
[Info] [Freenect2DeviceImpl] canceling rgb transfers...
[Debug] [DepthPacketStreamParser] skipping depth packet
[Debug] [DepthPacketStreamParser] skipping depth packet
[Info] [Freenect2DeviceImpl] canceling depth transfers...
[Info] [Freenect2DeviceImpl] stopped
[Info] [Freenect2DeviceImpl] closing...
[Info] [Freenect2DeviceImpl] releasing usb interfaces...
[Info] [Freenect2DeviceImpl] deallocating usb transfer pools...
[Info] [Freenect2DeviceImpl] closing usb device...
[Info] [Freenect2DeviceImpl] closed
[Info] [Freenect2DeviceImpl] closing...
[Info] [Freenect2DeviceImpl] already closed, doing nothing
[TurboJpegRgbPacketProcessor] 10.0384 10.1629 18.9524 20.9346 27.5923 mean=18.5922 std=3.23163 n=44
[CpuDepthPacketProcessor] 101.371 101.485 102.303 109.367 111.332 mean=102.829 std=2.25233 n=27

5.4.4 Conclusive Comparisons

Let’s do a simple comparison of the speed using 3 different parameters: cuda, -gpu=0, cpu.

Commands RGB Mean Time (ms) Depth Mean Time (ms)
./bin/Protonect cuda 18.8951 0.521472
./bin/Protonect -gpu=0 18.2873 6.46404
./bin/Protonect cpu 18.5922 102.829

The conclusion is quite obvious:

  • for depth: CudaDepthPacketProcessor > OpenGLDepthPacketProcessor > CpuDepthPacketProcessor.
  • for RGB: Always TurboJpegRgbPacketProcessor

5.5 pylibfreenect2 - libfreenect2‘s Python Wrapper

So far, in my personal point of view, pylibfreenect2 is the BEST Python wrapper of libfreenect2. But, according to the official documentation of libfreenect2, it’s clearly written:

Audio. Raw audio is accessible via Linux USB audio. There is no support for the calibrated directional audio.

5.6 Kinect2‘s Audio pyalsaaudio

5.6.1 lsusb Again

1
2
3
4
➜  ~ lsusb | grep Microsoft 
Bus 002 Device 003: ID 045e:02c4 Microsoft Corp.
Bus 002 Device 002: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor
Bus 001 Device 002: ID 045e:02d9 Microsoft Corp. NuiSensor Adaptor

5.6.2 arecord -l

1
2
➜  ~ arecord -l | grep NUI             
card 1: Sensor [Xbox NUI Sensor], device 0: USB Audio [USB Audio]
  • Card number: 1
  • Device number: 0

5.6.3 List Details of hw:1,0

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
➜  arecord -D hw:1,0 --dump-hw-params
Recording WAVE 'stdin' : Unsigned 8 bit, Rate 8000 Hz, Mono
HW Params of device "hw:1,0":
--------------------
ACCESS: MMAP_INTERLEAVED RW_INTERLEAVED
FORMAT: S32_LE
SUBFORMAT: STD
SAMPLE_BITS: 32
FRAME_BITS: 128
CHANNELS: 4
RATE: 16000
PERIOD_TIME: [1000 2048000]
PERIOD_SIZE: [16 32768]
PERIOD_BYTES: [256 524288]
PERIODS: [2 1024]
BUFFER_TIME: [2000 4096000]
BUFFER_SIZE: [32 65536]
BUFFER_BYTES: [512 1048576]
TICK_TIME: ALL
--------------------
arecord: set_params:1368: Sample format non available
Available formats:
- S32_LE

A couple of concerns:

  • I personally think FRAME_BITS is a typo. It should be FRAME_SIZE.
  • What the sample rate is 16000, but NOT 44100? 😀

5.6.4 Demonstration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import alsaaudio
import wave
import time

# Open the audio device
inp = alsaaudio.PCM(alsaaudio.PCM_CAPTURE, alsaaudio.PCM_NORMAL, "plughw:Sensor")

# Set audio parameters
inp.setformat(alsaaudio.PCM_FORMAT_S32_LE)
inp.setchannels(4)
inp.setrate(16000)
inp.setperiodsize(128)

# Prepare a file to write the audio data
with wave.open("test.wav", "wb") as f:
f.setnchannels(4)
f.setsampwidth(4)
f.setframerate(16000)

# Record audio for 10 seconds
timeout = 10 # seconds
start_time = time.time()
while time.time() - start_time < timeout:
num_bytes, buffer = inp.read()
f.writeframes(buffer)

# Close the audio device
inp.close()

print("Recording finished")

6. For Jetson AGX Xavier

Now, we’re done on my desktop. Let’s try Kinect2 on Jetson AGX Xavier.

6.1 libfreenect2 - Do NOT Enable ENABLE_TEGRAJPEG

By default, libfreenect2 has ENABLE_TEGRAJPEG set ON, which builds fine. However, the followig issue has been reported for over 8 years.

Solution for Jetson AGX Xavier:

libfreenect2 Protonect Cuda
libfreenect2 Protonect Cuda
libfreenect2 Protonect Cuda Speed
libfreenect2 Protonect Cuda

From the above output, we can easily come up with the following table:

Attribute RGB Depth
Resolution 1920x1080 512x424
Number of Channels 3 (Red, Green, Blue) 1 (Depth)
Bytes per Pixel 3 bytes 2 bytes
Average Processing Time ~22.4048 ms ~7.18678 ms
FPS ~44.6163 FPS ~110.134 FPS
Data per Frame (Bytes) 1920 * 1080 * 3 = 6,220,800 bytes 512 * 424 * 2 = 434,176 bytes
Data per Frame (MB) ~6.22 MB ~0.414 MB
Data per Second (MB/s) 6.22 MB * 44.6163 ≈ 277.57 MB/s 0.414 MB * 110.134 ≈ 45.60 MB/s

Total Data per Second: 277.57 MB + 45.60 MB ≈ 323.17 MB

6.2 Integrate pylibfreenect2 and pyalsaaudio with GStreamer for 3 Streams from Kinect2: depth, rgb, and audio

Sorry my friends. I’m NOT going to disclose my code, in Python.

6.2.1 Server Side

Kinect2 RTSP Server
Kinect2 RTSP Server
Kinect2 RTSP Server Stream Responses
Kinect2 RTSP Server Stream Responses

6.2.2 Client Side

RGB Stream
RGB Stream
Depth Stream
Depth Stream
Audio Stream
Audio Stream

I got one Jetson AGX Xavier unused for quite long. There are a couple of reasons prohibiting me from using it.

1. My Working Environment

1
2
3
4
5
6
7
8
9
➜  ~ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04.1 LTS
Release: 24.04
Codename: noble

➜ ~ uname -r
6.8.0-47-generic

2. Jetson Embedded Products

2.1 GPU Compute Capability

Jetson Model Compute Capability
Jetson Orin Series including: Jetson AGX Orin, Jetson Orin NX, Jetson Orin Nano 8.7
Jetson Xavier Series including: Jetson AGX Xavier, Jetson Xavier NX 7.2
Jetson TX2 6.2
Jetson Nano including: default Jetson Nano 4GB, Jetson Nano 2GB 5.3

2.2 JetPack Version Compatability

Please just refer to JetPack Archive.

2.2.1 Jetson Nano Series

Specification Jetson Nano 4GB Jetson Nano 2GB
JetPack Versions 4.2 ~ 4.6.5 4.4 ~ 4.6.5
CUDA Cores 128 128
Compute Capability 5.3 5.3
GPU Architecture Maxwell Maxwell
Memory 4GB LPDDR4 2GB LPDDR4
AI Performance 472 GFLOPs 472 GFLOPs

2.2.2 Jetson Xavier Series

Specification Jetson AGX Xavier 32GB Jetson AGX Xavier 8GB
JetPack Versions 4.2 ~ 5.1.4 4.4 ~ 5.1.4
CUDA Cores 512 512
Compute Capability 7.2 7.2
GPU Architecture Volta Volta
Memory 32GB LPDDR4x 8GB LPDDR4x
AI Performance 11 TFLOPs (FP16) 11 TFLOPs (FP16)

2.2.3 Jetson Orin Series

Specification Jetson AGX Orin 64GB Jetson AGX Orin 32GB Jetson Orin NX 16GB Jetson Orin NX 8GB Jetson Orin Nano 8GB Jetson Orin Nano 4GB
JetPack Versions 5.0 Developer Preview - 6.1 5.0 Developer Preview - 6.1 5.0 Developer Preview - 6.1 5.0 Developer Preview - 6.1 5.0 Developer Preview - 6.1 5.0 Developer Preview - 6.1
CUDA Cores 2048 2048 1024 1024 512 512
Compute Capability 8.7 8.7 8.7 8.7 8.7 8.7
GPU Architecture Ampere Ampere Ampere Ampere Ampere Ampere
Memory 64GB LPDDR5 32GB LPDDR5 16GB LPDDR5 8GB LPDDR5 8GB LPDDR5 4GB LPDDR5
AI Performance up to 200 TOPs up to 200 TOPs up to 70 TOPs up to 70 TOPs up to 40 TOPs up to 40 TOPs

2.3 JetPack Archive

2.4 Jetson Download Center

A variety of software can be downloaded from Jetson Download Center.

3. My Jetson Embedded Boards

I actually own 4 Jetson embedded boards and am trying 1 more:

3.1 Jetson Nano 2GB for JetBot

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
➜  ~ cat /proc/cpuinfo
processor : 0
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

processor : 1
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

processor : 2
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

processor : 3
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
➜  ~ cat /proc/meminfo
MemTotal: 2027240 kB
MemFree: 304868 kB
MemAvailable: 1619252 kB
Buffers: 301640 kB
Cached: 978212 kB
SwapCached: 0 kB
Active: 591384 kB
Inactive: 838300 kB
Active(anon): 77428 kB
Inactive(anon): 112776 kB
Active(file): 513956 kB
Inactive(file): 725524 kB
Unevictable: 828 kB
Mlocked: 0 kB
SwapTotal: 5207916 kB
SwapFree: 5207796 kB
Dirty: 272 kB
Writeback: 0 kB
AnonPages: 143560 kB
Mapped: 168900 kB
Shmem: 39548 kB
Slab: 217244 kB
SReclaimable: 162968 kB
SUnreclaim: 54276 kB
KernelStack: 4384 kB
PageTables: 3520 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 6221536 kB
Committed_AS: 1029676 kB
VmallocTotal: 263061440 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
AnonHugePages: 40960 kB
ShmemHugePages: 0 kB
ShmemPmdMapped: 0 kB
NvMapMemFree: 2048 kB
NvMapMemUsed: 21324 kB
CmaTotal: 65536 kB
CmaFree: 13236 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
1
2
3
4
5
6
7
8
9
10
➜  ~ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk0p1 58G 29G 28G 51% /
none 951M 0 951M 0% /dev
tmpfs 990M 4.0K 990M 1% /dev/shm
tmpfs 990M 38M 953M 4% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 990M 0 990M 0% /sys/fs/cgroup
tmpfs 198M 4.0K 198M 1% /run/user/121
tmpfs 198M 0 198M 0% /run/user/1000

3.2 Jetson Nano from ShenZhen Longer Vision Technology

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
➜  ~ cat /proc/cpuinfo
processor : 0
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

processor : 1
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

processor : 2
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1

processor : 3
model name : ARMv8 Processor rev 1 (v8l)
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
➜  ~ cat /proc/meminfo
MemTotal: 4059260 kB
MemFree: 1409344 kB
MemAvailable: 2538768 kB
Buffers: 30120 kB
Cached: 1254004 kB
SwapCached: 0 kB
Active: 1017736 kB
Inactive: 1170840 kB
Active(anon): 905764 kB
Inactive(anon): 45076 kB
Active(file): 111972 kB
Inactive(file): 1125764 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 2029616 kB
SwapFree: 2029616 kB
Dirty: 152 kB
Writeback: 0 kB
AnonPages: 898652 kB
Mapped: 560276 kB
Shmem: 46388 kB
Slab: 102792 kB
SReclaimable: 46832 kB
SUnreclaim: 55960 kB
KernelStack: 6160 kB
PageTables: 7752 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 4059244 kB
Committed_AS: 2500320 kB
VmallocTotal: 263061440 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
AnonHugePages: 131072 kB
ShmemHugePages: 0 kB
ShmemPmdMapped: 0 kB
NvMapMemFree: 0 kB
NvMapMemUsed: 258692 kB
CmaTotal: 475136 kB
CmaFree: 441280 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
1
2
3
4
5
6
7
8
9
10
➜  ~ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk0p1 14G 12G 1.2G 92% /
none 1.8G 0 1.8G 0% /dev
tmpfs 2.0G 19M 2.0G 1% /dev/shm
tmpfs 2.0G 28M 2.0G 2% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
tmpfs 397M 12K 397M 1% /run/user/120
tmpfs 397M 0 397M 0% /run/user/1000

3.3 Jetson Xavier NX from ShenZhen Longer Vision Technology

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
➜  ~ cat /proc/cpuinfo
processor : 0
model name : ARMv8 Processor rev 0 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer : 0x4e
CPU architecture: 8
CPU variant : 0x0
CPU part : 0x004
CPU revision : 0
MTS version : 50168445

processor : 1
model name : ARMv8 Processor rev 0 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer : 0x4e
CPU architecture: 8
CPU variant : 0x0
CPU part : 0x004
CPU revision : 0
MTS version : 50168445

processor : 2
model name : ARMv8 Processor rev 0 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer : 0x4e
CPU architecture: 8
CPU variant : 0x0
CPU part : 0x004
CPU revision : 0
MTS version : 50168445

processor : 3
model name : ARMv8 Processor rev 0 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer : 0x4e
CPU architecture: 8
CPU variant : 0x0
CPU part : 0x004
CPU revision : 0
MTS version : 50168445

processor : 4
model name : ARMv8 Processor rev 0 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer : 0x4e
CPU architecture: 8
CPU variant : 0x0
CPU part : 0x004
CPU revision : 0
MTS version : 50168445

processor : 5
model name : ARMv8 Processor rev 0 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp
CPU implementer : 0x4e
CPU architecture: 8
CPU variant : 0x0
CPU part : 0x004
CPU revision : 0
MTS version : 50168445
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
➜  ~ cat /proc/meminfo
MemTotal: 7958064 kB
MemFree: 6376676 kB
MemAvailable: 6968608 kB
Buffers: 31648 kB
Cached: 706620 kB
SwapCached: 0 kB
Active: 422156 kB
Inactive: 581804 kB
Active(anon): 267236 kB
Inactive(anon): 27908 kB
Active(file): 154920 kB
Inactive(file): 553896 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 3979008 kB
SwapFree: 3979008 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 254048 kB
Mapped: 187980 kB
Shmem: 29456 kB
Slab: 132884 kB
SReclaimable: 60592 kB
SUnreclaim: 72292 kB
KernelStack: 6640 kB
PageTables: 5892 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 7958040 kB
Committed_AS: 1977692 kB
VmallocTotal: 263061440 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
AnonHugePages: 90112 kB
ShmemHugePages: 0 kB
ShmemPmdMapped: 0 kB
NvMapMemFree: 704 kB
NvMapMemUsed: 27852 kB
CmaTotal: 753664 kB
CmaFree: 704152 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
1
2
3
4
5
6
7
8
9
10
➜  ~ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk0p1 14G 13G 253M 99% /
none 3.5G 0 3.5G 0% /dev
tmpfs 3.8G 4.0K 3.8G 1% /dev/shm
tmpfs 3.8G 29M 3.8G 1% /run
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup
tmpfs 778M 12K 778M 1% /run/user/120
tmpfs 778M 0 778M 0% /run/user/1000

3.4 Jetson AGX Xavier

3.4.1 Entering Recovery Mode

After recovery mode is entered, you’ll see:

1
2
➜  ~ lsusb | grep NVIDIA
Bus 009 Device 031: ID 0955:7019 NVIDIA Corp. APX

3.4.2 Failed to Flash via NVidia SDK Manager

Select Jetson AGX Xavier No available releases for host OS: Ubuntu 24.04
NVidia SDK Manager Jetson AGX Xavier NVidia SDK Manager No Available Release for Host

3.4.3 Try SD Car Image

Download JP514-xnx-sd-card-image_b11.zip.

1
2
3
4
5
6
7
$ sudo dd if=./sd-blob.img of=/dev/sdb bs=1M status=progress conv=fsync 
[sudo] password for lvision:
17746100224 bytes (18 GB, 17 GiB) copied, 26 s, 683 MB/s18153996288 bytes (18 GB, 17 GiB) copied, 26.5595 s, 684 MB/s

17313+0 records in
17313+0 records out
18153996288 bytes (18 GB, 17 GiB) copied, 1617.17 s, 11.2 MB/s

However, Jetson AGX Xavier is NOT booting, but ONLY black screen.

The answer is at NVidia Forum Issue 189348 - Black Screen: Do you know that you are talking about AGX Xavier but not NX? Only NX and Nano have and support sdcard image. Other platforms do not.
It’s also clearly clarified in the table on NVidia’s official page: NVidia SDK Manager.

3.4.4 Negotiated to Use Ubuntu 20.04

Still problemabic. Two main issues:

  • That IP: 192.168.55.1 won’t let me flash successfully
  • CUDA is not able to be installed successfully

3.4.4.2 Again: Jetson Linux Developer Guide Quick Start

1
sudo ./flash.sh jetson-agx-xavier-devkit internal

works for me.

3.4.4.3 ssh and neofetch

ssh into Jetson AGX Xavier Nano and neofetch

4. Demonstrate GPU On Jetson AGX Xavier

4.1 tegrastats, jetson_release, jetson_clocks

tegrastats + jetson_release + jetson_clocks

4.2 jtop from jetson-stats

JTop Jetson AGX Xavier

5. YOLOv11 On Jetson AGX Xavier

Similar difficulties happened on this board Jetson AGX Xavier, which is clearly elaborated in one of my blogs NVidia-Jetson-Orin-Nano-8G-Dev-Kit.md.

Solution:

YOLOv11 Initialization YOLOv11 GPU
YOLO v11 Initialization YOLO v11 GPU

One of my blogs YOLOv11.md didn’t talk about which model to use. And today, let’s try them out.

Today, we’re playing Intel RealSense Depth Camera D435 and
Intel RealSense Depth Modules and Processors.

1. My Working Environment

1
2
3
4
5
6
7
8
9
$ lsb_release -a             
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04 LTS
Release: 24.04
Codename: noble

$ uname -r
6.8.0-36-generic

2. Intel RealSense

2.1 Overview

Intel RealSense D435 Frontal

2.2 Hardware Specification

Refer to Tech Specs pdf.

3. lsusb and Demonstration

3.1 lsusb

Respectively, I got:

1
2
$ lsusb | grep -i RealSense
Bus 010 Device 012: ID 8086:0b07 Intel Corp. RealSense D435
1
2
$ lsusb | grep -i RealSense
Bus 010 Device 004: ID 8086:0ad4 Intel Corp. Intel(R) RealSense(TM) Depth Camera 430

3.2 realsense-viewer

3.2.1 Intel RealSense Depth Camera D435

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ realsense-viewer
05/07 15:16:42,021 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video0
05/07 15:16:42,099 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video1
05/07 15:16:42,101 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video2
05/07 15:16:42,103 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video3
05/07 15:16:42,104 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video4 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.3/video4linux/video4
05/07 15:16:42,106 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video5 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.3/video4linux/video5
05/07 15:16:42,108 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video6 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
05/07 15:16:42,211 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video7 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video7
05/07 15:16:42,498 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video0
05/07 15:16:42,499 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video1
05/07 15:16:42,500 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video2
05/07 15:16:42,501 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video3
05/07 15:16:42,502 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video4 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.3/video4linux/video4
05/07 15:16:42,503 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video5 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.3/video4linux/video5
05/07 15:16:42,504 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video6 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
05/07 15:16:42,505 INFO [135660990636032] (backend-v4l2.cpp:753) Enumerating UVC video7 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video7
05/07 15:16:42,506 INFO [135660990636032] (context.cpp:116) ... /sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-4/10-4:1.0/video4linux/video0
05/07 15:16:42,506 INFO [135660990636032] (context.cpp:116) ... /sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
05/07 15:16:42,506 INFO [135660990636032] (context.cpp:128) Found 2 RealSense devices (0xff requested & 0xff from device-mask in settings)
...
realsense-viewer GUI Firmware Upgrading from within realsense-viewer GUI
D435 RealSense Viewer D435 Firmware Upgrading
Viewing Depth Image in 2D Viewing Depth Image in 3D
D435 Depth 2D D435 Depth 3D

3.2.2 Intel RealSense Depth Modules and Processors D430

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ realsense-viewer  
05/07 17:00:55,958 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video0
05/07 17:00:56,036 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video1
05/07 17:00:56,039 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video2
05/07 17:00:56,041 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video3
05/07 17:00:56,043 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video6 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
05/07 17:00:56,146 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video7 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video7
05/07 17:00:56,387 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video0 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video0
05/07 17:00:56,389 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video1 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video1
05/07 17:00:56,390 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video2 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video2
05/07 17:00:56,391 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video3 realpath=/sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video3
05/07 17:00:56,392 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video6 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
05/07 17:00:56,393 INFO [133348205735936] (backend-v4l2.cpp:753) Enumerating UVC video7 realpath=/sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video7
05/07 17:00:56,394 INFO [133348205735936] (context.cpp:116) ... /sys/devices/pci0000:40/0000:40:01.1/0000:41:00.0/0000:42:08.0/0000:46:00.3/usb10/10-3/10-3:1.0/video4linux/video0
05/07 17:00:56,394 INFO [133348205735936] (context.cpp:116) ... /sys/devices/pci0000:00/0000:00:08.1/0000:05:00.3/usb1/1-2/1-2:1.0/video4linux/video6
05/07 17:00:56,394 INFO [133348205735936] (context.cpp:128) Found 2 RealSense devices (0xff requested & 0xff from device-mask in settings)
D430 Depth Camera 1 - 2D IR Sparse Pattern D430 Depth Camera 1 - 3D
D430 Depth 2D Sparse Pattern D430 Depth 3D
D430 Depth Camera 2 - 2D IR Sparse Pattern D430 Depth Camera 2 - 3D
D430 Depth 2D Sparse Pattern D430 Depth 3D
Firmware To Upgrade Firmware Upgraded Successfully
D430 Firmware To Upgrade D430 Firmware Upgraded Successfully

3.3 rs-hello-realsense

I tried Azure Kinect DK 2 years ago, and rencently I’d love to some fun with it again.

1. My Working Environment

1
2
3
4
5
6
7
8
9
$ lsb_release -a             
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04 LTS
Release: 24.04
Codename: noble

$ uname -r
6.8.0-36-generic

2. Azure Kinect DK

2.1 Overview

Azure Kinect Sensor

2.2 Hardware Specification

Refer to Azure Kinect DK hardware specifications.

3. lsusb and Demonstration

[!NOTE:]
Azure Kinect DK is quite picky at USB cables.

3.1 Nothing Detected

For some USB cables, lsusb shows nothing related to Azure Kinect DK.

1
$ lsusb | grep Azure

Please also refer to: cannot connect via Thunderbolt 3.

3.2 Detected

For other USB cables, both lsusb and AzureKinectFirmwareTool are able to detect Azure Kinect DK.

1
2
3
4
5
6
7
8
9
10
$ AzureKinectFirmwareTool -q
== Azure Kinect DK Firmware Tool ==
Device Serial Number: 000593414512
Current Firmware Versions:
RGB camera firmware: 1.6.110
Depth camera firmware: 1.6.80
Depth config file: 6109.7
Audio firmware: 1.6.14
Build Config: Production
Certificate Type: Microsoft

3.2.1 k4aviewer No Available Devices

However, when we run k4aviewer, it still shows No Available Devices.

1
2
3
4
5
$ lsusb | grep Azure
Bus 009 Device 007: ID 045e:097b Microsoft Corp. Generic Hub [Azure Kinect]
Bus 009 Device 008: ID 045e:097d Microsoft Corp. Azure Kinect 4K Camera
Bus 009 Device 009: ID 045e:097c Microsoft Corp. Azure Kinect Depth Camera
Bus 009 Device 010: ID 045e:097e Microsoft Corp. Azure Kinect Microphone Array
No Available Devices No Available Devices, Errors
K4A Viewer No Available Devices K4A Viewer No Available Devices, Errors

3.2.2 Detect Okay And Run Successfully

Fortunately, I got one cable working partially perperly, but lsusb failed to detect Azure Kinect DK‘s Microphone Array.

1
2
3
4
$ lsusb | grep -i Azure 
Bus 010 Device 002: ID 045e:097a Microsoft Corp. Generic Superspeed Hub [Azure Kinect]
Bus 010 Device 003: ID 045e:097c Microsoft Corp. Azure Kinect Depth Camera
Bus 010 Device 004: ID 045e:097d Microsoft Corp. Azure Kinect 4K Camera

K4A Viewer

3.3 I Believe and I Found It

Luckily, I found a cable, which is able to detect ALL of the following:

1
2
3
4
5
6
$ lsusb | grep Azure   
Bus 001 Device 010: ID 045e:097b Microsoft Corp. Generic Hub [Azure Kinect]
Bus 001 Device 011: ID 045e:097e Microsoft Corp. Azure Kinect Microphone Array
Bus 002 Device 014: ID 045e:097a Microsoft Corp. Generic Superspeed Hub [Azure Kinect]
Bus 002 Device 015: ID 045e:097c Microsoft Corp. Azure Kinect Depth Camera
Bus 002 Device 016: ID 045e:097d Microsoft Corp. Azure Kinect 4K Camera

3.4 4K Camera Is Working Fine Like An Independent UVC Camera

1
2
3
4
5
6
7
8
$ v4l2-ctl --list-devices

Azure Kinect 4K Camera (usb-0000:05:00.3-2.1):
/dev/video1
/dev/video2
/dev/media0

Cannot open device /dev/video0, exiting.

It can be simpilied decmonstrated that GUVCView is able to successfully detect Bus 002 Device 016: ID 045e:097d Microsoft Corp. Azure Kinect 4K Camera as a UVC Camera and carry out the real-time streaming.

4. libdepthengine.so

Currently, Azure Kinect DK supports Ubuntu 18.04 ONLY. However, the libdepthengine.so.2.0 is working fine under my Ubuntu 24.04, which can be extracted directly from the package libk4a1.4_1.4.2_amd64.deb. Save libdepthengine.so.2.0 under any $PATH, and create a symbolic link if you want to.

5. Build Azure-Kinect-Sensor-SDK Longer Vision From Source

Don’t use the original Azure-Kinect-Sensor-SDK. Use Azure-Kinect-Sensor-SDK Longer Vision instead.

6. Python Wrappers for Azure-Kinect-Sensor-SDK

Wrappers Description
k4a Python wrapper the official Python Wrapper with Azure-Kinect-Sensor-SDK
pyk4a a third-party Python Wrapper

7. The Bug: replace_sample(). capturesync_drop, releasing capture early due to full queue TS

Please refer to the following 2 Github issues:

Again, amazing China. This time, Nanjing. What a city!!!!!

Ancient Qinhuai @ Qinhuai River, Nanjing Ancient Qinhuai @ Qinhuai River, Nanjing
GuQinHuai 01 GuQinHuai 02
WanQingLou LiXiangJun
Lights 01 Lights 02
Lights BaiLuZhou QinHuaiRenJia 01
QinHuaiRenJia 02 QinHuaiRenJia 03
NanjingSouth High Speed Train Railway Station Exterior NanjingSouth High Speed Train Railway Station Inside
Nanjing Old Station Renovated ChangYu Noodle

Raspberry Pi has already well-established a comprehensive community support world wide. Among ALL the operating systems supporting Raspberry Pi, Home Assistant stands out, shining brightly as an open-source home automation operating system in the field of IoT.

1. Overview

1.1 What is Home Assistant?

Home Assistant is the ``open source home automation that puts local control and privacy first. Powered by a worldwide community of tinkerers and DIY enthusiasts. Perfect to run on a Raspberry Pi or a local server.’’ (cited from first page of the official website of Home Assistant.)

Home Assistant OS is a minimal operating system optimized for running Home Assistant, and it does not use traditional package management systems like apt. Instead, it relies on Docker containers to manage and run services, including Home Assistant and its add-ons.

1.2 Web UI

After simply flashed Home Assistant OS 12.3 onto a 16GB TF card and insert it into my old Raspberry Pi 3B V1.2, we can now take a brief look at Home Assistant web UI at http://homeassistant.local:8123/.

Home Assistant Overview Home Assistant Dashboard
Home Assistant Overview Home Assistant Dashboard
Home Assistant Terminal SSH Home Assistant Run Terminal SSH

2. ssh Into Home Assistant OS

2.1 Configuration

2.1.1 Installation of Home Assistant Add-on Terminal & SSH

Please have the add-on Home Assistant Add-on Terminal & SSH installed from Home Assistant -> Settings -> Add-ons.

2.1.2 Edit in YAML

Please do remember to add these 2 lines:

1
2
3
ssh:
enable: true
port: 22

at the end of the YAML configuration:

1
2
3
4
5
6
7
8
authorized_keys: []
password: []
apks: []
server:
tcp_forwarding: false
ssh:
enable: true
port: 22

2.2 ssh Into Home Assistant OS From Remote Desktop

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
➜  ~ ssh root@192.168.1.83
The authenticity of host '192.168.1.83 (192.168.1.83)' can't be established.
ED25519 key fingerprint is SHA256:MZDkXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXJmxE.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.1.83' (ED25519) to the list of known hosts.

▄██▄ _ _
▄██████▄ | | | | ___ _ __ ___ ___
▄████▀▀████▄ | |_| |/ _ \| '_ ` _ \ / _ \
▄█████ █████▄ | _ | (_) | | | | | | __/
▄██████▄ ▄██████▄ |_| |_|\___/|_| |_| |_|\___| _
████████ ██▀ ▀██ / \ ___ ___(_)___| |_ __ _ _ __ | |_
███▀▀███ ██ ▄██ / _ \ / __/ __| / __| __/ _` | '_ \| __|
██ ██ ▀ ▄█████ / ___ \\__ \__ \ \__ \ || (_| | | | | |_
███▄▄ ▀█ ▄███████ /_/ \_\___/___/_|___/\__\__,_|_| |_|\__|
▀█████▄ ███████▀

Welcome to the Home Assistant command line.

System information
IPv4 addresses for enu1u1: 192.168.1.83/24
IPv6 addresses for enu1u1: 2001:569:5993:5900:1b4a:e1e6:140c:aac0/64, fe80::ab1f:8f98:b81:a479/64
IPv4 addresses for wlan0:
IPv4 addresses for wlu1u4:

OS Version: Home Assistant OS 12.3
Home Assistant Core: 2024.5.5

Home Assistant URL: http://homeassistant.local:8123
Observer URL: http://homeassistant.local:4357
[core-ssh ~]$

3. ESPHome with Home Assistant

One of my very old blogs LVT ESP32-Cam has talked about ESP32 Camera. Today, I found that blog posted 4 years ago before COVID-19, and I’m trying to integrate some of my [ESP32 Cameras] into Home Assistant. Namely, use Home Assistant running on the Raspberry Pi to manage a couple of my ESP32 Cameras.

3.1 ESP32 Boards Management

Please ensure the following 2 boards support have been installed successfully.

Arduino IDE Esp32

3.2 CameraWebServer

Current ESP32 Examples provide an webcam server example: ESP32 Example CameraWebServer. Or, we can simply open up the example from within Arduino IDE: File->Examples->ESP32->Camera->CameraWebServer.

3.2.1 Four ESP32 Cameras

Both are compatible with the configuration CAMERA_MODEL_AI_THINKER in ESP32 Example CameraWebServer. Therefore, three places are to be correctly setup:

  • #define CAMERA_MODEL_AI_THINKER // Has PSRAM
  • const char *ssid = "XXXXXXXXX";
  • const char *password = "YYYYYYYYY";

3.2.2 Verify and Upload

Verify

Upload

3.2.3 Demonstration

CameraWebServer

However, as you can see, the captured image is not quite stable. And, I did find the WiFi connection is easy to get disconnected. What to do???

[!NOTE:] What to do???
Please continue reading.

3.3 RTSP with MJPEG

As of today, June 4, 2024, there is NO H.264 or H.265 streaming server support. Therefore, I’m going to follow the following 2 blogs and build up RTSP server with MJPEG video streams.

3.4 Integrate ESPHome Into Home Assistant

Getting Started with ESPHome and Home Assistant

3.5 Demonstration

ESPHome ESPHome
ESPHome Build Failed ESPHome Compile
ESPHome Device Discovered ESPHome Device Info
ESPHome Devices ESPHome Flash
ESPHome Installation ESPHome Install Connecting Hang
ESPHome Installed 3232 Connection Refused ESPHome Visit
Home Assistant Overview Home Assistant Media Live
ESPHome ESP32Cam Configured ESPHome ESP32Cam Integrated

I got a LicheeRV Nano before I left China for Canada a couple of weeks ago. Nowadays China is amazing. Let’s take a look at my hometown Wuhan of year 2024.

Sunset @ Yangtze River 2nd Bridge Sunset @ Yangtze River TianXingZhou Bridge
Yangtze River 2nd Bridge Sunset Yangtze River TianXingZhou Bridge Sunset
Yangtze River Grove Sunset Yangtze River Embankment Grassland Constructions
YuChiYuZui 01 YuChiYuZui 02

Now, let’s start our today’s topic: LicheeRV Nano.

1. Introduction

1.1 First Look

LicheeRV Nano LicheeRV Nano with N2425D
LicheeRV Nano 01 LicheeRV Nano 01

1.2 Official Documentation

Actually, ALL you need can be found at:

1.3 LicheeRV-Nano-WE

According to the LicheeRV Nano Version Comparison, there are 4 versions totally.

  • LicheeRV-Nano-B
  • LicheeRV-Nano-E
  • LicheeRV-Nano-W
  • LicheeRV-Nano-WE

Clearly, the one I purchased is a LicheeRV-Nano-WE.

2. lsusb

1
2
3
4
➜  ~ lsusb
......
Bus 009 Device 010: ID 3346:1009 sipeed licheervnano
......

3. WiFi Setup

After LicheeRV Nano is connected to your host computer via USB Type-C, enter the mounted rootfs and add the following new file:

1
2
3
4
5
➜  rootfs sudo vim ./etc/network/wpa_supplicant.conf
network={
ssid="XXXXX"
psk="YYYYY"
}

Then, reboot. Now, you should be able to find out your LicheeRV Nano by sudo nmap -sN 192.168.1.254/24.

4. LicheeRV Nano Overview

4.1 ssh Into LicheeRV Nano

Both the default username and password of LicheeRV Nano are root.

1
2
3
4
5
6
7
8
➜  ~ ssh root@192.168.1.78
root@192.168.1.78's password:

# uname -a
Linux licheervnano-8c93 5.10.4-tag- #1 PREEMPT Mon Apr 22 11:13:44 HKT 2024 riscv64 GNU/Linux

# hostname
licheervnano-8c93

4.2 CPU and Memory

Clearly, LicheeRV Nano is armed with only 1 single Risc-V 64bit CPU, and about 160M flash memory.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
# cat /proc/cpuinfo
processor : 0
hart : 0
isa : rv64imafdvcsu
mmu : sv39

# cat /proc/meminfo
MemTotal: 162672 kB
MemFree: 98276 kB
MemAvailable: 128976 kB
Buffers: 3416 kB
Cached: 28776 kB
SwapCached: 0 kB
Active: 11792 kB
Inactive: 26892 kB
Active(anon): 144 kB
Inactive(anon): 6592 kB
Active(file): 11648 kB
Inactive(file): 20300 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 4 kB
Writeback: 0 kB
AnonPages: 6508 kB
Mapped: 10092 kB
Shmem: 248 kB
KReclaimable: 5164 kB
Slab: 13988 kB
SReclaimable: 5164 kB
SUnreclaim: 8824 kB
KernelStack: 1248 kB
PageTables: 524 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 81336 kB
Committed_AS: 9868 kB
VmallocTotal: 67108863 kB
VmallocUsed: 5468 kB
VmallocChunk: 0 kB
Percpu: 64 kB
CmaTotal: 0 kB
CmaFree: 0 kB

4.3 Python

By default, Python 3.11.6 has been installed on LicheeRV Nano.

1
2
# python --version
Python 3.11.6

5. LicheeRV-Nano-Build

The default LicheeRV-Nano-Build README.md seems to be problematic. I did it in my way:

5.1 Clone LicheeRV-Nano-Build

1
2
3
➜  git clone https://github.com/sipeed/LicheeRV-Nano-Build --depth=1
cd LicheeRV-Nano-Build
➜ LicheeRV-Nano-Build git:(main)

5.2 Setup

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
➜  LicheeRV-Nano-Build git:(main) ✗ source build/cvisetup.sh
/proc/self/fd/19:7: cv1800b_board_sel: assignment to invalid subscript range
-------------------------------------------------------------------------------------------------------
Usage:
(1) menuconfig - Use menu to configure your board.
ex: $ menuconfig

(2) defconfig $CHIP_ARCH - List EVB boards($BOARD) by CHIP_ARCH.
** sg200x ** -> ['sg2000', 'sg2002']
** cv181x ** -> ['cv1812cp', 'cv1812h', 'cv1813h']
** cv180x ** -> ['cv1800b']
ex: $ defconfig cv181x

(3) defconfig $BOARD - Choose EVB board settings.
ex: $ defconfig cv1813h_wevb_0007a_spinor
ex: $ defconfig cv1812cp_wevb_0006a_spinor
-------------------------------------------------------------------------------------------------------

5.3 GUI-based menuconfig

I personally prefer the GUI-based configuration.

5.3.1 Copy menuconfig.py

1
2
3
➜  LicheeRV-Nano-Build git:(main) ✗ cd build/scripts 
➜ scripts git:(main) ✗ cp menuconfig.py .py
➜ scripts git:(main) ✗

5.3.2 Modify ./build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
diff --git a/build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig b/build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig
index 4138f665b..58a5ea02d 100644
--- a/build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig
+++ b/build/boards/sg200x/sg2002_licheervnano_sd/sg2002_licheervnano_sd_defconfig
@@ -2,12 +2,12 @@ CONFIG_CHIP_sg2002=y
CONFIG_BOARD_licheervnano_sd=y
CONFIG_DDR_CFG_ddr3_1866_x16=y
CONFIG_ARCH="riscv"
-CONFIG_CROSS_COMPILE="riscv64-unknown-linux-musl-"
+CONFIG_CROSS_COMPILE="riscv64-linux-gnu-"
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_KERNEL_ENTRY_HACK=y
CONFIG_KERNEL_ENTRY_HACK_ADDR="0x80200000"
CONFIG_TOOLCHAIN_MUSL_RISCV64=y
-CONFIG_FLASH_SIZE_SHRINK=y
+# CONFIG_FLASH_SIZE_SHRINK=y
CONFIG_BOOT_IMAGE_SINGLE_DTB=y
# CONFIG_FLASH_SIZE_SHRINK is not set
CONFIG_CP_EXT_WIRELESS=y
@@ -16,7 +16,7 @@ CONFIG_MIPI_PANEL_ZCT2133V1=y
CONFIG_SENSOR_GCORE_GC4653=y
CONFIG_SENSOR_OV_OS04A10=y
CONFIG_UBOOT_2021_10=y
-CONFIG_KERNEL_SRC_5.10=y
+CONFIG_KERNEL_SRC="linux_5.10"
CONFIG_KERNEL_UNCOMPRESSED=y
# CONFIG_SKIP_RAMDISK is not set
CONFIG_SENSOR_TUNING_PARAM_cv181x_src_gcore_gc4653=y

By the way, even until now, I’ve got NO idea what the MIPI_PANEL should be?

5.3.3 Run menuconfig

1
➜  LicheeRV-Nano-Build git:(main) ✗ menuconfig

LicheeRV Nano Build menuconfig

5.4 Build

5.4.1 Exports

1
2
3
➜  LicheeRV-Nano-Build git:(main) ✗ export IMGTOOL_PATH=tools/common/image_tool                      
➜ LicheeRV-Nano-Build git:(main) ✗ export COMMON_TOOLS_PATH=tools/common
➜ LicheeRV-Nano-Build git:(main) ✗ export FLASH_PARTITION_XML=boards/default/partition/partition_none.xml

Happy Chinese Dragon Year !!! Today, let’s continue this SBC: Milk-V Duo 256M.

1. Some Suggestions

  • Auto resizing in order to use the full TF card). So far, I’ll have to do it manually. Anyway, now I can use my full TF card as:
1
2
3
4
5
6
7
[root@milkv-duo]~# df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 28.5G 146.0M 27.2G 1% /
devtmpfs 83.1M 0 83.1M 0% /dev
tmpfs 83.1M 0 83.1M 0% /dev/shm
tmpfs 83.1M 48.0K 83.1M 0% /tmp
tmpfs 83.1M 32.0K 83.1M 0% /run

Without resizing, as in my previous blog Milk-V Duo 256M - 01

1
2
3
4
5
6
7
[root@milkv-duo]~# df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 752.0M 152.6M 557.0M 22% /
devtmpfs 83.1M 0 83.1M 0% /dev
tmpfs 83.1M 0 83.1M 0% /dev/shm
tmpfs 83.1M 48.0K 83.1M 0% /tmp
tmpfs 83.1M 32.0K 83.1M 0% /run
  • Fix eth0’s MAC address. For the wired connection on this particular small SBC Milk-V Duo 256M, I’d strongly recommend to use the physical MAC address for eth0, rather than a dynamic one. Whenever rebooting, Milk-V Duo 256M will change its MAC address, as well as the IP address, which is seriously NOT what I expected. Therefore, what we need to do is to modify file /etc/network/interfaces as suggested by StackExchange issue Change Mac Address permanently inside /etc/network/interfaces:
1
2
3
4
5
6
auto eth0
iface eth0 inet static
address 192.168.1.51
netmask 255.255.255.0
gateway 192.168.1.1
hwaddress ether 00:11:22:33:44:55

-

2. Yolo5 Example Based on TPU

Finally, I got duo-buildroot-sdk built today. So, yes, happy Chinese new year of Dragon, 2024. Let’s have some fun of Milk-V Duo 256M.

1. Introduction

From Milk-V Duo 256M‘s official website, the following picture clearly shows it has

MilkV Duo 256M Core chips

The detailed specifications are also highlighted in the table on the official website of Milk-V Duo 256M.

2. Build duo-buildroot-sdk

2.1 Failed to build duo-buildroot-sdk using Ubuntu 22.04

Even if I strictly followed the manual on duo-buildroot-sdk‘s official documentation Compiled using Ubuntu 22.04, I still failed to build an image out for flashing, with the following error messages inside the log file build/br.log:

1
2
3
4
5
6
7
8
......
2024-02-08T13:34:11 tic: symbol lookup error: tic: undefined symbol: _nc_disable_period
2024-02-08T13:34:11 ? tic could not build ....../duo-buildroot-sdk-Duo-V1.0.8/buildroot-2021.05/output/milkv-duo256m_musl_riscv64/per-package/ncurses/host/riscv64-buildroot-linux-musl/sysroot/usr/share/terminfo
2024-02-08T13:34:11 make[4]: *** [Makefile:92: install.data] Error 1
2024-02-08T13:34:12 make[3]: *** [Makefile:122: install] Error 2
2024-02-08T13:34:12 make[2]: *** [package/pkg-generic.mk:289: ....../duo-buildroot-sdk-Duo-V1.0.8/buildroot-2021.05/output/milkv-duo256m_musl_riscv64/build/ncurses-6.1/.stamp_staging_installed] Error 2
2024-02-08T13:34:12 make[2]: *** Waiting for unfinished jobs....
......

A possible solution is found at: https://lore.kernel.org/buildroot/bug-15685-163-ecW1EgIyRW@https.bugs.busybox.net%2F/T/.

2.2 Successfully build duo-buildroot-sdk using Docker

By strictly following the manual on duo-buildroot-sdk‘s official documentation Compiled using Docker, I now am able to build an image out as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
......
mkfs.fat: Warning: lowercase labels might not work properly on some systems
INFO: vfat(boot.vfat): adding file 'fip.bin' as 'fip.bin' ...
INFO: vfat(boot.vfat): cmd: "MTOOLS_SKIP_CHECK=1 mcopy -sp -i '~/install/soc_cv1812cp_milkv_duo256m_sd/boot.vfat' '~/install/soc_cv1812cp_milkv_duo256m_sd/fip.bin' '::'" (stderr):
INFO: vfat(boot.vfat): adding file 'rawimages/boot.sd' as 'rawimages/boot.sd' ...
INFO: vfat(boot.vfat): cmd: "MTOOLS_SKIP_CHECK=1 mcopy -sp -i '~/install/soc_cv1812cp_milkv_duo256m_sd/boot.vfat' '~/install/soc_cv1812cp_milkv_duo256m_sd/rawimages/boot.sd' '::'" (stderr):
INFO: ext4(rootfs.ext4): cmd: "genext2fs -d '~/install/soc_cv1812cp_milkv_duo256m_sd/tmp/root' --size-in-blocks=786432 -i 16384 '~/install/soc_cv1812cp_milkv_duo256m_sd/rootfs.ext4' " (stderr):
copying from directory ~/install/soc_cv1812cp_milkv_duo256m_sd/tmp/root
INFO: ext4(rootfs.ext4): cmd: "tune2fs -O 'extents,uninit_bg,dir_index,has_journal' '~/install/soc_cv1812cp_milkv_duo256m_sd/rootfs.ext4'" (stderr):
INFO: ext4(rootfs.ext4): cmd: "tune2fs -L 'rootfs' '~/install/soc_cv1812cp_milkv_duo256m_sd/rootfs.ext4'" (stderr):
INFO: ext4(rootfs.ext4): cmd: "e2fsck -pvfD '~/install/soc_cv1812cp_milkv_duo256m_sd/rootfs.ext4'" (stderr):
INFO: hdimage(milkv-duo256m.img): adding partition 'boot' (in MBR) from 'boot.vfat' ...
INFO: hdimage(milkv-duo256m.img): adding partition 'rootfs' (in MBR) from 'rootfs.ext4' ...
INFO: hdimage(milkv-duo256m.img): writing MBR
gnimage for milkv-duo256m success!
~/build
~
Create SD image successful: out/milkv-duo256m-20240208-1949.img

And, the last couple of lines in the log file build/br.log are:

1
2
3
4
5
6
7
8
9
10
11
......
2024-02-08T19:48:15
2024-02-08T19:48:15 Allocating group tables: 0/25 done
2024-02-08T19:48:15 Writing inode tables: 0/25 done
2024-02-08T19:48:15 Creating journal (4096 blocks): done
2024-02-08T19:48:15 Copying files into the device: PATH="~/buildroot-2021.05/output/milkv-duo256m_musl_riscv64/host/bin:~/buildroot-2021.05/output/milkv-duo256m_musl_riscv64/host/sbin:~/host-tools/gcc/riscv64-linux-musl-x86_64/bin:~/host-tools/gcc/riscv64-linux-x86_64/bin:~/host-tools/gcc/riscv64-elf-x86_64/bin:~/host-tools/gcc/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-elf/bin:~/host-tools/gcc/gcc-linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf/bin:~/host-tools/gcc/gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:~/host-tools/gcc/arm-cvitek-linux-uclibcgnueabihf/bin" xz -9 -C crc32 -c -T 9 ~/buildroot-2021.05/output/milkv-duo256m_musl_riscv64/images/rootfs.tar > ~/buildroot-2021.05/output/milkv-duo256m_musl_riscv64/images/rootfs.tar.xz
2024-02-08T19:48:16 done
2024-02-08T19:48:16 Writing superblocks and filesystem accounting information: 0/25 done
2024-02-08T19:48:16
2024-02-08T19:48:16 ln -sf rootfs.ext2 ~/buildroot-2021.05/output/milkv-duo256m_musl_riscv64/images/rootfs.ext4
2024-02-08T19:49:28 make[1]: Leaving directory '~/buildroot-2021.05'

3. Boot after Flashing

After flashing the generated out/milkv-duo256m-20240208-1949.img into the SD card, we can now boot up Milk-V Duo 256M by connecting it with my laptop via a USB cable.

3.1 lsusb

1
2
3
......
Bus 001 Device 009: ID 3346:1009 Cvitek RNDIS
......

3.2 SSH into Milk-V Duo 256M via USB Connection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
➜  ~ ssh root@192.168.42.1
The authenticity of host '192.168.42.1 (192.168.42.1)' can't be established.
ED25519 key fingerprint is SHA256:NyuhOwhRh9sfzW+C2D5OIaTIbGCuVI1v1jrJv4DaM+U.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.42.1' (ED25519) to the list of known hosts.
root@192.168.42.1's password:
[root@milkv-duo]~# ls
[root@milkv-duo]~# pwd
/root
[root@milkv-duo]~# uname -a
Linux milkv-duo 5.10.4-tag- #1 PREEMPT Thu Feb 8 19:28:21 CST 2024 riscv64 GNU/Linux
[root@milkv-duo]~# lsb_release -a
-sh: lsb_release: not found
[root@milkv-duo]~# neofetch
-sh: neofetch: not found
[root@milkv-duo]~# ll /
total 33
drwxr-xr-x 18 root root 1024 Feb 8 2024 ./
drwxr-xr-x 18 root root 1024 Feb 8 2024 ../
drwxr-xr-x 2 root root 3072 Feb 8 2024 bin/
drwxr-xr-x 7 root root 1540 Jan 1 00:00 dev/
drwxr-xr-x 8 root root 1024 Jan 1 00:00 etc/
drwxr-xr-x 3 root root 1024 Jan 30 2024 lib/
lrwxrwxrwx 1 root root 3 Feb 8 2024 lib64 -> lib/
lrwxrwxrwx 1 root root 11 Feb 8 2024 linuxrc -> bin/busybox*
drwx------ 2 root root 16384 Feb 8 2024 lost+found/
drwxr-xr-x 2 root root 1024 Jan 30 2024 media/
drwxr-xr-x 6 root root 1024 Jan 30 2024 mnt/
drwxr-xr-x 2 root root 1024 Jan 30 2024 opt/
dr-xr-xr-x 122 root root 0 Jan 1 00:00 proc/
drwx------ 2 root root 1024 Jan 1 00:03 root/
drwxr-xr-x 4 root root 200 Jan 1 00:00 run/
drwxr-xr-x 2 root root 3072 Feb 8 2024 sbin/
dr-xr-xr-x 11 root root 0 Jan 1 00:00 sys/
drwxrwxrwt 3 root root 160 Jan 1 00:00 tmp/
drwxr-xr-x 8 root root 1024 Feb 8 2024 usr/
drwxr-xr-x 5 root root 1024 Feb 8 2024 var/
[root@milkv-duo]~# df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 752.0M 152.6M 557.0M 22% /
devtmpfs 83.1M 0 83.1M 0% /dev
tmpfs 83.1M 0 83.1M 0% /dev/shm
tmpfs 83.1M 48.0K 83.1M 0% /tmp
tmpfs 83.1M 32.0K 83.1M 0% /run
[root@milkv-duo]~# cat /proc/cpuinfo
processor : 0
hart : 0
isa : rv64imafdvcsu
mmu : sv39

[root@milkv-duo]~# cat /proc/meminfo
MemTotal: 170288 kB
MemFree: 147812 kB
MemAvailable: 148580 kB
Buffers: 872 kB
Cached: 3024 kB
SwapCached: 0 kB
Active: 2520 kB
Inactive: 2808 kB
Active(anon): 72 kB
Inactive(anon): 1440 kB
Active(file): 2448 kB
Inactive(file): 1368 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 44 kB
Writeback: 0 kB
AnonPages: 1444 kB
Mapped: 2532 kB
Shmem: 80 kB
KReclaimable: 2664 kB
Slab: 7220 kB
SReclaimable: 2664 kB
SUnreclaim: 4556 kB
KernelStack: 1024 kB
PageTables: 404 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 85144 kB
Committed_AS: 5004 kB
VmallocTotal: 67108863 kB
VmallocUsed: 4408 kB
VmallocChunk: 0 kB
Percpu: 32 kB
CmaTotal: 0 kB
CmaFree: 0 kB
[root@milkv-duo]~#

3.3 SSH into Milk-V Duo 256M via Wired Connection

After my clumsy soldering, Milk-V Duo 256M is now able to be connected via my home Intranet. Now, let’s ssh into Milk-V Duo 256M via its IP address.

1
2
3
4
5
6
7
8
9
10
11
12
13
[root@milkv-duo]~# ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 0a:95:ef:1e:ec:c2 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.51/24 brd 192.168.1.255 scope global dynamic noprefixroute eth0
valid_lft 259082sec preferred_lft 226682sec
3: usb0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 5e:59:d4:f7:99:f7 brd ff:ff:ff:ff:ff:ff
inet 192.168.42.1/24 brd 192.168.42.255 scope global usb0
valid_lft forever preferred_lft forever

4. Camera Test - Face Detection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
[root@milkv-duo]~# cd /bin
[root@milkv-duo]/bin# ls
arch chattr cpio dumpkmap getopt link lsattr mount nuke pwd setarch su uname
ash chgrp date echo grep linux32 mk_cmds mountpoint pidof resume setpriv sync usleep
base32 chmod dd egrep gunzip linux64 mkdir mt ping rm setserial tar vi
base64 chown df false gzip ln mknod mv pipe_progress rmdir sh touch watch
busybox compile_et dmesg fdflush hostname login mktemp netstat printenv run-parts sleep true zcat
cat cp dnsdomainname fgrep kill ls more nice ps sed stty umount
[root@milkv-duo]/bin# camera-test.sh
[SAMPLE_COMM_SNS_ParseIni]-1950: Parse /mnt/data/sensor_cfg.ini
[parse_source_devnum]-1605: devNum = 1
[parse_sensor_name]-1686: sensor = GCORE_GC2083_MIPI_2M_30FPS_10BIT
[parse_sensor_busid]-1714: bus_id = 2
[parse_sensor_i2caddr]-1725: sns_i2c_addr = 37
[parse_sensor_mipidev]-1736: mipi_dev = 0
[parse_sensor_laneid]-1747: Lane_id = 1, 0, 2, -1, -1
[parse_sensor_pnswap]-1758: pn_swap = 0, 0, 0, 0, 0
MMF Version:7e0cc6a08-musl_riscv64
Create VBPool[0], size: (3110400 * 3) = 9331200 bytes
Create VBPool[1], size: (3110400 * 3) = 9331200 bytes
Create VBPool[2], size: (2764800 * 1) = 2764800 bytes
Total memory of VB pool: 21427200 bytes
Initialize SYS and VB
Initialize VI
ISP Vipipe(0) Allocate pa(0x8c771000) va(0x0x3fbed36000) size(291120)
stSnsrMode.u16Width 1920 stSnsrMode.u16Height 1080 25.000000 wdrMode 0 pstSnsObj 0x3fbf86b400
[SAMPLE_COMM_VI_StartMIPI]-483: sensor 0 stDevAttr.devno 0
awbInit ver 6.8@2021500
0 R:1400 B:3100 CT:2850
1 R:1500 B:2500 CT:3900
2 R:2300 B:1600 CT:6500
Golden 1024 1024 1024
WB Quadratic:0
isWdr:0
ViPipe:0,===GC2083 1080P 30fps 10bit LINE Init OK!===
********************************************************************************
cvi_bin_isp message
gerritId: 36403 commitId: c69c5863e
md5: cab880835a2ad5184de5ed7762404b84
sensorNum 1
sensorName0 2083

PQBIN message
gerritId: 80171 commitId: 5c9d8fc5d
md5: ba5a510e093ad42db6788e6c2d13169e
sensorNum 3
sensorName0 2053

author: wanqiang.he desc: 思博慧CV1812H_GC2083_RGB_mode_V1.0.0
createTime: 2023-08-04 16:48:08version: V1.1
tool Version: v3.0.5.24 mode:
********************************************************************************
sensorName(0) mismatch, mwSns:2083 != pqBinSns:2053
[SAMPLE_COMM_ISP_Thread]-95: ISP Dev 0 running!
Initialize VPSS
---------VPSS[0]---------
Input size: (1920x1080)
Input format: (19)
VPSS physical device number: 1
Src Frame Rate: -1
Dst Frame Rate: -1
--------CHN[0]-------
Output size: (1920x1080)
Depth: 1
Do normalization: 0
Src Frame Rate: -1
Dst Frame Rate: -1
----------------------
--------CHN[1]-------
Output size: (1920x1080)
Depth: 1
Do normalization: 0
Src Frame Rate: -1
Dst Frame Rate: -1
----------------------
------------------------
Bind VI with VPSS Grp(0), Chn(0)
Attach VBPool(0) to VPSS Grp(0) Chn(0)
Attach VBPool(1) to VPSS Grp(0) Chn(1)
Initialize VENC
venc codec: h264
venc frame size: 1920x1080
Initialize RTSP
rtsp://192.168.1.51/h264
prio:0
version: 1.4.0
scrfd768432 Build at 2023-12-25 01:21:44 For platform cv181x
Max SharedMem size:1658880
anchor:-8,-8,8,8
anchor:-16,-16,16,16
bbox:bbox_8_Conv_dequant
landmark:kps_8_Conv_dequant
score:score_8_Sigmoid_dequant
anchor:-32,-32,32,32
anchor:-64,-64,64,64
bbox:bbox_16_Conv_dequant
landmark:kps_16_Conv_dequant
score:score_16_Sigmoid_dequant
anchor:-128,-128,128,128
anchor:-256,-256,256,256
bbox:bbox_32_Conv_dequant
landmark:kps_32_Conv_dequant
score:score_32_Sigmoid_dequant
Enter TDL thread
Enter encoder thread
0 R:1165 B:3087 CT:2688
1 R:1464 B:2327 CT:3937
2 R:1974 B:1613 CT:7225
Golden 1464 1024 2327
wdrLEOnly:1
face count: 0
face count: 0
face count: 0
face count: 0
face count: 0
face count: 0
face count: 0
......

One captured image over rtsp://ipaddress/h264 is shown as follows:

MilkV Duo Streaming

Now, it’s the time to go to bed. Happy Chinese new year’s Eve. The new year’s day of 2024 is February 10, 2024.

Finally, in China, I setup my old laptop.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
➜  ~ neofetch
.-/+oossssoo+/-. lvision@lvision-GT72-6QE
`:+ssssssssssssssssss+:` ------------------------
-+ssssssssssssssssssyyssss+- OS: Ubuntu 22.04.3 LTS x86_64
.ossssssssssssssssssdMMMNysssso. Host: GT72 6QE REV:1.0
/ssssssssssshdmmNNmmyNMMMMhssssss/ Kernel: 6.5.0-15-generic
+ssssssssshmydMMMMMMMNddddyssssssss+ Uptime: 26 mins
/sssssssshNMMMyhhyyyyhmNMMMNhssssssss/ Packages: 3057 (dpkg), 13 (snap)
.ssssssssdMMMNhsssssssssshNMMMdssssssss. Shell: zsh 5.8.1
+sssshhhyNMMNyssssssssssssyNMMMysssssss+ Resolution: 1920x1080
ossyNMMMNyMMhsssssssssssssshmmmhssssssso DE: GNOME 42.9
ossyNMMMNyMMhsssssssssssssshmmmhssssssso WM: Mutter
+sssshhhyNMMNyssssssssssssyNMMMysssssss+ WM Theme: Adwaita
.ssssssssdMMMNhsssssssssshNMMMdssssssss. Theme: Yaru [GTK2/3]
/sssssssshNMMMyhhyyyyhdNMMMNhssssssss/ Icons: Yaru [GTK2/3]
+sssssssssdmydMMMMMMMMddddyssssssss+ Terminal: gnome-terminal
/ssssssssssshdmNNNNmyNMMMMhssssss/ CPU: Intel i7-6700HQ (8) @ 3.500GHz
.ossssssssssssssssssdMMMNysssso. GPU: NVIDIA GeForce GTX 980M
-+sssssssssssssssssyyyssss+- Memory: 3075MiB / 64254MiB
`:+ssssssssssssssssss+:`
.-/+oossssoo+/-.



➜ ~

1. Overview

1.1 What Is ESP32-S3

ESP32-S3 is a dual-core XTensa LX7 MCU, capable of running at 240 MHz. Apart from its 512 KB of internal SRAM, it also comes with integrated 2.4 GHz, 802.11 b/g/n Wi-Fi and Bluetooth 5 (LE) connectivity that provides long-range support. (Cite from Espressif‘s official online doc ESP32-S3).

In this blog, I’m going to heavily follow ESP32-S3-EYE Getting Started Guide.

1.2 Directly Plug-In USB Type C

1
2
3
➜  ~ lsusb | grep Espressif  
Bus 009 Device 034: ID 303a:700f Espressif ESP32-S3-EYE
➜ ~

1.3 Entering Flash Mode

Steps to enter flash mode:

  • Plug out USB Type C
  • Press BOOT button
  • Plug in USB Type C
  • Release BOOT button
1
2
3
➜  ~ lsusb| grep Espressif
Bus 009 Device 035: ID 303a:1001 Espressif USB JTAG/serial debug unit
➜ ~

1.4 Flash Default Firmware esp32-s3-eye-v2.2-firmware-v0.2.0-en.bin

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
➜  v2.2 git:(master) esptool.py --chip esp32s3 --port /dev/ttyACM0 --baud 460800 write_flash -z 0x0 esp32-s3-eye-v2.2-firmware-v0.2.0-en.bin
esptool.py v4.8.1
Serial port /dev/ttyACM0
Connecting...
Chip is ESP32-S3 (QFN56) (revision v0.2)
Features: WiFi, BLE, Embedded PSRAM 8MB (AP_3v3)
Crystal is 40MHz
MAC: ec:da:3b:9d:58:18
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00762fff...
Compressed 7745536 bytes to 4981103...
Wrote 7745536 bytes (4981103 compressed) at 0x00000000 in 48.0 seconds (effective 1292.1 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
➜ v2.2 git:(master)

1.5 Demonstration

ESP-S3-EYE Booting ESP-S3-EYE Camera ESP-S3-EYE Face Detection
Luckfox Pico Ultra W Enable CSI Luckfox Pico Ultra W CSI Enabled Luckfox Pico Ultra W CSI Enabled

2. WiFi

For simplicity, you can just use the Google app ESP BLE Provisioning.