CanMV K230 - (2)

Posted on 2026-05-01 Edited on 2026-04-26 In Single Board Computers , ARM

In the previous blog CanMV K230 - (1), I didn’t finish implementing the remote display of the AI detection results. The issue lies in the AI transmission protocol. Clearly, what we need to transmit includes:

the raw data stream,
the AI detection/recognition results,
and possibly control messages as well.

In theory, the overall protocol should consist of these three parts. In this blog, I will implement a simple custom AI transmission protocol, designed to align with industry standards.

CanMV K230 - (1)

Posted on 2026-04-14 Edited on 2026-04-26 In Single Board Computers , ARM

1. Introduction

A variety of CanMV SBCs are manufactured upon Kendryte‘s K230, which is based on th design of T-Head‘s XuanTie C908.

1.1 Online Resource

1.2 LCKFB Lushan Pi

Several K230 manufacturers including LCKFB have each designed their own development boards. I purchased the LCKFB Lushan Pi developed and produced by Lichuang (JLC). See the figure below for details:

Clearly, LCKFB Lushan Pi provides 3 MIPI interfaces, and I have 3 GC20X3 MIPI cameras set up for it.

2. Connect LCKFB Lushan Pi and PC Using A Type-C USB Cable

2.1 Flash

For simplicity, I chose CanMV_K230_LCKFB_micropython_v1.6-0-ge4441cc_nncase_v2.9.0.img.gz as the initial firmware image for flashing.

2.2 `lsusb`

➜  ~ lsusb
...
Bus 009 Device 016: ID 1209:abd1 Generic OpenMV Cam
...

Oh my God… this CanMV is actually using an OpenMV camera module??? (Please refer to my ancient blog OpenMV)

2.3 `ll`

1 2	➜ ~ ll /dev/ttyACM* crw-rw---- 1 root dialout 166, 0 Apr 20 21:29 /dev/ttyACM0

2.4 From Serial Port

However, it looks it does NOT contain a demo using 3 cameras.

In order to demonstrate the following two examples provided at k230_rtsmart_examples, we need to either flash a different firmware K230 RTOS SDK or build these examples using triple MIPI cameras based on the environment provided by the firmware K230_LCKFB_rtsmart.

triple_camera_ai: need a display connected to K230 to demonstrate
triple_camera_ai_rtsp: have to rebuild the source from code

3. Connect LCKFB Lushan Pi and PC Using Serial Cables and A USB-to-TTL Convertor

3.1 Flash

This time, RtSmart-K230_LCKFB_rtsmart_v0.7-0-g0946d74_nncase_v2.9.0.img.gz is flashed onto LCKFB Lushan Pi.

3.2 Cable Connection

Please remember the following 3 points:

You need a type-C USB cable (the white cable in the picture) used as the power of LCKFB Lushan Pi
You need a USB-to-TTL (for instance: CH341 or PL2303) for communication via serial port between LCKFB Lushan Pi and your PC
You need to ensure the RED cable (for power as well) is disconnected. Otherwise, the USB-to-TTL is going to be overheated.

3.3 `lsusb`

➜  ~ lsusb
......
Bus 009 Device 049: ID 1a86:7523 QinHeng Electronics CH340 serial converter
Bus 009 Device 052: ID 1209:abd1 Generic OpenMV Cam
......

3.4 `ll`

➜  ~ ll /dev/ttyACM* 
crw-rw---- 1 root dialout 166, 0 Apr 20 21:32 /dev/ttyACM0
➜  ~ ll /dev/ttyUSB*
crw-rw---- 1 root dialout 188, 0 Apr 20 21:32 /dev/ttyUSB0
➜  ~

As you can see, in my case, I’m using a CH340.

3.5 List Applications In Serial

Well, this time, we can see triple_camera_ai is already built in the firmware RtSmart-K230_LCKFB_rtsmart_v0.7-0-g0946d74_nncase_v2.9.0.img.gz and flashed into LCKFB Lushan Pi with binary .elf. However, without a display directly connected to my LCKFB Lushan Pi, it’s better to demonstrate with display on a remote PC. Therefore, let’s build triple_camera_ai_rtsp from source.

3.6 Demonstrate triple_camera_ai

4. Build triple_camera_ai_rtsp From Source

4.1 Environmental Configuration

Please refer to K230 RTOS How to Build.

What I’ve done are shown as the following screenshots:

Prepare the source code using repo	k230_rtos_lckfb_defconfig

Deselect CanMV Components Configuration but Select UserSpace Examples Configuration	Enable Auto Detect DRAM Size

Enable Build Triple Camera AI Programs	Entire Process of Configuration

4.2 Build K230 RTOS SDK

4.2.1 Toolchain Preparation

The most up-to-date K230 toolchain can be found on Kendryte Download

4.2.2 Build

The reason why we have to build K230 RTOS SDK is because:

it generates some files required by those UserSpace examples, including triple_camera_ai_rtsp.
the built firmware RtSmart-K230_LCKFB_rtsmart_local_nncase_v2.11.0.img is indeed different from the official release RtSmart-K230_LCKFB_rtsmart_v0.7-0-g0946d74_nncase_v2.9.0.img.gz.

4.3 triple_camera_ai_rtsp Demonstration

4.3.1 RTL8152B USB To Wired Converter

4.3.1.1 Find Out IP of Realtek Device Using `nmap`

4.3.1.2 Network Interface On K230 Using `ifconfig`

4.3.1.3 Demonstrate MPP Example sample_rtspserver

4.3.4 triple_camera_ai_rtsp Demonstration

Clearly, ISP2 (camera2) detects my face as a single object of interest.

OpenClaw On A DshanPi-A1 (RK3576)

Posted on 2026-04-13 In Single Board Computers , ARM

DshanPi-A1 has been out for quite a while. Some rather baffling legal matters have slowed down my pace of writing blog posts, but they cannot hinder the steady progress of scientific research in China. Today, let’s have some fun exploring this SBC DshanPi-A1.

1. Wiki

There is a wealth of documentation and learning materials available online. Here, a few examples are provided as:

2. Flash Armbian for DshanPi-A1

2.1 Preparation

Armbian for DshanPi-A1: 3 options are provided on Armbian‘s website. I chose Debian 13 for my DshanPi-A1.
rkdeveloptool: manually built from source on my host desktop
rk3576_spl_loader_v1.09.107.bin: directly downloaded from DshanPi-A1 On 100ask

2.2 Flash In 3 Steps

2.2.1 Step 1: Download Bootloader

1
2
3

➜  DshanPi sudo rkdeveloptool db rk3576_spl_loader_v1.09.107.bin
[sudo] password for lvision: 
Downloading bootloader succeeded.

2.2.2 Step 2: Flash OS

1
2
3

➜  Armbian sudo rkdeveloptool wl 0 Armbian_community_26.2.0-trunk.703_Dshanpi-a1_trixie_vendor_6.1.115_minimal.img
[sudo] password for lvision: 
Write LBA from file (100%)

2.2.3 Step 3: Reset DshanPi-A1

1 2	➜ Armbian sudo rkdeveloptool rd Reset Device OK.

3. Boot And Have Fun

I’m NOT going to repeat those duplicate process for every SBC, but simply demonstrate it directly.

Armbian for DshanPi-A1	DshanPi-A1 has 32G eMMC

4. OpenClaw Installation

Clearly, this OpenClaw is still far too dangerous. Without the necessary expertise, I would not take on such tasks. This blog post ends here and will not continue further.

Maix Bit 2 - WiFi

Posted on 2026-01-26 Edited on 2026-04-13 In Single Board Computers , Risc-V

In my previous blog about Maix Bit, we covered camera sensor, lcd display, and SPI with logic analyzer. However, WiFi is NEVER talked about.

1. Cable Connection

1.1 Pin Definitions of ESP32-C3 Super Mini

Cited from First Look at the Super Mini ESP32-C3:

1.2 Sipeed MaixBit Datasheet

Please refer to:

The key thing to keep in mind is: pin 26, 27, 28, 29 are reserved for TF card by default. Cited from Sipeed MaixBit Datasheet V2.0 as:

Maix Bit V2.0 Slik	K210 IO	Function	Remark
26	IO26	SPI0_MISO	Reserved for TF Card
27	IO27	SPI0_SCLK	Reserved for TF Card
28	IO28	SPI0_MOSI	Reserved for TF Card
29	IO29	SPI0_CS0	Reserved for TF Card

Therefore, in my test, I’m going to use the pins 22,23,24,25 and 21 (optional) instead, and summarized in the NEXT section.

1.3 Wired Connection Between Maix Bit board and ESP32-C3 Super Mini

Signal	Maix Bit	Direction	ESP32-C3 Super Mini	Description	Typical Wire Color	Notes
GND	GND	Shared	GND	Ground reference	Black	Must be common between boards
SCLK (SCK)	22	==>	4	SPI clock	yellow	Clock lines are often yellow
MISO	23	<==	5	Master In, Slave Out	green	Data from ESP32-C3 Super Mini to MaixBit
MOSI	24	==>	6	Master Out, Slave In	blue	Data from MaixBit to ESP32-C3 Super Mini
CS (SS)	25	==>	7	Chip Select (active low)	white	One CS per SPI slave
RDY	21	<==	10	Ready / Flow-control signal	purple	Optional but recommended

2. Hardware Info

2.1 `lsusb`

1 2	Bus 009 Device 033: ID 303a:1001 Espressif USB JTAG/serial debug unit Bus 009 Device 034: ID 0403:6001 Future Technology Devices International, Ltd FT232 Serial (UART) IC

2.2 `kflash` and `esptool`

Clearly,

/dev/ttyUSB0: Maix Bit
/dev/ttyACM0: ESP32-C3 Super Mini

3. Flash

3.1 kflash MaixPy-v1 onto Maix Bit

3.2 `esptool` onto ESP32-C3 Super Mini

`idf.py set-target esp32c3`	partitions.csv

`idf.py build`	flash and monitor

4. Code

Github Repo: maixbit_esp32c3_spi_jpeg_udp
On Maix Bit: k210 main.py
On ESP32-C3 Super Mini: ESP32-C3 app_main.c
On my PC: udp_jpeg_opencv_display.py

5. Video Demonstration

MaixDuino 4 - AI

Posted on 2026-01-26 Edited on 2026-04-13 In Single Board Computers , Risc-V

Extend my previous three blogs about MaixDuino

, let’s talk about the AI features of MaixDuino today.

1. References to Read

MaixDuino 3 - WiFi

Posted on 2026-01-21 Edited on 2026-04-13 In Single Board Computers , Risc-V

1. Breif Summary

In my previous two MaixDuino blog posts MaixDuino 1 and MaixDuino 2, flashing was done using the Arduino IDE. In hindsight, this was neither necessary nor particularly convenient. Especially after flashing the Sipeed official firmware MaixPy-v1 onto the MaixDuino board, there is no real need to rely on the Arduino IDE anymore.

This blog post continues the MaixDuino exploration and completes two experiments:

A very simple implementation of dual-camera image capture on the MaixDuino board, with alternating display on the onboard LCD screen (Refer to LCD API), without relying on Arduino IDE at all;
Streaming the stereo camera images over the MaixDuino’s onboard Wi-Fi to a remote display endpoint.

2. Cable Connection

Maix Bit board Connected to a ESP32-C3 Super Mini	MaixDuino board Already Embedded a ESP32-WROOM-32

3. Stream From Two Cameras

3.1 Flash MaixPy-v1 Onto `/dev/ttyUSB0`

3.2 Flash Maixduino_esp32_fimware Onto `/dev/ttyUSB1` (Optional)

Clearly, there is a typo here Maixduino_esp32_fimware
Since this is Optional, I just have this step ignored for now.

3.3 Python Scripts For Streaming Two Cameras

The full Python scripts of streaming from 2 cameras can be found in my this Github Repo maixduino_stereocam_esp32:

3.4 Demonstration on LCD

One Image Captured From Left Camera	One Image Captured From Right Camera

4. WiFi

None of my previous 3 blogs talked about WiFi.

MaixDuino‘s Sipeed-M1 uses the FIXED hard SPI1 to communicate with its onboard ESP32-WROOM-32, where you need to use network.ESP32_SPI for communication via WiFi, as follows:

nic = network.ESP32_SPI(
            cs=gh(gpiohs["cs"]),
            rst=gh(gpiohs["rst"]),
            rdy=gh(gpiohs["rdy"]),
            mosi=gh(gpiohs["mosi"]),
            miso=gh(gpiohs["miso"]),
            sclk=gh(gpiohs["sclk"]),
            spi=1,
        )

4.1 Python Scripts

4.2 Demonstration

4.2.1 MaixPy REPL

4.2.2 Camera with LCD

4.2.3 On Remote PC

JeVois-Pro Deep Learning Smart Camera - 2

Posted on 2026-01-05 Edited on 2026-01-06 In Machine Vision , ARMLinux

How time flies. It is ALREADY 2026.

I had never realized before that the JeVois-Pro actually has such powerful computing capability. Today, while digging through some of my old gears, I suddenly came to realize that in addition to the 5 TOPs of computing capability on the SoC itself, my JeVois-Pro also has a Google Coral M.2 Accelerator A+E key installed, adding another 4 TOPs of computing power.

From my earlier blog about JeVois-Pro, it’s clear that I never really took full advantage of the computing power of the JeVois-Pro, and I also didn’t succeed in turning the JeVois-Pro into an RTSP server. However, in this blog post, I will carry out a comprehensive test.

1. JeVois-Pro Hardware

Top View	Bottom View

JeVois-Pro Camera IMX290	Google Coral M.2 Accelerator A+E key

JeVois-Pro Assembled

2. Some Command Lines

2.1 `neofetch`

2.2 `lscpu`

2.3 `lspci`

2.4 `TFLite` is now LiteRT

3.

NVidia Jetson AGX Xavier - Deepstream Live Vision Misc (3)

Posted on 2025-12-15 Edited on 2025-12-26 In AI , Edge Computing

I’m back from China and now in Greater Vancouver, celebrating Christmas. 😍❤️😊

Today, let me continue my second blog post about Jetson AGX Xavier.

1. About This Project

The following two illustrations were generated with the assistance of DeepSeek.

1.1 Complete System Architecture

flowchart TD
    A[YouTube Live Stream] --> B[FFmpeg RTMP Receiver]
    B --> C[RTSP Stream rtsp://127.0.0.1:8554/yt]
    
    C --> D[Mediamtx RTSP Server]
    
    D --> E[Original Stream HLS Conversion]
    E --> F[Original HLS Stream<br>/hls/yt/]
    
    D --> G{AI Processing Branch}
    
    G --> H[DeepStream AI Processing]
    H --> I[AI Processed RTSP Stream<br>rtsp://127.0.0.1:8554/ai]
    
    I --> J[AI Stream HLS Conversion]
    J --> K[AI HLS Stream<br>/hls/ai/]
    
    F --> L[Nginx Web Server]
    K --> L
    
    L --> M[User Access<br>https://live.visionmisc.com]
    
    subgraph "Systemd Service Management"
        S1[mediamtx.service]
        S2[yt-hls-converter.service]
        S3[deepstream-ai.service]
        S4[ai-hls-converter.service]
        S5[nginx.service]
    end
    
    S1 --> D
    S2 --> E
    S3 --> H
    S4 --> J
    S5 --> L

1.2 System Features Summary

mindmap
  root(Real-time AI Video Processing System)
    
    Architecture Features
      Modular Design
        Independent Service Components
        Loose Coupling
        Easy to Extend
      Real-time Processing
        Low Latency Pipeline
        Parallel Processing
        Streaming Architecture
    
    Technology Stack
      Deep Learning
        DeepStream SDK
        YOLOv3-tiny Model
        NVIDIA GPU Acceleration
      Streaming Media
        RTSP Protocol
        HLS Delivery
        Mediamtx Server
      System Management
        Systemd Services
        Auto-restart
        Log Monitoring
    
    Functional Features
      Dual Stream Output
        Original Video Stream
        AI Processed Stream
      Real-time Detection
        Object Recognition
        Bounding Box Annotation
        Multi-class Detection
      Web Interface
        Responsive Design
        Real-time Status
        Stream Switching
    
    Deployment Advantages
      One-click Installation
      Auto Configuration
      Easy Maintenance
      Resource Efficient

1.3 Key Systemd Services

➜  system systemctl list-units --type=service --state=active | grep -E "(mediamtx|hls|deepstream|mtx)"

  ai-hls-converter.service           loaded active running AI RTSP to HLS Converter                                                 
  deepstream-ai.service              loaded active running DeepStream AI Processing                                                 
  mediamtx.service                   loaded active running MediaMTX (RTSP/HLS/WebRTC Media Server)                                  
  youtube-to-mtx@lvision.service     loaded active running YouTube -> MediaMTX RTSP Publisher (yt) for user lvision                 
  yt-hls-converter.service           loaded active running YouTube RTSP to HLS Converter                                            
➜  system

1.3.2 mediamtx.service

➜  system cat mediamtx.service
[Unit]
Description=MediaMTX (RTSP/HLS/WebRTC Media Server)
After=network-online.target
Wants=network-online.target
# Add this line to allow other services to declare dependencies
Before=yt-hls-converter.service ai-hls-converter.service deepstream-ai.service

[Service]
Type=simple
# Add user/group to maintain consistency with other services
User=lvision
Group=lvision
# Add environment variables to ensure correct working environment
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

ExecStart=/opt/mediamtx/mediamtx /opt/mediamtx/mediamtx.yml
WorkingDirectory=/opt/mediamtx

# Adjust restart policy - keep your choice, but consider stricter restart
Restart=on-failure
RestartSec=2
# Add startup timeout - mediamtx starts quickly, but just in case
TimeoutStartSec=10

# Log configuration
StandardOutput=journal
StandardError=journal
SyslogIdentifier=mediamtx

# Security hardening (keep your settings)
NoNewPrivileges=true
PrivateTmp=true
# Optional additional security settings
ProtectSystem=strict
ReadWritePaths=/opt/mediamtx /tmp

[Install]
WantedBy=multi-user.target

1.3.3 yt-hls-converter.service

➜  system cat yt-hls-converter.service
[Unit]
Description=YouTube RTSP to HLS Converter
After=network.target mediamtx.service
Wants=mediamtx.service
# If you want to start only after /yt path is ready, you can add:
# After=mediamtx.service
# BindsTo=mediamtx.service

[Service]
Type=simple
User=lvision
Group=lvision
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# Wait for mediamtx to fully start
# ExecStartPre=/bin/sleep 5
ExecStart=/usr/local/bin/rtsp_to_hls_yt.sh

Restart=always
RestartSec=5

StandardOutput=journal
StandardError=journal
SyslogIdentifier=yt-hls-converter

[Install]
WantedBy=multi-user.target

1.3.4 ai-hls-converter.service

➜  system cat ai-hls-converter.service 
[Unit]
Description=AI RTSP to HLS Converter
# Key Modification 1: Add the dependency of deepstream-ai.service
After=deepstream-ai.service mediamtx.service network.target
Wants=deepstream-ai.service mediamtx.service
# If you want to start only after /ai path is ready, you can add:
# After=mediamtx.service
# BindsTo=mediamtx.service

[Service]
Type=simple
User=lvision
Group=lvision
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# Key Modification 2: Wait for deepstream-ai to fully start (takes longer)
# ExecStartPre=/bin/sleep 10
ExecStart=/usr/local/bin/rtsp_to_hls_ai.sh

Restart=always
RestartSec=5

StandardOutput=journal
StandardError=journal
# Key Modification 3: different SyslogIdentifier
SyslogIdentifier=ai-hls-converter

[Install]
WantedBy=multi-user.target
➜  system

1.3.5 deepstream-ai.service

➜  system cat deepstream-ai.service 
[Unit]
Description=DeepStream AI Processing
After=mediamtx.service
Wants=mediamtx.service
Requires=mediamtx.service

[Service]
Type=simple
User=lvision
Group=lvision
Environment="LD_LIBRARY_PATH=/opt/nvidia/deepstream/deepstream-6.3/lib:/usr/local/cuda/lib64"
Environment="GST_DEBUG=WARNING"
Environment="NVBUF_TRANSFORM_USE_EGL=0"
Environment="__GLX_VENDOR_LIBRARY_NAME=nvidia"
WorkingDirectory=/opt/vision
ExecStart=/usr/bin/python3 /opt/vision/yt_ai_app.py -i rtsp://127.0.0.1:8554/yt -o rtsp://127.0.0.1:8554/ai -c /opt/nvidia/deepstream/deepstream-6.3/sources/objectDetector_Yolo/config_infer_primary_yoloV3_tiny.txt
Restart=always
RestartSec=10

StandardOutput=journal
StandardError=journal
SupplementaryGroups=video
Nice=-5

[Install]
WantedBy=multi-user.target

1.4 Configurations and Running Scripts

1.4.1 MediaMTX

➜  system tail -n 40 /opt/mediamtx/mediamtx.yml

# It's possible to use regular expressions by using a tilde as prefix,
# for example "~^(test1|test2)$" will match both "test1" and "test2",
# for example "~^prefix" will match all paths that start with "prefix".
paths:
  # example:
  # my_camera:
  #   source: rtsp://my_camera
  yt:
    runOnDemand: /usr/local/bin/youtube_to_mtx.sh
    runOnDemandRestart: yes
    runOnDemandStartTimeout: 30s
    runOnDemandCloseAfter: 10s

  ai:
    runOnDemand: /usr/local/bin/rtsp_to_hls_ai.sh
    runOnDemandRestart: yes
    runOnDemandStartTimeout: 30s
    runOnDemandCloseAfter: 10s
  
......

  # Settings under path "all_others" are applied to all paths that
  # do not match another entry.
  all_others:
➜  system

1.4.2 `youtube_to_mtx.sh`

➜  ~ cat /usr/local/bin/youtube_to_mtx.sh
#!/usr/bin/env bash
set -euo pipefail

# Fixed YouTube URL
YOUTUBE_URL="https://youtu.be/57w2gYXjRic"
MTX_RTSP="rtsp://127.0.0.1:8554/yt"

# Log to stderr (mediamtx will capture)
echo "=== YouTube Stream Start $(date) ===" >&2
echo "Source: $YOUTUBE_URL" >&2
echo "Destination: $MTX_RTSP" >&2

# Get best stream URL (filter out warning messages)
echo "Fetching stream URL..." >&2
STREAM_URL=$(yt-dlp -f "best[height<=1080]" -g "$YOUTUBE_URL" 2>/dev/null | grep -E "^https?://" | head -1)

if [ -z "$STREAM_URL" ]; then
    echo "Error: Unable to fetch stream URL" >&2
    # Debug: show full output
    echo "Attempting to fetch stream URL (with debugging):" >&2
    yt-dlp -f "best[height<=1080]" -g "$YOUTUBE_URL" >&2
    exit 1
fi

echo "Stream URL fetched successfully" >&2
echo "Starting stream to MediaMTX..." >&2

# Key: Copy video stream directly, no re-encoding (save resources)
exec ffmpeg -hide_banner -loglevel warning \
    -re \
    -i "$STREAM_URL" \
    -c:v copy \
    -an \
    -f rtsp \
    -rtsp_transport tcp \
    "$MTX_RTSP"

1.4.3 `rtsp_to_hls_ai.sh`

➜  ~ cat /usr/local/bin/rtsp_to_hls_ai.sh
#!/usr/bin/env bash
set -euo pipefail

# Modification 1: RTSP URL points to AI stream
RTSP_URL="rtsp://127.0.0.1:8554/ai"
# Modification 2: HLS directory points to ai
HLS_DIR="/var/www/live.visionmisc.com/html/hls/ai"
HLS_M3U8="${HLS_DIR}/index.m3u8"
# Modification 3: log file is now named as AI version
LOG_FILE="/home/lvision/logs/rtsp_to_hls_ai.log"

# Create directories
mkdir -p "${HLS_DIR}"
mkdir -p "$(dirname "${LOG_FILE}")"  # 这会创建 /home/lvision/logs/

# Simple logging function
log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "${LOG_FILE}"
}

log "=== Starting AI RTSP -> HLS Converter ==="

# Clean up old files
find "${HLS_DIR}" -name "*.ts" -delete 2>/dev/null || true
rm -f "${HLS_M3U8}" 2>/dev/null || true

# Added: Wait for AI RTSP stream to be ready (DeepStream may need time to start)
log "Waiting for AI RTSP stream to be ready..."
MAX_WAIT=60
for i in $(seq 1 $MAX_WAIT); do
    if timeout 2s ffprobe -rtsp_transport tcp "${RTSP_URL}" 2>&1 | grep -q "Stream.*Video"; then
        log "AI RTSP stream is ready (waited ${i} seconds)"
        break
    fi
    if [ $i -eq $MAX_WAIT ]; then
        log "Warning: AI RTSP stream not ready within ${MAX_WAIT} seconds, but will continue trying"
    fi
    sleep 1
done

# Simple logging function
log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "${LOG_FILE}"
}


# Main loop
while true; do
    log "Attempting to connect to AI RTSP stream: $RTSP_URL"
    
    log "Starting HLS conversion for AI stream..."
    if ffmpeg -hide_banner -loglevel warning \
        -rtsp_transport tcp \
        -i "${RTSP_URL}" \
        -an \
        -c:v copy \
        -f hls \
        -hls_time 2 \
        -hls_list_size 6 \
        -hls_flags delete_segments+append_list \
        -hls_segment_filename "${HLS_DIR}/segment_%05d.ts" \
        "${HLS_M3U8}" 2>&1 | grep -E "(Stream|Opening|error|failed)" | head -2; then
        log "AI stream conversion process reported an error"
    else
        log "AI stream conversion loop ended or interrupted, preparing to retry..."
    fi
    
    log "Waiting 5 seconds before retrying connection..."
    sleep 5
done
➜  ~

2. Demonstration

Pedestrian Detection Based on DeepStream 6.3 YoloV3-Tiny on a NVidia Jetson AGX Xavier

Car Detected Based on DeepStream 6.3 YoloV3-Tiny on a NVidia Jetson AGX Xavier

🔴 AI Vision Live - Longer Vision Technology

🚀 AI Vision Live

Raspberry Pi 5 + Hailo AI - (3)

Posted on 2025-12-14 Edited on 2025-12-22 In AI , Edge Computing

Two of my previous blogs have been talking about Hailo, respectively:

Since I’m now back to Vancouver for Christmas, let me write my 3rd blog about Hailo today.

1. Some Updates

1.1 Switched To Using Hailo PCIE M.2 Hat.

Top View of Raspberry Pi 5 Hailo PCIE M.2 Hat

Side View of Raspberry Pi 5 Hailo PCIE M.2 Hat

1.2 Raspberry Pi 5 Enviroment `fastfetch`

1.3 Hailo 4.23.0 Released

Good news !!! Hailo 4.23.0 has been released, and it supports Raspberry Pi 5 with native Python 3.13.5 installation, which is incredibly convenient. The only thing to remember: make sure to install DKMS by sudo apt install dkms.

Please visit Hailo Software Downloads to download and install the following three MUST software packages:

1.3.1 HailoRT – Ubuntu package (deb) for arm64 `hailort_4.23.0_arm64.deb`

➜  ~ dpkg -L hailort
/.
/etc
/etc/default
/etc/default/hailort_service
/lib
/lib/systemd
/lib/systemd/system
/lib/systemd/system/hailort.service
/usr
/usr/bin
/usr/bin/hailortcli
/usr/include
/usr/include/gstreamer-1.0
/usr/include/gstreamer-1.0/gst
/usr/include/gstreamer-1.0/gst/hailo
/usr/include/gstreamer-1.0/gst/hailo/include
/usr/include/gstreamer-1.0/gst/hailo/include/hailo_gst.h
/usr/include/gstreamer-1.0/gst/hailo/tensor_meta.hpp
/usr/include/hailo
/usr/include/hailo/buffer.hpp
/usr/include/hailo/device.hpp
/usr/include/hailo/dma_mapped_buffer.hpp
/usr/include/hailo/event.hpp
/usr/include/hailo/expected.hpp
/usr/include/hailo/genai
/usr/include/hailo/genai/common.hpp
/usr/include/hailo/genai/llm
/usr/include/hailo/genai/llm/llm.hpp
/usr/include/hailo/genai/text2image
/usr/include/hailo/genai/text2image/text2image.hpp
/usr/include/hailo/genai/vdevice_genai.hpp
/usr/include/hailo/hailo_gst_tensor_metadata.hpp
/usr/include/hailo/hailo_session.hpp
/usr/include/hailo/hailort.h
/usr/include/hailo/hailort.hpp
/usr/include/hailo/hailort_common.hpp
/usr/include/hailo/hailort_defaults.hpp
/usr/include/hailo/hailort_dma-heap.h
/usr/include/hailo/hef.hpp
/usr/include/hailo/infer_model.hpp
/usr/include/hailo/inference_pipeline.hpp
/usr/include/hailo/network_group.hpp
/usr/include/hailo/network_rate_calculator.hpp
/usr/include/hailo/platform.h
/usr/include/hailo/quantization.hpp
/usr/include/hailo/runtime_statistics.hpp
/usr/include/hailo/stream.hpp
/usr/include/hailo/transform.hpp
/usr/include/hailo/vdevice.hpp
/usr/include/hailo/vstream.hpp
/usr/lib
/usr/lib/aarch64-linux-gnu
/usr/lib/aarch64-linux-gnu/gstreamer-1.0
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgsthailo.so
/usr/lib/cmake
/usr/lib/cmake/HailoRT
/usr/lib/cmake/HailoRT/HailoRTConfig.cmake
/usr/lib/cmake/HailoRT/HailoRTConfigVersion.cmake
/usr/lib/cmake/HailoRT/HailoRTTargets-release.cmake
/usr/lib/cmake/HailoRT/HailoRTTargets.cmake
/usr/lib/libhailort.so.4.23.0
/usr/local
/usr/local/bin
/usr/local/bin/hailort_service
/usr/share
/usr/share/doc
/usr/share/doc/hailort
/usr/share/doc/hailort/copyright
➜  ~

1.3.2 HailoRT – PCIe driver Ubuntu package (deb) `hailort-pcie-driver_4.23.0_all.deb`

➜  ~ dpkg -L hailort-pcie-driver
/.
/lib
/lib/firmware
/lib/firmware/hailo
/lib/firmware/hailo/hailo8_fw.4.23.0.bin
/lib/udev
/lib/udev/rules.d
/lib/udev/rules.d/51-hailo-udev.rules
/share
/share/opt
/share/opt/hailo
/share/opt/hailo/common
/share/opt/hailo/common/fw_operation.c
/share/opt/hailo/common/fw_operation.h
/share/opt/hailo/common/fw_validation.c
/share/opt/hailo/common/fw_validation.h
/share/opt/hailo/common/hailo_ioctl_common.h
/share/opt/hailo/common/hailo_resource.c
/share/opt/hailo/common/hailo_resource.h
/share/opt/hailo/common/pcie_common.c
/share/opt/hailo/common/pcie_common.h
/share/opt/hailo/common/soc_structs.h
/share/opt/hailo/common/utils.h
/share/opt/hailo/common/vdma_common.c
/share/opt/hailo/common/vdma_common.h
/share/opt/hailo/linux
/share/opt/hailo/linux/pcie
/share/opt/hailo/linux/pcie/51-hailo-udev.rules
/share/opt/hailo/linux/pcie/Kbuild
/share/opt/hailo/linux/pcie/Makefile
/share/opt/hailo/linux/pcie/Readme.md
/share/opt/hailo/linux/pcie/dkms.conf.in
/share/opt/hailo/linux/pcie/dkms_uninstall.sh
/share/opt/hailo/linux/pcie/hailo_pci.conf
/share/opt/hailo/linux/pcie/src
/share/opt/hailo/linux/pcie/src/fops.c
/share/opt/hailo/linux/pcie/src/fops.h
/share/opt/hailo/linux/pcie/src/nnc.c
/share/opt/hailo/linux/pcie/src/nnc.h
/share/opt/hailo/linux/pcie/src/pcie.c
/share/opt/hailo/linux/pcie/src/pcie.h
/share/opt/hailo/linux/pcie/src/soc.c
/share/opt/hailo/linux/pcie/src/soc.h
/share/opt/hailo/linux/pcie/src/sysfs.c
/share/opt/hailo/linux/pcie/src/sysfs.h
/share/opt/hailo/linux/pcie/tools
/share/opt/hailo/linux/pcie/tools/hailo_load.sh
/share/opt/hailo/linux/pcie/tools/hailo_pci_driver_dkms_remove.sh
/share/opt/hailo/linux/pcie/tools/hailo_unload.sh
/share/opt/hailo/linux/utils
/share/opt/hailo/linux/utils/compact.h
/share/opt/hailo/linux/utils/fw_common.h
/share/opt/hailo/linux/utils/integrated_nnc_utils.c
/share/opt/hailo/linux/utils/integrated_nnc_utils.h
/share/opt/hailo/linux/utils/logs.c
/share/opt/hailo/linux/utils/logs.h
/share/opt/hailo/linux/vdma
/share/opt/hailo/linux/vdma/ioctl.c
/share/opt/hailo/linux/vdma/ioctl.h
/share/opt/hailo/linux/vdma/memory.c
/share/opt/hailo/linux/vdma/memory.h
/share/opt/hailo/linux/vdma/vdma.c
/share/opt/hailo/linux/vdma/vdma.h
/usr
/usr/share
/usr/share/doc
/usr/share/doc/hailort
/usr/share/doc/hailort/hailo_firmware_eula
➜  ~

1.3.3 HailoRT – Python package (whl) for Python 3.13, aarch64 `hailort-4.23.0-cp313-cp313-linux_aarch64.whl`

1.3.4 TAPPAS Python Binding (Optional)

1
2
3

➜  ~ pip list | rg hailo
hailort                      4.23.0
➜  ~

Finally, let’s test if Hailo is correctly configured on our Raspberry Pi 5.

2. Models

2.1 Prepare HAILO8L Models

2.2 `hailortcli run` `*.hef`s

3. Demonstration

3.1 On My Cellphone

Live Stream - Deep Night Beautiful Vancouver	Live Stream - Beautiful Dublin	Live Stream - Bears In Katmai Nation Park, Alaska

3.2 From My Desktop

NVidia Jetson AGX Xavier - Deepstream (2)

Posted on 2025-09-12 Edited on 2025-12-15 In AI , Edge Computing

Good new. I’m flying back to China for some traditional Chinese food - 热干面 😍❤️😊
In my first blog post about Jetson AGX Xavier early in year 2021, I just did a very simple demo with any industrial camera using Aravis. Today, I’m going to make full use of the CUDA GPU Compute Capability of my Jetson AGX Xavier, running DeepStream to live cam stream the city of Vancouver and twitter it out.

By the way, I forget to mention my this blog Kinect 2 on Jetson AGX Xavier.

1. Environment

1.1 `neofetch`

1.2 `jtop`

1.3 `tegrastats and jetson_release`

2. DeepStream

2.1 Install DeepStream On Jetson AGX Xavier

You first have to find the corresponding DeepStream version from its official website DeepStream. Since I’m using a Jetson AGX Xavier, I’d choose DeepStream 6.3 and have it installed on my device.

2.2 Set Up Configuration

2.3 Model Configuration

2.4 Demonstration

Process On a Video without Using deepstream Command

Results of Detection

2.5 Conclusion

DeepStream 6.3 does not provide an out-of-the-box sample for rendering different object classes with different bounding box colors.

In DeepStream 6.3, deepstream-app uses the default nvdsosd behavior, which applies a single bounding box style to ALL detected objects.

The application-level configuration parser does not support class-wise color mapping, and nvdsosd color customization is not exposed through deepstream-app config files.

As a result, rendering different colors for different object classes requires custom OSD logic or a modified application, rather than configuration-only changes.