17 posts tagged with "RK3576"

Board Introduction​

DshanPi-A1 is a high-performance AI embedded development board developed by Shenzhen Baiwen Web (Weidongshan Team). It is based on the Rockchip RK3576 chip and is designed specifically for AI education, edge computing, and smart device development.

Core Specifications​

Product Features​

1️⃣ Powerful AI Processing Capability​

· 6TOPS dedicated NPU can smoothly run lightweight large language models such as DeepSeek and Qwen

· Supports mainstream AI frameworks (TensorFlow, PyTorch, MXNet, etc.)

· Suitable for AI application development such as image recognition, speech processing, and smart surveillance

2️⃣ Rich Multimedia Performance​

· Supports 8K video decoding, can be used as an HD media center

· HDMI IN function allows the development board to be used as a secondary display for a PC

· Built-in audio codec, supports 3.5mm audio output

3️⃣ Comprehensive Development Support​

· Ships with DShanOS (Baiwen Web's self-developed Linux distribution)

· Comes with free teaching courses and documentation, lowering the learning barrier

· Supports Armbian system, with official image downloads available

· Provides a complete SDK and sample code for easy secondary development

4️⃣ Flexible Expansion Capability​

· All GPIO interfaces are exposed, making it easy to connect various sensors and actuators

· Supports multiple communication protocols, suitable for IoT application development

· Can connect external cameras, displays, WiFi/4G modules, and more to expand functionality

Application Scenarios​

· AI Education and Learning: Suitable for embedded AI course teaching and experimentation

· Edge Computing Devices: Can deploy lightweight AI models to enable local intelligent decision-making

· Smart Home Hub: Build a high-performance, low-power smart home control center

· Industrial Automation: Suitable for smart device monitoring and data acquisition

· Commercial Display: Supports 8K display, can be used for advertising machines and information publishing systems

· AI Vision Systems: Can be used for face recognition, object detection, and other scenarios

Companion Resources​

· Official documentation site: https://wiki.dshanpi.org/docs/dshanpi-a1/

· Developer community: Weidongshan Embedded Developer Community (100ask.org)

· QQ technical exchange group: 798273638

· Free tutorials: Provides AI development courses from basic to advanced

buildroot SDK Installation​

1. Obtain the Official Virtual Machine​

https://pan.baidu.com/s/15M8zuHOwl_SITl6cSk_7Vg?pwd=eaax Extraction code: eaax

2. Install and open vmware, run the virtual machine​

Enter the ubuntu system, the account password is ubuntu

3. Build the SDK​

Open the virtual machine and execute the following command to enter the SDK root directory:

cd ~/100ask-rk3576_SDK/

Select the configuration file:

./build.sh lunch

4. Build​

Run the following command

./build.sh

Wait for the build to finish

Device Tree Modification to Enable the Camera Node​

Enter

/home/ubuntu/100ask-rk3576_SDK/kernel/arch/arm64/boot/dts/rockchip

Modify the file shown in the picture

Change to if1

Return to the SDK directory and use

./build.sh kernel

Finally

./build.sh updateimg

Finally, upload and flash (remember to enter MASKROM)

Results​

But there is still a bug, the screen camera color is still not displayed correctly.

Problem Solving​

At first, I suspected that the camera's .xml (.json) was not loaded. After debugging for a long time, there was still no result.

After extensive debugging, I finally found a solution:

Step 1: Discover Key Clues from media-ctl Output​

In the output of media-ctl -p -d /dev/media0, I noticed this key information:

entity 63: m00_b_rk628-csi9-0051 (1 pad, 1 link)
	type V4L2 subdev subtype Sensor flags 0
	device node name /dev/v4l-subdev2
	 pad 0: Source
	  [fmt:UYVY8_2X8/64x64@10000/600000 field:none]
	  -> "rockchip-csi2-dphy0":0 [ENABLED] ←note the [ENABLED] here

Analysis Points:

· A rk628-csi sensor entity exists in the system

· Its link state is [ENABLED], meaning it is occupying CSI hardware resources

· But the resolution is only 64x64, which is obviously not a normal camera output

Step 2: Discover from dmesg Logs that IMX415 Driver Is Normal​

From the output of dmesg | grep -i imx415:

[3.459474] imx415 3-0037: Detected imx415 id 0000e0
[3.515523] imx415 3-0037: Consider updating driver imx415 to match on endpoints
[3.515557] rockchip-csi2-dphy csi2-dphy3: dphy3 matches m01_f_imx415 3-0037: bustype 5

Analysis Points:

· The IMX415 sensor was correctly detected (ID: 0000e0)

· The driver loaded successfully and even established a matching relationship with CSI-DPHY

· But there is no message indicating successful V4L2 sub-device creation

Step 3: Infer Hardware Connections from Device Tree Structure​

From the device tree snippet:

&csi2_dphy3 {
    port@0 {
        mipi_in_ucam3: endpoint@1 {
            remote-endpoint = <&imx415_out0>; ←IMX415 connects here
            data-lanes = <1 2 3 4>;
        };
    };
    port@1 {
        csidphy3_out: endpoint@0 {
            remote-endpoint = <&mipi3_csi2_input>;
        };
    };
};

Analysis Points:

· IMX415 is connected to the system via CSI-DPHY3

· But the actual media-ctl output shows that rockchip-csi2-dphy0 is occupied

· This indicates that there may be a resource allocation issue among multiple CSI interfaces

Step 4: Infer Registration Failure from the Missing V4L2 Sub-device​

When running the check command:

ls /sys/bus/i2c/devices/3-0037/v4l-subdev*/media_device/
# Output: No such file or directory

Analysis Points:

· The I2C device 3-0037 exists and the driver binding is normal

· But no corresponding V4L2 sub-device was created

· This usually means the driver probed successfully, but the subsequent V4L2 registration failed

Step 5: Connect All Clues to Form a Complete Picture​

Stringing the above clues together:

1. Symptom: IMX415 driver loaded but no V4L2 device

2. Evidence 1: rk628-csi is occupying the CSI link and its status is [ENABLED]

3. Evidence 2: IMX415 I2C communication is normal but the media device is missing

4. Evidence 3: The device tree shows that the two may share CSI hardware resources

Logical Reasoning:

· If the IMX415 hardware is faulty -> I2C probing should fail

· If there is an IMX415 driver issue -> dmesg should have error logs

· If the device tree configuration is wrong -> CSI matching would not succeed

· The only reasonable explanation: the hardware resources are occupied by another device

Step 6: Start Practicing​

Recompile the device tree and flash the development board, it runs perfectly!

Summary​

1. Hardware resource conflicts in embedded devices are common hidden issues, especially when multiple devices share links such as CSI and I2C. The device occupation status should be a key focus of investigation.

2. During development, make full use of tools such as media-ctl and dmesg, combined with device tree configuration analysis, to quickly locate resource allocation issues and avoid blind debugging.

3. For scenarios where the driver loads normally but functions abnormally, prioritize investigating key aspects such as hardware resource occupation and V4L2 sub-device registration to narrow down the scope of the problem.

4. The CSI interface resource allocation of the DshanPi-A1 development board must be precisely configured through the device tree. After modification, strictly follow the Buildroot build process to regenerate the image to ensure the configuration takes effect.

Development Environment​

PC side: ubuntu22.04-x86-64

Board side: buildroot

The specific theoretical part won't be elaborated here; there's plenty online. This records the entire operation process, divided into two main parts: 1. PC side 2. Board side

1. PC Side​

Setting Up the RKNN-Toolkit2 Environment​

# Download the code repository
mkdir rknn
cd rknn
wget https://dl.100ask.net/Hardware/MPU/RK3576-DshanPi-A1/utils/rknn-toolkit2.zip
unzip rknn-toolkit2.zip
wget https://dl.100ask.net/Hardware/MPU/RK3576-DshanPi-A1/utils/rknn_model_zoo.zip
unzip rknn_model_zoo.zip
# Set up conda environment
wget -c https://repo.anaconda.com/archive/Anaconda3-2025.06-1-Linux-x86_64.sh
bash Anaconda3-2025.06-1-Linux-x86_64.sh
	Please, press ENTER to continue
	>>> 
	Do you accept the license terms? [yes|no]
	>>> yes
	Anaconda3 will now be installed into this location:
    /home/ubuntu/anaconda3
      - Press ENTER to confirm the location
      - Press CTRL-C to abort the installation
      - Or specify a different location below

    [/home/ubuntu/anaconda3] >>> 
 	You can undo this by running `conda init --reverse $SHELL`? [yes|no]
	[no] >>> yes
	Thank you for installing Anaconda3!
# Activate environment variables
source ~/.bashrc
# Create RKNN environment
conda create -n rknn-toolkit2 python=3.8
conda activate rknn-toolkit2
# Install RKNN-Toolkit2
cd rknn-toolkit2/rknn-toolkit2/packages/x86_64/
conda install compilers cmake
pip install -r requirements_cp38-2.3.2.txt
pip install rknn_toolkit2-2.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
# Verify installation
    (rknn-toolkit2) ubuntu@ubuntu-2204:~/rknn/rknn-toolkit2/rknn-toolkit2/packages/x86_64$ python3
    Python 3.8.20 (default, Oct  3 2024, 15:24:27)
    [GCC 11.2.0] :: Anaconda, Inc. on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from rknn.api import RKNN
    >>> exit()
    (rknn-toolkit2) ubuntu@ubuntu-2204:~/rknn/rknn-toolkit2/rknn-toolkit2/packages/x86_64$

2. On-Board Inference​

First, here is my environment: I use the buildroot that comes with the SDK, adapted by the 100ASK team, with rknnruntime already enabled, and I did not update the runtime.

root@rk3576-buildroot:/# uname -a
	Linux rk3576-buildroot 6.1.75 #3 SMP Fri Nov 28 09:41:14 EST 2025 aarch64 GNU/Linux
root@rk3576-buildroot:/# find ./ -name *rknn*
    ./rockchip-test/npu2/model/RK356X/mobilenet_v1.rknn
    ./rockchip-test/npu2/model/RK3588/vgg16_max_pool_fp16.rknn
    ./sys/kernel/debug/clk/hclk_rknn_root
    ./sys/kernel/debug/clk/clk_rknn_dsu0
    ./sys/kernel/debug/clk/aclk_rknn0
    ./sys/kernel/debug/clk/aclk_rknn1
    ./sys/kernel/debug/clk/aclk_rknn_cbuf
    ./sys/kernel/debug/clk/hclk_rknn_cbuf
    ./usr/share/model/RK3562/mobilenet_v1.rknn
    ./usr/share/model/RK3566_RK3568/mobilenet_v1.rknn
    ./usr/share/model/RK3588/mobilenet_v1.rknn
    ./usr/share/model/RK3576/mobilenet_v1.rknn
    ./usr/lib/librknnrt.so
    ./usr/bin/start_rknn.sh
    ./usr/bin/rknn_common_test
    ./usr/bin/restart_rknn.sh
    ./usr/bin/rknn_server

tip: My buildroot configuration ./build.sh bconfig

 [*] Rockchip NPU power control for linux                                                  │ │
  │ │                   [ ] Rockchip NPU power control combine for linux                                          │ │
  │ │                   [ ] Rockchip recovery for linux                                                           │ │
  │ │                   [ ] rkadk                                                                                 │ │
  │ │                   [ ] rknpu                                                                                 │ │
  │ │                   [ ] rknpu pcie                                                                            │ │
  │ │                   [ ] python-rknn                                                                           │ │
  │ │                   [*] rknpu2                                                                                │ │
  │ │                   [*] rknpu2 example                                                                        │ │
  │ │                   [ ] rknpu firmware                                                                        │ │
  │ │                   [ ] RKPARTYBOX demo                                                                       │ │
  │ │                   [*] rockchip script                 

As you can see, I did not change the rknpu configuration; buildroot already configures the rknpu driver and rknnruntime by default.

Problem 1: Solving ADB Permission Issues on the PC Side​

When running on-board inference on the PC, the following error occurs:

：(rknn-toolkit2) ubuntu@ubuntu-2204:~/rknn/rknn_model_zoo/examples/yolov8/python$ sudo python3 yolov8.py --target rk3576 --model_path ../model/yolov8.rknn --img_show
Traceback (most recent call last):
  File "/home/ubuntu/rknn/rknn_model_zoo/examples/yolov8/python/yolov8.py", line 2, in <module>
    import cv2
ModuleNotFoundError: No module named 'cv2'
(rknn-toolkit2) ubuntu@ubuntu-2204:~/rknn/rknn_model_zoo/examples/yolov8/python$ python3 yolov8.py --target rk3576 --model_path ../model/yolov8.rknn --img_show
I rknn-toolkit2 version: 2.3.2
--> Init runtime environment
adb: unable to connect for root: insufficient permissions for device: user in plugdev group; are your udev rules wrong?
See [http://developer.android.com/tools/device.html] for more information
I target set by user is: rk3576
E init_runtime: Get board target failed, ret code: 1. error: insufficient permissions for device: user in plugdev group; are your udev rules wrong?
See [http://developer.android.com/tools/device.html] for more information

E init_runtime: Traceback (most recent call last):
  File "rknn/api/rknn_log.py", line 344, in rknn.api.rknn_log.error_catch_decorator.error_catch_wrapper
  File "rknn/api/rknn_base.py", line 2566, in rknn.api.rknn_base.RKNNBase.init_runtime
  File "rknn/api/rknn_runtime.py", line 223, in rknn.api.rknn_runtime.RKNNRuntime.__init__
  File "rknn/api/rknn_platform.py", line 607, in rknn.api.rknn_platform.get_board_info
RuntimeError

Note this line

adb: unable to connect for root: insufficient permissions for device: user in plugdev group; are your udev rules wrong?
See [http://developer.android.com/tools/device.html] for more information

It indicates insufficient adbd permissions; fix it with:

# Add udev rules
echo 'SUBSYSTEM=="usb", ATTR{idVendor}=="2207", MODE="0666", GROUP="plugdev"' | sudo tee /etc/udev/rules.d/51-android.rules
# Reload rules
sudo udevadm control --reload-rules
sudo udevadm trigger
# Restart ADB
adb kill-server
adb start-server
adb devices
    List of devices attached
    8074683be1050187        device

Problem 2: Solving the Board-Side adbd Port 5037 Not Open Issue​

Symptom:

# Run on-board inference
 python3 yolov8.py --target rk3576 --model_path ../model/yolov8.rknn --img_show
    I rknn-toolkit2 version: 2.3.2
    --> Init runtime environment
    adbd is already running as root
    I target set by user is: rk3576
    I Get hardware info: target_platform = rk3576, os = Linux, aarch = aarch64
    I Check RK3576 board npu runtime version
    W kill server failed while restarting, ret code: 1. warning: killall: rknn_server: no process killed
      Please skip it if rknn_server not running on board.
    I Starting ntp or adb, target is RK3576
    I Start adb...
    I Connect to Device success!
    I NPUTransfer(3672747): Starting NPU Transfer Client, Transfer version 2.2.2 (12abf2a@2024-09-02T03:22:41)
    E RKNNAPI: rknn_init,  server connect fail!  ret = -9(ERROR_PIPE)!
    E init_runtime: The rknn_server on the concected device is abnormal, please start the rknn_server on the device according to:
                   https://github.com/airockchip/rknn-toolkit2/blob/master/doc/rknn_server_proxy.md
    W init_runtime: ===================== WARN(1) =====================
    E rknn-toolkit2 version: 2.3.2
    E init_runtime: Traceback (most recent call last):
      File "rknn/api/rknn_log.py", line 344, in rknn.api.rknn_log.error_catch_decorator.error_catch_wrapper

Note

    I NPUTransfer(3672747): Starting NPU Transfer Client, Transfer version 2.2.2 (12abf2a@2024-09-02T03:22:41)
    E RKNNAPI: rknn_init,  server connect fail!  ret = -9(ERROR_PIPE)!
    E init_runtime: The rknn_server on the concected device is abnormal, please start the rknn_server on the device according to:
                   https://github.com/airockchip/rknn-toolkit2/blob/master/doc/rknn_server_proxy.md

Following the hint, go to https://github.com/airockchip/rknn-toolkit2/blob/master/doc/rknn_server_proxy.md to find the solution

In this document's

6. FAQ​

Problem 1​

On Debian systems, the rknn_server service has been started in the background, but the following error still occurs during on-board inference:

D NPUTransfer: ERROR: socket read fd = 4, n = -1: Connection reset by peer
D NPUTransfer: Transfer client closed, fd = 4
E RKNNAPI: rknn_init,  server connect fail!  ret = -9(ERROR_PIPE)!
E build_graph: The rknn_server on the concected device is abnormal, please start the rknn_server on the device according to:
               https://github.com/airockchip/rknn-toolkit2/blob/master/doc/rknn_server_proxy.md

Solution: This is usually because the adbd program on the Debian firmware is not listening on port 5037. You can run the following command on the board to check:

netstat -n -t -u -a

If the output does not contain port 5037, run the following commands to download and update the adbd program, and reboot the board; otherwise, skip the following steps.

wget -O adbd.zip https://ftzr.zbox.filez.com/v2/delivery/data/7f0ac30dfa474892841fcb2cd29ad924/adbd.zip
unzip adbd.zip
adb push adbd/linux-aarch64/adbd /usr/bin/adbd

Enter the device shell command to add executable permission to adbd

adb shell "chmod +x /usr/bin/adbd"
adb reboot

After rebooting the device, follow the startup steps to start the rknn_server service and try on-board inference again.

Although this document uses Debian firmware and I use buildroot, the symptom is the same. Follow its solution to troubleshoot:

# This is the board side
root@rk3576-buildroot:/# netstat -n -t -u -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:53              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 :::53                   :::*                    LISTEN
tcp        0      0 :::22                   :::*                    LISTEN
tcp        0      0 :::5555                 :::*                    LISTEN
udp        0      0 0.0.0.0:53              0.0.0.0:*
udp        0      0 0.0.0.0:67              0.0.0.0:*
udp        0      0 0.0.0.0:68              0.0.0.0:*
udp        0      0 127.0.0.1:323           0.0.0.0:*
udp        0      0 :::53                   :::*
udp        0      0 ::1:323                 0.0.0.0:*
udp        0      0 :::546                  :::*

As you can see, port 5037 is indeed not open

# This is the PC side
wget -O adbd.zip https://ftzr.zbox.filez.com/v2/delivery/data/7f0ac30dfa474892841fcb2cd29ad924/adbd.zip
unzip adbd.zip
adb push adbd/linux-aarch64/adbd /usr/bin/adbd
adb shell "chmod +x /usr/bin/adbd"
adb reboot

After rebooting the development board, retest

# This is the board side
restart_rknn.sh
root@rk3576-buildroot:/# netstat -n -t -u -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 127.0.0.1:5037          0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:53              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:5555            0.0.0.0:*               LISTEN
tcp        0      0 :::22                   :::*                    LISTEN
tcp        0      0 :::53                   :::*                    LISTEN
udp        0      0 0.0.0.0:53              0.0.0.0:*
udp        0      0 0.0.0.0:67              0.0.0.0:*
udp        0      0 0.0.0.0:68              0.0.0.0:*
udp        0      0 127.0.0.1:323           0.0.0.0:*
udp        0      0 :::546                  :::*
udp        0      0 :::53                   :::*
udp        0      0 ::1:323                 0.0.0.0:*

As you can see, port 5037 is now being listened on. Continue trying on-board inference

# This is the PC side
(rknn-toolkit2) ubuntu@ubuntu-2204:~/rknn/rknn_model_zoo/examples/yolov8/python$ python3 yolov8.py --target rk3576 --model_path ../model/yolov8.rknn --img_show
I rknn-toolkit2 version: 2.3.2
--> Init runtime environment
adb: unable to connect for root: closed
I target set by user is: rk3576
I Get hardware info: target_platform = rk3576, os = Linux, aarch = aarch64
I Check RK3576 board npu runtime version
I Starting ntp or adb, target is RK3576
I Start adb...
I Connect to Device success!
I NPUTransfer(3675220): Starting NPU Transfer Client, Transfer version 2.2.2 (12abf2a@2024-09-02T03:22:41)
I NPUTransfer(3675220): TransferBuffer: min aligned size: 1024
D RKNNAPI: ==============================================
D RKNNAPI: RKNN VERSION:
D RKNNAPI:   API: 2.3.2 (1842325 build@2025-03-30T09:55:23)

On-board inference succeeded, showing the face recognition image

3. On-Board Inference (Native)​

Since buildroot is used, and the RK platform after rk1808 does not support python deployment in buildroot, setting up a python inference software stack yourself is time-consuming and encounters many issues. For the python demo, refer to the 100ASK RKNN环境搭建 | 东山Π; it won't be elaborated here. Using the cpp interface, we still use the aforementioned yolov8 for on-board inference:

Prepare the Model​

cd ~/rknn/rknn_model_zoo/examples/yolov8/model
sh download_model.sh
ls -lah yolov8n.onnx
	-rw-rw-r-- 1 ubuntu ubuntu 13M Nov 29 07:36 yolov8n.onnx

Model Conversion​

cd ../python/
(rknn-toolkit2) ubuntu@ubuntu-2204:~/rknn/rknn_model_zoo/examples/yolov8/python$ python convert.py ../model/yolov8n.onnx rk3576 i8 ../model/yolov8n.rknn

python convert.py ../model/yolov5s_relu.onnx rk3576 i8 ../model/yolov5s_relu.rknn 

    I rknn-toolkit2 version: 2.3.2
    --> Config model
    done
    --> Loading model
    I Loading : 100%|██████████████████████████████████████████████| 126/126 [00:00<00:00, 43282.74it/s]
    done
    --> Building model
    I OpFusing 0: 100%|█████████████████████████████████████████████| 100/100 [00:00<00:00, 1590.15it/s]
    I OpFusing 1 : 100%|█████████████████████████████████████████████| 100/100 [00:00<00:00, 860.03it/s]
    I OpFusing 0 : 100%|█████████████████████████████████████████████| 100/100 [00:00<00:00, 739.11it/s]
    I OpFusing 1 : 100%|█████████████████████████████████████████████| 100/100 [00:00<00:00, 652.15it/s]
    I OpFusing 2 : 100%|█████████████████████████████████████████████| 100/100 [00:00<00:00, 229.74it/s]
    W build: found outlier value, this may affect quantization accuracy
                            const name                        abs_mean    abs_std     outlier value
                            model.0.conv.weight               2.44        2.47        -17.494
                            model.22.cv3.2.1.conv.weight      0.09        0.14        -10.215
                            model.22.cv3.1.1.conv.weight      0.12        0.19        13.361, 13.317
                            model.22.cv3.0.1.conv.weight      0.18        0.20        -11.216
    I GraphPreparing : 100%|████████████████████████████████████████| 161/161 [00:00<00:00, 3291.17it/s]
    I Quantizating : 100%|████████████████████████████████████████████| 161/161 [00:04<00:00, 34.87it/s]
    W build: The default input dtype of 'images' is changed from 'float32' to 'int8' in rknn model for performance!
                           Please take care of this change when deploy rknn model with Runtime API!
    W build: The default output dtype of '318' is changed from 'float32' to 'int8' in rknn model for performance!
                          Please take care of this change when deploy rknn model with Runtime API!
    W build: The default output dtype of 'onnx::ReduceSum_326' is changed from 'float32' to 'int8' in rknn model for performance!
                          Please take care of this change when deploy rknn model with Runtime API!
    W build: The default output dtype of '331' is changed from 'float32' to 'int8' in rknn model for performance!
                          Please take care of this change when deploy rknn model with Runtime API!
    W build: The default output dtype of '338' is changed from 'float32' to 'int8' in rknn model for performance!
                          Please take care of this change when deploy rknn model with Runtime API!
    W build: The default output dtype of 'onnx::ReduceSum_346' is changed from 'float32' to 'int8' in rknn model for performance!
                          Please take care of this change when deploy rknn model with Runtime API!
    W build: The default output dtype of '350' is changed from 'float32' to 'int8' in rknn model for performance!
                          Please take care of this change when deploy rknn model with Runtime API!
    W build: The default output dtype of '357' is changed from 'float32' to 'int8' in rknn model for performance!
                          Please take care of this change when deploy rknn model with Runtime API!
    W build: The default output dtype of 'onnx::ReduceSum_365' is changed from 'float32' to 'int8' in rknn model for performance!
                          Please take care of this change when deploy rknn model with Runtime API!
    W build: The default output dtype of '369' is changed from 'float32' to 'int8' in rknn model for performance!
                          Please take care of this change when deploy rknn model with Runtime API!
    I rknn building ...
    I rknn building done.
    done
    --> Export rknn model
    done
cd ../model
ls -lah yolov8n.rknn
	-rw-rw-r-- 1 ubuntu ubuntu 6.2M Nov 29 07:39 yolov8n.rknn

Run the RKNN C Example​

First, compile the C example, then deploy the executable file, model file, and resource files to the board.

Compilation​

For compilation, use the build-linux.sh script in the rknn_model_zoo directory; you need to configure the toolchain first. Modify build-linux.sh:

GCC_COMPILER=/home/ubuntu/rk3576/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu
chmod +x ./build-linux.sh
./build-linux.sh -t rk3576 -a aarch64 -d yolov8
	-- Set runtime path of "/home/ubuntu/rknn/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/./rknn_yolov8_demo" to "$ORIGIN/../lib"
	-- Installing: /home/ubuntu/rknn/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/model/bus.jpg
	-- Installing: /home/ubuntu/rknn/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/model/coco_80_labels_list.txt
	-- Installing: /home/ubuntu/rknn/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/model/yolov8.rknn
	-- Installing: /home/ubuntu/rknn/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/model/yolov8n.rknn
	-- Installing: /home/ubuntu/rknn/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/lib/librknnrt.so
	-- Installing: /home/ubuntu/rknn/rknn_model_zoo/install/rk3576_linux_aarch64/rknn_yolov8_demo/lib/librga.so
# Take a look
ubuntu@ubuntu-2204:~/rknn/rknn_model_zoo/install$ tree -L 4
.
└── rk3576_linux_aarch64
    └── rknn_yolov8_demo
        ├── lib
        │   ├── librga.so
        │   └── librknnrt.so
        ├── model
        │   ├── bus.jpg
        │   ├── coco_80_labels_list.txt
        │   ├── yolov8n.rknn
        │   └── yolov8.rknn
        ├── rknn_yolov8_demo
        └── rknn_yolov8_demo_zero_copy

This rknn_yolov8_demo is the set of files to be deployed to the board

adb push install/rk3576_linux_aarch64/rknn_yolov8_demo /data/
install/rk3576_linux_aarch64/rknn_yolov8_demo/: 8 files pushed. 3.6 MB/s (23008641 bytes in 6.049s)

Run on the board side

root@rk3576-buildroot:/data/rknn_yolov8_demo# ./rknn_yolov8_demo ./model/yolov8.rknn ./model/bus.jpg
    load lable ./model/coco_80_labels_list.txt
    model input num: 1, output num: 9
    input tensors:
      index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
    output tensors:
      index=0, name=318, n_dims=4, dims=[1, 64, 80, 80], n_elems=409600, size=409600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-58, scale=0.117659
      index=1, name=onnx::ReduceSum_326, n_dims=4, dims=[1, 80, 80, 80], n_elems=512000, size=512000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003104
      index=2, name=331, n_dims=4, dims=[1, 1, 80, 80], n_elems=6400, size=6400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003173
      index=3, name=338, n_dims=4, dims=[1, 64, 40, 40], n_elems=102400, size=102400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-45, scale=0.093747
      index=4, name=onnx::ReduceSum_346, n_dims=4, dims=[1, 80, 40, 40], n_elems=128000, size=128000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003594
      index=5, name=350, n_dims=4, dims=[1, 1, 40, 40], n_elems=1600, size=1600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003627
      index=6, name=357, n_dims=4, dims=[1, 64, 20, 20], n_elems=25600, size=25600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-34, scale=0.083036
      index=7, name=onnx::ReduceSum_365, n_dims=4, dims=[1, 80, 20, 20], n_elems=32000, size=32000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003874
      index=8, name=369, n_dims=4, dims=[1, 1, 20, 20], n_elems=400, size=400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
    model is NHWC input fmt
    model input height=640, width=640, channel=3
    origin size=640x640 crop size=640x640
    input image: 640 x 640, subsampling: 4:2:0, colorspace: YCbCr, orientation: 1
    scale=1.000000 dst_box=(0 0 639 639) allow_slight_change=1 _left_offset=0 _top_offset=0 padding_w=0 padding_h=0
    rga_api version 1.10.1_[0]
    rknn_run
    person @ (211 241 282 507) 0.864
    person @ (109 235 225 536) 0.856
    bus @ (99 136 552 455) 0.856
    person @ (476 223 560 521) 0.848
    person @ (80 326 116 513) 0.280
    write_image path: out.png width=640 height=640 channel=3 data=0x3e2e9200

View on the PC side

adb pull /data/rknn_yolov8_demo/out.png ./

Matches expectations

At this point, the RKNN environment has been set up successfully

In embedded system development, display configuration is the foundation for implementing user interfaces. Weston, as the reference implementation of the Wayland compositor, is widely used in embedded devices and desktop environments. Faced with different hardware configurations and application scenarios, precise control of display output is required. Below, based on the DshanPI-A1 buildroot firmware version, we provide Weston configuration methods for three typical scenarios: single-screen exclusive, dual-screen mirrored, and dual-screen extended display, along with a detailed operation guide.

Weston Architecture and Configuration Principles​

Before configuration, it helps to briefly understand the basic principles.

Weston Display System Architecture​

Weston adopts a modular design; its core components include:

• Backend: Responsible for interacting with the underlying graphics system, such as DRM, X11, Wayland, etc.
• Compositor: Manages window compositing, rendering, and output
• Shell: Provides the user interface framework, such as desktop, panel, etc.

In embedded systems, the DRM backend drm-backend.so is typically used; it directly interacts with the Linux kernel's Direct Rendering Manager, providing efficient hardware acceleration support.

Configuration File Structure​

Weston's main configuration file is located at /etc/xdg/weston/weston.ini and is organized in INI format. The key configuration sections include:

• [core]: Core configuration, defines backend behavior and global parameters
• [output]: Display output configuration, controls the properties of each physical interface
• [shell]: Desktop environment related settings
• [libinput]: Input device configuration
• [device]: Advanced configuration for specific devices

HDMI Exclusive Display Configuration​

In some embedded application scenarios, the device needs to force the HDMI interface as the only display output while disabling the built-in screen (such as the DSI-interface LCD). This configuration is common in:

• Industrial consoles permanently connected to external monitors
• Digital signage systems using large-screen displays
• Professional applications requiring high-resolution output

The following is the detailed configuration:

First, configure the environment variables

export WESTON_DRM_MIRROR=false
export WESTON_DRM_PREFER_EXTERNAL=0
export WESTON_DRM_SINGLE_HEAD=0
export WESTON_DRM_MASTER_OUTPUT="HDMI-A-1"

Then write to /etc/xdg/weston/weston.ini

[core]
backend=drm-backend.so
require-input=true         # Must connect an input device to start
require-outputs=true       # Must detect output devices
idle-time=0                # Disable screen sleep
repaint-window=16          # Repaint window, ~60Hz refresh rate

# Disable auto-detection
use-udev=false             # Turn off udev auto-detection, manually control output

[output]
name=HDMI-A-1             # Specify the HDMI-A-1 interface
mode=1920x1080@60         # 1080p resolution, 60Hz refresh rate
transform=normal          # No rotation transform
scale=1.5                 # 150% scaling, adapts to high DPI

# Disable DSI interface
[output]
name=DSI-1
mode=off                  # Turn off this output

[shell]
panel-scale=2             # Panel elements 200% scaling
cursor-size=32            # Mouse pointer size
locking=false             # Disable screen locking
startup-animation=none    # Disable startup animation

[keyboard]
vt-switching=true         # Allow virtual terminal switching

[libinput]
touchscreen_calibrator=true  # Enable touchscreen calibration
enable-tap=true           	 # Enable tap gesture
natural-scroll=true          # Natural scroll direction

[device]
name=wch.cn USB2IIC_CTP_CONTROL   # Specific touch device
rotation=normal                   # Normal orientation

Test​

Boot

Run 3D test

In practical applications, the DSI driver output should be turned off

Tips: Note, there's a pitfall here that prevented my HDMI screen from starting at first​

Look at this Weston startup output log:

root@rk3576-buildroot:/# weston
Date: 2025-12-03 UTC
..........
[03:20:26.528] Output HDMI-A-1 (crtc 72) video modes:
               1024x600@59.8, preferred, 50.2 MHz
               1920x1080@60.0 16:9, 148.5 MHz
               1920x1080@59.9 16:9, 148.4 MHz
               1920x1080i@60.0, 74.2 MHz
               1920x1080i@60.0 16:9, 74.2 MHz
               1920x1080i@59.9 16:9, 74.2 MHz
               1920x1080@50.0, current, 148.5 MHz
               1920x1080@50.0 16:9, 148.5 MHz
               1920x1080i@50.0, 74.2 MHz
               1920x1080i@50.0 16:9, 74.2 MHz
               1280x1024@75.0, 135.0 MHz
               1280x720@60.0 16:9, 74.2 MHz
               1280x720@59.9 16:9, 74.2 MHz
               1280x720@50.0, 74.2 MHz
               1280x720@50.0 16:9, 74.2 MHz
               1024x768@75.0, 78.8 MHz
               1024x768@70.1, 75.0 MHz
               1024x768@60.0, 65.0 MHz
               832x624@74.6, 57.3 MHz
               800x600@75.0, 49.5 MHz
               800x600@72.2, 50.0 MHz
               800x600@60.3, 40.0 MHz
               800x600@56.2, 36.0 MHz
               720x576@50.0, 27.0 MHz
               720x576@50.0 4:3, 27.0 MHz
               720x576@50.0 16:9, 27.0 MHz
               720x480@60.0 4:3, 27.0 MHz
               720x480@60.0 16:9, 27.0 MHz
               720x480@59.9 4:3, 27.0 MHz
               720x480@59.9 16:9, 27.0 MHz
               640x480@75.0, 31.5 MHz
               640x480@72.8, 31.5 MHz
               640x480@60.0 4:3, 25.2 MHz
               640x480@59.9, 25.2 MHz
               720x400@70.1, 28.3 MHz
.....

Note this resolution

1024x600@59.8, preferred, 50.2 MHz is my screen's actual physical resolution and the default configuration. If you directly configure

# Force-specify output
[output]
name=HDMI-A-1
#mode=1920x1080@50
mode=1024x600@59.8
transform=normal
scale=1.5

At startup:

xkbcommon: ERROR: couldn't find a Compose file for locale "en_US.UTF-8" (mapped to "en_US.UTF-8")
could not create XKB compose table for locale 'en_US.UTF-8'.  Disabiling compose
xkbcommon: ERROR: couldn't find a Compose file for locale "en_US.UTF-8" (mapped to "en_US.UTF-8")
could not create XKB compose table for locale 'en_US.UTF-8'.  Disabiling compose
[  695.260819] dwhdmi-rockchip 27da0000.hdmi: use tmds mode
[  695.279593] rockchip-vop2 27d00000.vop: [drm:vop2_crtc_atomic_enable] Update mode to 1024x600p60, type: 11(if:HDMI0, flag:0x0) for vp0 dclk: 50250000
[  695.279636] rockchip-hdptx-phy-hdmi 2b000000.hdmiphy: hdptx_ropll_cmn_config bus_width:7aae4 rate:1485000
[  695.279810] rockchip-hdptx-phy-hdmi 2b000000.hdmiphy: hdptx phy pll locked!
[  695.279872] rockchip-hdptx-phy-hdmi 2b000000.hdmiphy: hdptx_ropll_cmn_config bus_width:7aae4 rate:502500
[  695.280061] rockchip-hdptx-phy-hdmi 2b000000.hdmiphy: hdptx phy pll locked!
[  695.280069] rockchip-vop2 27d00000.vop: [drm:vop2_crtc_atomic_enable] set dclk_vp0 to 50250000, get 50250000
[  695.280120] dwhdmi-rockchip 27da0000.hdmi: final tmdsclk = 50250000
[  695.280189] dwhdmi-rockchip 27da0000.hdmi: don't use dsc mode
[  695.280198] dwhdmi-rockchip 27da0000.hdmi: dw hdmi qp use tmds mode
[  695.280206] rockchip-hdptx-phy-hdmi 2b000000.hdmiphy: bus_width:0x7aae4,bit_rate:502500
[  695.285263] rockchip-hdptx-phy-hdmi 2b000000.hdmiphy: hdptx phy lane can't ready!
[  695.285271] phy phy-2b000000.hdmiphy.4: phy poweron failed --> -22
[  695.285278] dwhdmi-rockchip 27da0000.hdmi: dw_hdmi_qp_setup hdmi set operation mode failed
[  695.285317] dwhdmi-rockchip 27da0000.hdmi: Rate 50250000 missing; compute N dynamically
[  695.286726] dwhdmi-rockchip 27da0000.hdmi: Rate 50250000 missing; compute N dynamically
[  695.315462] dwhdmi-rockchip 27da0000.hdmi: use tmds mode

Note:

[  695.285263] rockchip-hdptx-phy-hdmi 2b000000.hdmiphy: hdptx phy lane can't ready!
[  695.285271] phy phy-2b000000.hdmiphy.4: phy poweron failed --> -22

The screen cannot start, so here's a lesson learned:

When testing a screen, the physical resolution may be incompatible; you need to test multiple resolutions to find a compatible one.

Key Configuration Points​

Importance of use-udev=false: By default, Weston auto-detects all connected display devices via udev. When set to false, Weston will only use the outputs explicitly specified in the configuration file, providing the basis for precise control.

Output Priority Control: When multiple [output] sections exist, Weston processes them in the order they appear in the configuration file. Placing the outputs to be disabled after the active output and setting mode=off ensures correct display control.

Scaling Configuration Strategy: Embedded devices often need to adjust the physical size of UI elements. The scale parameter allows you to independently control the scaling ratio of each output, which is especially important when connecting monitors with different DPIs.

DSI Exclusive Mode​

Opposite to HDMI exclusive, some applications need to use only the device's built-in screen, such as:

• Mobile devices or portable instruments
• Battery-powered devices that conserve power
• Application scenarios that don't need external displays

/etc/xdg/weston/weston.ini

[core]
backend=drm-backend.so

# Allow running without input devices
require-input=false

# Allow running without output devices
require-outputs=none

# Disable screen idle timeout by default
idle-time=0

# Key: Disable auto-detection of all connections
use-udev=false

# The repaint-window is used to calculate repaint delay(ms) after flipped.
#   value <= 0: delay = abs(value)
#   value > 0: delay = vblank_duration - value
repaint-window=-1

# Allow blending with lower drm planes
# gbm-format=argb8888

[shell]
# top(default)|bottom|left|right|none, none to disable panel
# panel-position=none

# Scale panel size
panel-scale=2

# Set cursor size
cursor-size=32

# none|minutes(default)|minutes-24h|seconds|seconds-24h
# clock-format=minutes-24h
clock-with-date=false

# Disable screen locking
locking=false

# Disable the desktop starting up animation
startup-animation=none

[libinput]
# Uncomment below to enable touch screen calibrator(weston-touch-calibrator)
# touchscreen_calibrator=true
# calibration_helper=/bin/weston-calibration-helper.sh

[keyboard]
# Comment this to enable vt switching
vt-switching=false

# Configs for auto key repeat
# repeat-rate=40
# repeat-delay=400
[output]
name=DSI-1
mode=480x800
transform=rotate-180
scale=0.2

# Explicitly disable DSI-1
[output]
name=HDMI-A-1
mode=off

Test​

Boot

Run 3D test

Key Points​

Rotation Configuration: The screen mounting orientation of embedded devices may differ. The transform parameter supports multiple rotation options:

• normal: No rotation
• rotate-90: 90 degrees clockwise
• rotate-180: 180 degrees
• rotate-270: 270 degrees clockwise
• flipped: Horizontal flip
• flipped-rotate-180: Combined transform

DPI Adaptation Strategy: Small, high-resolution screens require appropriate UI scaling. By experimenting with different scale values, find a UI element size with suitable physical dimensions. This configuration uses 0.2 (20%) scaling to ensure UI elements can be operated normally at 480x800 resolution.

Dual-Screen Mirrored Display​

Dual-screen mirrored display (mirror mode) is suitable for:

• Demonstration and teaching scenarios
• Synchronized display between the main console and an observation screen
• Troubleshooting and debugging

/etc/xdg/weston/weston.ini

[core]
backend=drm-backend.so
require-input=true
require-outputs=true
idle-time=0
repaint-window=16
mode=mirror
use-udev=true  # Dual-screen requires udev enabled

[output]
name=HDMI-A-1
mode=1920x1080
transform=rotate-270
scale=0.25

[output]
name=DSI-1
mode=1920x1080
transform=rotate-270
scale=0.25

[shell]
panel-scale=2
cursor-size=32
locking=false
startup-animation=none

[keyboard]
vt-switching=true

[libinput]
touchscreen_calibrator=true
enable-tap=true
natural-scroll=true

# Key fix: explicitly bind the touch device to HDMI output
[device]
name=wch.cn USB2IIC_CTP_CONTROL
output=HDMI-A-1  # Explicitly specify the HDMI screen
rotation=normal  # Adjust according to actual orientation

Test​

Boot

Run 3D test

Key Points​

Resolution Alignment: In mirror mode, the two outputs should use the same resolution; otherwise Weston will display at the lower resolution or scaled. This configuration uniformly uses 1920x1080 to ensure consistent display content.

Touch Input Binding: In a multi-screen environment, touch input needs to be explicitly bound to a specific screen. The output=HDMI-A-1 configuration ensures touch operations only affect the HDMI display, avoiding confusion in mirror mode.

Performance Optimization Considerations: Mirror mode requires the compositor to render the same content twice, which has some impact on system performance. Properly adjusting the repaint-window parameter can balance smoothness and system load.

Dual-Screen Extended Display​

Dual-screen extended display (extend mode) is suitable for:

• Multi-tasking work environments
• Separation of control panel and data display
• Complex professional application interfaces

The following is the configuration

weston.ini configuration

[core]
backend=drm-backend.so
require-input=true
require-outputs=true
idle-time=0
repaint-window=16
mode=extend  # Key: change to extend mode
use-udev=true

# HDMI screen (right side)
[output]
name=HDMI-A-1
mode=1920x1080
transform=rotate-270
scale=0.25
x=200  # DSI on the left, starting from DSI width
y=0

# DSI screen (left side)
[output]
name=DSI-1
mode=480x800
transform=rotate-270
scale=0.25
x=0
y=0

[shell]
panel-scale=2
cursor-size=32
locking=false
startup-animation=none

[keyboard]
vt-switching=true

[libinput]
touchscreen_calibrator=true
enable-tap=true
natural-scroll=true

[device]
name=wch.cn USB2IIC_CTP_CONTROL
output=HDMI-A-1  # Touch bound to HDMI screen
rotation=normal

Test​

At startup

Need to set environment variables to turn off mirror mode

# Environment variables
export WESTON_DRM_MIRROR=0         # Turn off mirror mode
export WESTON_DRM_PREFER_EXTERNAL=0 # Don't prefer external display
export WESTON_DRM_SINGLE_HEAD=0     # Enable multi-head support
pkill weston
weston &

After startup, the system recognizes two independent monitors, and the desktop can extend across screens. Each screen can run different applications, achieving a true multi-tasking environment.

Key Points​

Screen Layout Control: Weston's default screen layout may not match the actual physical layout. You can optimize it via the position parameters in weston.ini or by manually adjusting after startup.

Cross-Screen Window Management: In extend mode, windows can be moved between screens. You need to ensure the window manager and applications support the multi-screen environment.

Performance Considerations: Extend mode demands more graphics performance, especially when the two screens have very different resolutions. Rendering settings need to be adjusted based on hardware capabilities.

Summary​

Although Weston multi-screen configuration has some complexity, by deeply understanding its configuration principles and mastering the key parameters, you can achieve highly customized display solutions. Whether single-screen exclusive, dual-screen mirrored, or dual-screen extended, you need to comprehensively consider hardware characteristics, application requirements, and user experience.

In the actual configuration process, it is recommended to take an incremental testing approach: start from the basic configuration, verify individual functions, then gradually add complex features.

Demo Video​

https://www.bilibili.com/video/BV1m34VziE2s

1. Experiment Environment​

Core Hardware Acceleration Units​

The RK3576 chip integrates three key hardware acceleration units, each responsible for different stages of video processing:

• VPU (Video Processing Unit): Responsible for video decoding (e.g., H.264 hardware decoding); the core element mppvideodec invokes this unit.

• RGA (Image Acceleration Unit): Responsible for image scaling and format conversion (e.g., NV12→RGBA); can be enabled via mppvideodec parameters.

• GPU (Graphics Processing Unit): Responsible for video rendering and multi-frame compositing (e.g., glvideomixer compositing); managed by the Panfrost driver.

Demo Effect​

https://www.bilibili.com/video/BV1Z646zhEBP

1. Environment Information​

2. Implementation Principle​

1. Core component: gmrender-resurrect is a tool that receives DLNA service content and plays it via GStreamer, and can be directly configured as a DLNA client.

2. Hardware acceleration basis: GStreamer hardware-accelerated video playback has been implemented in advance, meeting the needs of HD stream decoding.

3. Bilibili screen cast adaptation: When Bilibili casts via DLNA, it sends an H264 stream packaged in FLV (similar to a live stream), which can be played directly with hardware acceleration via gmrender-resurrect.

Background and Problem​

• Device used: DshanPI-A1, running the Armbian system, with Wayland as the window system and an open-source GPU driver.

• Initial attempt: Using NoMachine for remote desktop, but there are two issues:

1. It creates a virtual desktop by default instead of the physical desktop;

2. It has poor Wayland support, launching the X11 desktop in compatibility mode, which prevents OpenGL from using GPU acceleration.

DshanPi-A1 focuses on AI education scenarios, providing a sustainable and evolving hardware and software ecosystem for teaching needs from beginner to advanced levels, in an integrated form of a "single-board computer + topic-based projects (paid)". It offers a mid-to-high-end performance SBC experience, including rich interfaces such as PCIe, USB3.0, dual gigabit Ethernet, HDMI, HDMI-IN, DP, and more.