DshanPI-A1 Review Part 2: Gesture Recognition Programming Environment Setup and Screen Debugging
In this review, I will install the necessary tools for the gesture recognition system and debug the screen.
Hardware and Environment Preparation
Before starting, let's clarify the equipment and environment on hand:
-
Core board: Dshanpi-A1, with the Rockchip RK3576 chip as the main SoC.
-
Screen: A 480x800 resolution MIPI screen.
-
System: Buildroot Linux system.
-
Official SDK
Install Development Tools
Here is the list for this time:
| Package/Configuration Category | Recommended Options and Purpose |
|---|---|
| Python Environment | python3: Core interpreter. python-pip: Used to install Python packages not included in Buildroot. python-numpy: Provides efficient numerical computation support for OpenCV and other libraries. python-setuptools: A base build dependency for some Python packages. |
| Computer Vision and Image Processing | opencv4: Be sure to enable python3 support. Provides the core computer vision library for image processing and gesture recognition algorithms. opencv4 contrib modules: Includes additional, more advanced algorithms. |
| Camera and Display Support | gstreamer1 and related plugins: Build pipelines for camera image capture and screen display. gst1-plugins-base, gst1-plugins-good, gst1-plugins-bad, gst1-plugins-ugly: Provide a rich set of codecs and functional elements. gst1-python: Allows creating and manipulating GStreamer pipelines in Python. |
SDK Configuration Process
1. Select the Chip Type
./build.sh chip
2. Enter buildroot Configuration
cd buildroot
make menuconfig
3. Select Target packages
4. Install the Python Environment
When you cannot find the installation path, press the / key to search:
Enter python3 to search:
Enter the displayed Location path to configure:
5. Save the Configuration and Build
make
Return to the SDK main directory and run:
./build.sh rootfs
./build.sh updateimg
Finally, flash and run it on the development board.
Development Board Debugging
Check Tool Installation
python3 --version
pip3 --version
python3 -c "import numpy; print('NumPy version:', numpy.__version__)"
python3 -c "import cv2; print('OpenCV version:', cv2.__version__)"
Screen Debugging
Problem Analysis
The system is already running the Weston compositor, which means we have a graphical interface environment. Attempting to directly operate the FrameBuffer (/dev/fb0) is ineffective because Weston has already occupied the display interface.
Through system inspection, we found:
- The
/dev/fb0device exists - The screen status is
connected - The resolution is 480x800
Solution: GStreamer + Wayland
GStreamer Basic Test
gst-launch-1.0 videotestsrc pattern=smpte ! video/x-raw,width=480,height=800 ! waylandsink sync=false
Camera Direct-to-Display Test
gst-launch-1.0 v4l2src device=/dev/video11 ! video/x-raw,width=640,height=480 ! videoconvert ! waylandsink sync=false
Note: Please replace the device parameter with your camera's device node
Test Script
#!/usr/bin/env python3
# fixed_display_test.py
import subprocess
import time
import os
def check_camera_devices():
"""Check available camera devices"""
print("=== Camera Device Check ===")
try:
# Use v4l2-ctl to check devices
result = subprocess.run(["v4l2-ctl", "--list-devices"],
capture_output=True, text=True)
if result.returncode == 0:
print("Found video devices:")n print(result.stdout)
else:
print("v4l2-ctl command execution failed")
except Exception as e:
print(f"Failed to check camera devices: {e}")
# Test common camera devices
camera_devices = ["/dev/video11", "/dev/video0", "/dev/video1", "/dev/video2"]
print("\nTesting camera devices:")
for device in camera_devices:
if os.path.exists(device):
print(f"Testing device: {device}")
try:
# Try to test the camera using GStreamer
cmd = [
"gst-launch-1.0",
"-v",
"v4l2src", f"device={device}", "!",
"video/x-raw,width=640,height=480,framerate=15/1", "!",
"videoconvert", "!",
"waylandsink", "sync=false"
]
process = subprocess.Popen(cmd)
time.sleep(3) # Display for 3 seconds
process.terminate()
process.wait()
print(f" {device}: Camera working normally")
return device
except Exception as e:
print(f"{device}: Test failed - {e}")
else:
print(f"{device}: Device does not exist")
return None
def test_static_patterns():
"""Test static patterns (will not change)"""
print("\n=== Static Pattern Test ===")
# Set Wayland environment
os.environ['WAYLAND_DISPLAY'] = 'wayland-0'
# Test static patterns (will not change)
static_patterns = [
("smpte100", "SMPTE 100% color bars"),
("ball", "Clock pattern"),
("blink", "Blink pattern"),
("pinwheel", "Pinwheel pattern"),
("spokes", "Spokes pattern"),
]
for pattern, description in static_patterns:
print(f"Displaying: {description}")
try:
cmd = [
"gst-launch-1.0",
"videotestsrc", f"pattern={pattern}", "!",
"video/x-raw,width=480,height=800,framerate=15/1", "!",
"waylandsink", "sync=false"
]
process = subprocess.Popen(cmd)
time.sleep(3)
process.terminate()
process.wait()
print(f"{description} displayed successfully")
except Exception as e:
print(f"{description} display failed: {e}")
def test_custom_resolution():
"""Test custom resolution display"""
print("\n=== Custom Resolution Test ===")
resolutions = [
(480, 800, "Portrait 480x800"),
(800, 480, "Landscape 800x480"),
(640, 480, "Standard 640x480"),
(400, 800, "Portrait 400x800"),
]
for width, height, desc in resolutions:
print(f"Testing resolution: {desc}")
try:
cmd = [
"gst-launch-1.0",
"videotestsrc", "pattern=smpte100", "!",
f"video/x-raw,width={width},height={height},framerate=15/1", "!",
"videoconvert", "!",
"waylandsink", "sync=false"
]
process = subprocess.Popen(cmd)
time.sleep(2)
process.terminate()
process.wait()
print(f" {desc} displayed successfully")
except Exception as e:
print(f" {desc} display failed: {e}")
if __name__ == "__main__":
print("=" * 50)
# 1. Check camera
camera_device = check_camera_devices()
# 2. Test static patterns
test_static_patterns()
# 3. Test different resolutions
test_custom_resolution()
print("\n" + "=" * 50)
if camera_device:
print(f"Available camera device: {camera_device}")
else:
print("No available camera device found")
print("All tests completed")
Run Results
I will put the demo video in the attachments
Bonus Chapter: FileZilla File Transfer
FileZilla Connection Settings
FileZilla - The free FTP solution
Check SSH Service
ss -tuln | grep 22
Network Sharing Settings
Select the network that can access the internet, and click Properties:
Connect to the Development Board
ifconfig
Connect using FileZilla:
- IP: Development board IP address
- Username: root
- Password: rockchip
- Port: 22
Summary
Looking back at the entire debugging process, the following key points are worth special attention:
-
Display Path Selection: On systems running a compositor such as Weston, prioritize the GStreamer + waylandsink solution for displaying images, rather than directly operating the FrameBuffer.
-
Camera Device Node: Be sure to use the
v4l2-ctl --list-devicescommand to confirm the device node corresponding to the camera, and specify it correctly in the code. -
Screen Resolution: Note that the screen resolution detected by the system (which can be queried via
cat /sys/class/drm/card0-DSI-1/modes) may be slightly different from the physical resolution. When creating display frames, use this as the reference or make adjustments accordingly.
At this point, we have successfully set up the gesture recognition programming environment on the Dshanpi A1 development board and resolved the display issues with the MIPI screen and camera. Although the process was full of twists and turns, it laid a solid foundation for the subsequent actual writing of gesture recognition algorithms. I hope my experience can be of help to everyone!