DshanPI-A1 Audio Recording, Playback and Noise Analysis
Audio Playback
Speaker Device
Let's first look at how to use this speaker. First, list the audio playback devices.
aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: rockchipes8388 [rockchip-es8388], device 0: dailink-multicodecs ES8323 HiFi-0 [dailink-multicodecs ES8323 HiFi-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: rockchiphdmiin [rockchip,hdmiin], device 0: 2a640000.sai-dummy_codec dummy_codec-0 [2a640000.sai-dummy_codec dummy_codec-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 2: rockchipdp0 [rockchip-dp0], device 0: rockchip-dp0 spdif-hifi-0 [rockchip-dp0 spdif-hifi-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 3: rockchiphdmi [rockchip-hdmi], device 0: rockchip-hdmi i2s-hifi-0 [rockchip-hdmi i2s-hifi-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
It can be seen that a total of 4 audio cards are detected (card 0 ~ card 3)
- Play audio to speaker/headphones: use
card 0, device 0 - HDMI audio output: use
card 3, device 0 - DisplayPort audio: use
card 2, device 0 - Capture HDMI input audio: use
card 1, device 0
The speaker is card0: subdevice#0, and the corresponding ALSA device is hw:0,0
**** List of PLAYBACK Hardware Devices ****
card 0: rockchipes8388 [rockchip-es8388], device 0: dailink-multicodecs ES8323 HiFi-0 [dailink-multicodecs ES8323 HiFi-0]
Subdevices: 1/1
Subdevice #0: subdevice #0
Create and Play a Simple Test Tone
# Create a 1kHz sine wave WAV file at 8kHz sample rate (5 seconds)
ffmpeg -f lavfi -i "sine=frequency=1000:duration=5" -c:a pcm_s16le -ar 8000 test_tone.wav
# Play the file
aplay test_tone.wav
Playing WAVE 'test_tone.wav' : Signed 16 bit Little Endian, Rate 8000 Hz, Mono
The test commands all succeeded, but no sound was heard. We need to check the properties of this speaker. It is very likely an audio routing issue. First, we need to determine the troubleshooting approach:
The audio chip (ES8388) is designed with:
- Software control layer:
Speaker Switch- controls whether the audio stream is sent to the chip - Hardware control layer:
OUT1/OUT2 Switch- controls the chip's physical pin output
Audio Playback Debugging
First, open the audio visualization tool to take a look.
alsamixer -c 0
It can be seen that the playback status is abnormal: MM 00. We need to further confirm which configuration is the problem. Use the PulseAudio control tool.
pactl list short sinks
0 alsa_output.0.HiFi__hw_rockchipes8388__sink module-alsa-card.c s16le 2ch 44100Hz SUSPENDED
1 alsa_output.1.stereo-fallback module-alsa-card.c s16le 2ch 44100Hz SUSPENDED
Some explanations:
- Sink: the endpoint of audio output
- PulseAudio architecture
Application -> Audio stream -> PulseAudio server -> Sink -> Hardware
(player) (mixer) (output device) (sound card)
Status description
- RUNNING: audio is playing
- IDLE: idle, ready
- SUSPENDED: suspended, power-saving mode
- UNLINKED: not connected
Device details
- sink 0:
alsa_output.0.HiFi__hw_rockchipes8388__sink- Corresponds to ALSA sound card 0 (ES8388 audio chip)
- High fidelity (HiFi) output
- sink 1:
alsa_output.1.stereo-fallback- Fallback/backup output device
- Used when the primary device is unavailable
Both audio sinks are in SUSPENDED state
0 ... SUSPENDED
1 ... SUSPENDED
SUSPENDED state means:
- PulseAudio thinks there is no audio stream to play
- To save power, it automatically suspends audio output
- The system is ready to receive audio, but there is currently no active audio stream
Root cause chain:
-
At system startup -> PulseAudio loads audio devices
-
No active audio stream -> PulseAudio suspends the device (SUSPENDED)
-
Hardware routing not activated -> OUT1/OUT2 switches are off by default
-
When starting to play audio:
- PulseAudio wakes up the device
- But the hardware switches (OUT/OUT2) are still off
- Need to manually
amixer -c 0 sset 'OUT1' on
After my testing, after turning on
amixer -c 0 sset 'OUT2' onthe speaker audio can be played normally. Here as a record, I list some useful commands I used during debugging.
#1. View more detailed sink information
pactl list sinks
Sink #0
...
Sink #1
...
# 2. View current audio streams
pactl list sink-inputs
#3. ALSA content control
amixer -c 0 scontents
...
amixer -c 0 get 'Speaker'
amixer -c 0 get 'Master'
amixer -c 0 get 'Headphone'
There are multiple solutions to this problem, listed below.
Solution 1: Prevent auto-suspend
# Edit PulseAudio configuration
vi /etc/pulse/default.pa
# Prevent auto-suspend
load-module module-suspend-on-idle timeout=0 # 0 means never suspend
# Increase timeout
load-module module-suspend-on-idle timeout=3600 # 1 hour
# Restart PulseAudio
pulseaudio -k
pulseaudio --start
Solution 2: Automatically activate hardware at startup
# Create startup script /etc/pulse/audio-init.sh
#!/bin/bash
# Wait for PulseAudio to start
sleep 3
# Activate hardware output
amixer -c 0 sset 'OUT2' on
amixer -c 0 sset 'Speaker' on
# Set appropriate volume
amixer -c 0 sset 'Output 2' 90%
Solution 3: Use udev rules
# Create /etc/udev/rules.d/90-audio.rules
ACTION=="add", SUBSYSTEM=="sound", KERNEL=="card0", \
RUN+="/usr/bin/amixer -c 0 sset 'OUT2' on"
# Takes effect after reboot
Solution 4: Simplest temporary test
# Activate the device before playing
amixer -c 0 sset 'Speaker' on
amixer -c 0 sset 'OUT2' on
amixer -c 0 sset 'Output 2' 90%
# You can also permanently save settings (did not take effect)
alsactl store
Summary: The problem is actually:
- Software layer: PulseAudio is normal
- Driver layer: ALSA correctly identifies the device
- Hardware layer: OUT2 physical switch needs to be manually activated
tips: Audio routing
# Assume there are multiple audio devices:
# 0 - built-in speaker
# 1 - USB headset
# 2 - HDMI output
# Send Chrome audio to the headset
pactl move-sink-input $(pactl list short sink-inputs | grep chrome | awk '{print $1}') 1
# Send music player to HDMI
pactl move-sink-input $(pactl list short sink-inputs | grep spotify | awk '{print $1}') 2
Audio Recording
The hardware used is the 100ask 200w USB camera + audio MEMS integrated module, as shown in the figure below.
Device Information Acquisition
First, we need to obtain the information of the entire MEMS. It communicates with RK3576 via USB. There are several ways to see its information.
v4l2-sysfs-path
Video device: video36
video: video37
sound card: hw:4
pcm capture: hw:4,0
mixer: hw:4
Video device: video37
sound card: hw:4
pcm capture: hw:4,0
mixer: hw:4
.....
alsactl info
......
- card: 4
id: Camera
name: USB 2.0 Camera
longname: lihappe8 Corp. USB 2.0 Camera at usb-xhci-hcd.8.auto-1.2.2, high speed
driver_name: USB-Audio
mixer_name: USB Mixer
components: USB038f:0541
controls_count: 4
pcm:
- stream: CAPTURE
devices:
- device: 0
id: USB Audio
name: USB Audio
subdevices:
- subdevice: 0
name: subdevice #0
.....
It can be seen that the name of this device in ALSA is hw:4,0
Audio Recording Test
# Record a segment of background noise
arecord -D hw:4,0 -f S16_LE -r 8000 -c 2 -d 10 noise
Recording WAVE 'noise.wav' : Signed 16 bit Little Endian, Rate 8000 Hz, Stereo
aplay noise.wav
Playing WAVE 'noise.wav' : Signed 16 bit Little Endian, Rate 8000 Hz, Stereo
A clear "squeak" sound can be heard. The self-noise of this MEMS is still quite large. Noise reduction is needed later before normal voice communication can be performed.
Noise Analysis
Use sox to generate a spectrogram.
sox noise.wav -n spectrogram -o noise_spectrogram.pn
# Global spectrum
sox noise.wav -n spectrogram -d 10 -x 1200 -z 80 -o noise_full.png
sox noise.wav -r 2000 -c 1 noise_2k.wav
sox noise_2k.wav -n spectrogram -d 10 -x 1200 -z 80 -o noise_low.png
Overall observation: the background noise is "approximately white noise + strong low-frequency peaks + slight mid-frequency texture noise"
1. There is a very obvious low-frequency energy blob around 0~200Hz (especially <100Hz)
This generally means:
- Mechanical/power-related noise
- Fan, vibration, chassis resonance
- Power ripple (50Hz / 60Hz + fundamental)
- Microphone directivity/enclosure coupling amplifying ultra-low frequency noise
2. The 1kHz~4kHz range is random noise (approximately white noise) with low energy
This indicates the microphone self-noise is typical:
- MEMS microphone inherent noise
- ADC self-noise
- Amplifier input noise
This part is the system noise floor.
3. High frequency (>6kHz) has no strange peaks at all
This is very good -> no obvious:
- Digital EMI
- Clock leakage
- Sampling jitter noise
4. No obvious "howling mode" appears
None of the three figures show typical howling characteristics:
- Howling is usually a fixed-frequency bright line that remains unchanged
- The figures only show a pulse at the starting moment (may be the click sound when recording starts)
No stable peak persists over time.
Per-figure analysis
(A) First figure: Full spectrum (up to 4kHz)
The characteristics look like:
Overall reddish/purple, noise density is high but uniform Obvious vertical line around 50Hz 100Hz, 150Hz also have slight energy
-> This is almost certainly:
Power frequency noise (50/60Hz) + harmonics (100/150Hz)
Reasons:
- USB power supply brings a lot of 50/60Hz hum
- Poor isolation of the sound card or microphone analog front-end
- Unclean ground (such as USB common ground loop)
(B) Second figure: Version with narrowed dynamic range (-80dBFS)
This figure additionally exposes:
There is a very narrow horizontal line at 1.8kHz ~ 2.2kHz
Very faint, but stably present.
This indicates:
System clock/PLL interference leakage
Common in:
- I2S/MCLK leakage
- Clock coupling of the microphone on the PCB
- Digital power noise superimposed on the microphone analog part
This part will not cause howling, but will reduce SNR.
(C) Third figure: Low frequency to 1000Hz
Very typical:
The noise in the <150Hz region is much higher than other frequency bands
Like a large "bulb" shape, very obvious.
This indicates:
Low-frequency vibration + power noise are the main noise floor sources
Including:
- Chassis vibration, fan, tabletop resonance
- Power 50/60Hz + harmonics
- The microphone's own LF roll-off is insufficient
Comprehensive judgment: Background noise composition ratio
| Noise type | Ratio | Feature |
|---|---|---|
| Low-frequency mechanical/power noise (<200Hz) | 50% | Largest source, from power, chassis, vibration |
| Power frequency leakage 50/60Hz + harmonics | 25% | Strongest fixed peak in the figure |
| Microphone inherent white noise | 20% | Random noise scattered in 1k~4kHz |
| Digital clock leakage (around 2kHz) | 5% | A very weak but visible thin line |
PSD Noise Model Analysis
Use Python to generate a PSD noise model.
import numpy as np
import matplotlib.pyplot as plt
from scipy.io import wavfile
from scipy.signal import welch, find_peaks, spectrogram, butter, sosfilt
import IPython.display as ipd
import os
rate, data = wavfile.read("/mnt/data/noise.wav")
if data.ndim>1:
data = data.mean(axis=1)
data = data.astype(np.float32)
N = len(data)
duration = N / rate
# Calculate overall RMS and dBFS
# Assuming 16-bit PCM if dtype was int16; determine scale
# infer max possible value from original dtype by reloading header:
import struct
# Determine dtype max
# but we'll normalize by max of int16 if dtype came as int16, else use max(abs(data))
max_possible = 32768.0
rms = np.sqrt(np.mean(data**2))
dbfs_rms = 20*np.log10(rms / max_possible) if rms>0 else -np.inf
# Welch PSD
f, Pxx = welch(data, fs=rate, nperseg=4096, scaling='density')
# find peaks in PSD (in linear)
peaks, props = find_peaks(Pxx, height=np.max(Pxx)*0.15, distance=5)
peak_freqs = f[peaks]
peak_heights = props['peak_heights']
# Find dominant low-frequency peak under 500Hz
low_idx = np.where(f<=500)[0]
low_f = f[low_idx]
low_P = Pxx[low_idx]
lp_peaks, lp_props = find_peaks(low_P, height=np.max(low_P)*0.2)
lp_freqs = low_f[lp_peaks]
lp_heights = lp_props['peak_heights']
# Short-time energy to find transient (e.g., first second pulse)
frame_ms = 20
frame_len = int(rate * frame_ms/1000)
hop = frame_len//2
frames = []
for start in range(0, N-frame_len, hop):
frames.append(np.sum(data[start:start+frame_len]**2))
frames = np.array(frames)
frame_times = (np.arange(len(frames))*hop)/rate
# detect where energy spikes relative to median
median_e = np.median(frames)
spikes = np.where(frames > median_e*8)[0] # 8x median
spike_times = frame_times[spikes]
# Spectrogram
f_s, t_s, Sxx = spectrogram(data, fs=rate, nperseg=2048, noverlap=1024, scaling='density', mode='magnitude')
# Plot PSD with peaks marked
plt.figure(figsize=(10,5))
plt.semilogy(f, Pxx, color='tab:orange')
plt.scatter(peak_freqs, peak_heights, color='k', zorder=5)
for pf, ph in zip(peak_freqs, peak_heights):
plt.text(pf, ph*1.1, f"{pf:.0f} Hz", fontsize=8, ha='center')
plt.xlim(0, rate/2)
plt.xlabel("Frequency (Hz)")
plt.ylabel("PSD")
plt.title("Welch PSD with detected peaks")
plt.grid(alpha=0.3)
plt.tight_layout()
plt.savefig("/mnt/data/psd_peaks.png")
# Plot spectrogram (dB)
Sxx_db = 20*np.log10(Sxx + 1e-12)
plt.figure(figsize=(10,5))
plt.pcolormesh(t_s, f_s, Sxx_db, shading='gouraud')
plt.colorbar(label='dB')
plt.ylim(0, 4000)
plt.xlabel("Time (s)")
plt.ylabel("Frequency (Hz)")
plt.title("Spectrogram (dB)")
plt.tight_layout()
plt.savefig("/mnt/data/spectrogram_db.png")
# Prepare summary
summary = {
"sampling_rate": rate,
"duration_s": duration,
"rms": float(rms),
"dbfs_rms": float(dbfs_rms),
"dominant_peaks_hz": [float(p) for p in peak_freqs[:8]],
"dominant_peaks_vals": [float(p) for p in peak_heights[:8]],
"low_freq_peaks_hz": [float(p) for p in lp_freqs],
"low_freq_peaks_vals": [float(p) for p in lp_heights],
"spike_times_s": [float(s) for s in spike_times[:10]],
"spectrogram_image": "/mnt/data/spectrogram_db.png",
"psd_image": "/mnt/data/psd_peaks.png"
}
import json
with open("/mnt/data/noise_analysis_summary.json","w") as f:
json.dump(summary, f, indent=2)
# Display small tables and figures
from caas_jupyter_tools import display_dataframe_to_user
import pandas as pd
df_peaks = pd.DataFrame({
"freq_hz": peak_freqs,
"psd_val": peak_heights
})
display_dataframe_to_user("Detected PSD Peaks", df_peaks.head(20))
plt.figure(figsize=(10,3))
plt.plot(frame_times, 10*np.log10(frames+1e-12))
plt.xlabel("Time (s)")
plt.ylabel("Frame energy (dB)")
plt.title("Short-time frame energy (20ms frames)")
plt.grid(alpha=0.3)
plt.tight_layout()
plt.savefig("/mnt/data/frame_energy.png")
plt.show()
# Show audio player
ipd.display(ipd.Audio("/mnt/data/noise.wav"))
summary
Key quantitative results obtained:
- Sample rate: 8000 Hz
- Duration: 10.0 s
- Overall RMS = 223.11 (samples), converted to -43.34 dBFS (normalized to 16-bit full scale 32768): indicates the noise floor is moderately high.
- Detected significant frequency peaks (Welch PSD peak, in order of intensity, listing the first few):
- approximately 74.2 Hz, 85.94 Hz, 103.52 Hz, 113.28 Hz, 146.48 Hz, 167.97 Hz, and smaller 396 Hz, etc.
- Main peaks in the low-frequency band (≤500 Hz): 85.94, 95.70, 103.52, 113.28, 146.48 Hz - multiple near-frequency peaks, like a collection of power frequency/switching power supply harmonics or mechanical resonances.
- Transient energy peaks (short-time energy frames, 20 ms) detected at: approximately 0.36-0.38 s there are several short pulses (may be startup click/human touch/transient events).
Conclusion: The noise consists of obvious low-frequency "hum/ripple/vibration" + broadband white noise (mid-to-high frequency), with low frequency being the main energy location.
Noise Reduction Scheme
1. Add a 2nd-order high-pass filter (fc = 80 Hz) to the audio chain
- Directly remove most of the mechanical/power low-frequency without damaging the voice bandwidth (voice is mainly >100Hz).
- Implementation: use the butter/sos in the DSP library or implement biquad yourself (Direct Form I or II).
2 Targeted notch filter
- If there are still a few extremely narrow strong peaks after HPF (such as 103Hz), add one or two notches with Q=20~40. The higher the Q, the narrower the bandwidth, and the less it damages adjacent frequency bands, but the coefficients are closer to 1.
- Consider real-time requirements: need to cascade
hp -> notch(strong peak 1) -> notch(strong peak 2).
3 The essence is still hardware
- Replace or improve the analog power supply (low-noise LDO, more bypass capacitors, star ground) or use a digital MEMS microphone with better SNR.
- On embedded boards, keep switching power supply traces as far away from microphone differential/analog lines as possible.
Considerations for real-time audio quality
- If very low latency is required (e.g., echo cancellation / real-time loopback), use IIR (biquad) one-way real-time implementation (extremely low latency), but note that the phase will change. If phase change is not allowed (e.g., for more accurate positioning), using FIR + zero-phase will have a latency cost.
Reference Filter Coefficients
Note: The coefficients given below are in the standard second-order section (biquad) format, arranged as
[b0, b1, b2, a0, a1, a2](a0 has been normalized to 1 or needs to be normalized when given). When implementing, usually normalize a0 to 1, and then implement according to Direct Form I/II.
High-pass: 2nd-order Butterworth (fc = 80 Hz, fs = 8000 Hz)
sos_hp first section coefficients (a0 normalized to 1):
b0 = 0.9565432255568767
b1 = -1.9130864511137533
b2 = 0.9565432255568767
a0 = 1.0
a1 = -1.911197067426073
a2 = 0.9149758348014336
Several notch filters (notch, Q=30)
(Three examples, taken from detected peak positions) Format is the same as above (b0,b1,b2,a0,a1,a2), a0 has been normalized to 1 in the output below:
- notch @ 103.515625 Hz
b0 = 0.998648305437691
b1 = -1.99069933085876
b2 = 0.998648305437691
a0 = 1.0
a1 = -1.99069933085876
a2 = 0.9972966108753819
- notch @ 146.484375 Hz
b0 = 0.9980904047539934
b1 = -1.9829844796660758
b2 = 0.9980904047539934
a0 = 1.0
a1 = -1.9829844796660758
a2 = 0.9961808095079867
- notch @ 74.21875 Hz
b0 = 0.9990299707952924
b1 = -1.9946663265604048
b2 = 0.9990299707952924
a0 = 1.0
a1 = -1.9946663265604048
a2 = 0.9980599415905849
This noise analysis is the necessary material for traditional spectral subtraction audio filtering. Using the standard rnnoise AI noise reduction does not require it, but if you improve rnnoise yourself and do model fine-tuning, you need to refer to it to achieve better noise reduction results.
