A live sensor-fusion display combining a 360° LiDAR (FHL-LD19) and a 24GHz mmWave radar (RD-03D) on a single ESP32-S3, streamed over WiFi to a Clockwork Pi uConsole running a Python/Pygame receiver.

The ESP32 hosts its own WiFi access point and streams both sensor streams as a compact binary UDP protocol. The uConsole renders a full-screen radar-style display: an accumulated 360° LiDAR point cloud, live radar target tracking with speed readout, distance rings, zoom, and a synthesized sonar audio layer that gets faster and higher-pitched the closer a radar target gets.

Features

  • 360° LiDAR point cloud (FHL-LD19), accumulated over the last full rotation so the room outline stays visible between individual scan packets.
  • 24GHz mmWave radar tracking (RD-03D), up to 3 simultaneous targets with live speed readout (m/s).
  • Compact binary UDP protocol for both sensors (no JSON) to keep packet rate/size low and avoid WiFi congestion on the ESP32 SoftAP.
  • Non-blocking radar configuration — explicitly enables the RD-03D's multi-target tracking mode on boot and re-applies it periodically, without ever blocking the main loop.
  • Zoomable display — arrow keys Up/Down to zoom in/out live.
  • Synthesized sonar audio (NumPy sample synthesis, no audio files):
    • a quiet idle "heartbeat" ping
    • a proximity alarm that gets faster and higher-pitched the closer a radar target gets (motion-tracker style)
  • Diagnostic HUD: LiDAR/Radar connection status, packet rates, error counts.

Hardware

Part Notes
ESP32-S3 (16MB flash / 8MB PSRAM recommended) Hosts WiFi AP + both sensor UARTs
FHL-LD19 360° LiDAR UART, 230400 baud
RD-03D 24GHz mmWave radar (Ai-Thinker) UART, 256000 baud, needs 5V
Clockwork Pi uConsole (or any Linux machine with WiFi + audio) Runs the Python receiver

Wiring

Signal ESP32-S3 Pin
RD-03D TX → ESP32 RX GPIO4
RD-03D RX ← ESP32 TX GPIO5
LD19 TX → ESP32 RX GPIO15
LD19 RX ← ESP32 TX GPIO16 (mostly unused, LD19 only transmits)

The RD-03D requires a stable 5V supply — 3.3V is not enough to run it reliably.

Network

The ESP32 runs as its own WiFi access point:

  • SSID: RadarSystem
  • Password: radarlidar2026sicher
  • ESP32 IP: 192.168.4.1
  • Expected uConsole/client IP: 192.168.4.2 (first DHCP lease) — the firmware sends unicast UDP to this fixed address, since broadcast proved unreliable under WiFi load. If your client doesn't get .2, update clientIP in firmware/lidar_radar.ino.
Stream UDP Port
LiDAR 5005
Radar 5006

Setup

1. Flash the firmware

Open firmware/lidar_radar.ino in the Arduino IDE (ESP32 board package installed), select your ESP32-S3 board, and flash.

2. Connect the receiver device to the ESP32's WiFi

nmcli device wifi connect "RadarSystem" password "radarlidar2026sicher"

3. Install Python dependencies

pip install -r requirements.txt --break-system-packages

4. Run the receiver

python3 receiver/uconsole_receiver.py

Runs fullscreen. Press ESC to quit, ↑/↓ to zoom.

Binary UDP protocol

LiDAR packet:

Bytes Field
1 point count N (uint8)
5 × N per point: int16 angle (0.01° units, LE), uint16 distance (mm, LE), uint8 intensity

Radar packet:

Bytes Field
1 target count N (uint8, 0-3)
6 × N per target: int16 x_mm (LE), int16 y_mm (LE), int16 speed_mm_s (LE)

Calibration notes

  • ANGLE_OFFSET_DEG in uconsole_receiver.py (currently 90) corrects for the fixed rotational offset between the LD19's own zero-angle reference and the screen's "up" direction. If you remount the LiDAR, recalibrate by pointing a known landmark at a known bearing and adjusting this constant — see the commit history / issues for the calibration method used.
  • The RD-03D is a Doppler radar — it can only detect motion, not stationary presence. Standing still (or moving perpendicular to the sensor, i.e. no radial velocity component) will make a target disappear from tracking. This is a hardware limitation, not a bug.

Known limitations

  • RD-03D real-world range/stability is often lower than the datasheet's 8m/±60° spec.
  • No persistent target ID/tracking across drops — each frame's targets are independent.