projects / unitree g1 / 1
Hand Mimic — Real-Time Camera-to-Robot-Hand Teleoperation
A human hand is tracked by a webcam, and every finger movement is mirrored live to the robot's dexterous hand — with an annotated skeleton overlay showing exactly what's being sent to hardware.
- Real-time 21-landmark MediaPipe tracking mirrors finger articulation at ~30 ms latency.
- Scale-invariant pinch snapping and occlusion guards eliminate landmark loss inside fists.
- Direct RS-485 serial bus streaming to dual Inspire RH56 hands at 115,200 baud.
Phone Side · Inspire RH56 Hands
PC Side · MediaPipe Hand Tracking
Real-Time Hand Teleoperation — Dual Inspire RH56 dexterous hands mirroring human finger postures live at 30 Hz.
01Perception Pipeline — 21 Landmarks to Joint Angles
- Extracts 21 3D landmarks per hand at 30 Hz using GPU-accelerated MediaPipe perception.
- Finger triplet PIP interior angles map linearly to 0–1000 integer actuator registers.
Phone Side · Single Hand Close-Up
PC Side · 21-Landmark Tracking
Single Hand Teleoperation — Physical hand executing commanded joint values with live skeleton tracking.
Live Dual-Hand Articulation
Dual-Hand Independent Teleoperation — Dual Inspire RH56 hands articulating separate finger postures concurrently via real-time MediaPipe tracking.
02Retargeting & Pinch Detection
- Scale-invariant palm ratio detection snaps to calibrated hardware pinch presets automatically.
- Per-DOF exponential moving average smoothing suppresses sensor jitter on thumb channels.
Live Hardware Pinch Demo
Live Hardware Pinch Teleoperation — Single Inspire RH56 executing real-time thumb-to-index pinch tracking from laptop webcam input.
03Serial Hardware — RS-485 to the Hand
- Modbus-style register writes over RS-485 at 115200 baud decouple hand logic from body DDS.
- Hardware power grasp verification securely grips irregular objects like barcode scanners.
Live Hardware Grasp Demo
Physical Grasp Verification — Inspire RH56 executing adaptive power grasp to securely retrieve a handheld barcode scanner.