MuJoCo Arm Policy — RL Goal-Reaching Without an IK Solver

algorithm PPO (SB3) training ~53M timesteps obs/action 24-D / 7-D deploy DDS rt/arm_sdk

A PPO agent trained in a MuJoCo simulation of the G1 solves 7-joint inverse kinematics in real time — no analytical IK solver. WASDQE keys move a red goal dot in 3D; pressing R mirrors the sim arm onto the physical robot over DDS.

PPO reinforcement learning agent trained over 53M timesteps in MuJoCo physics.
Real-time 7-DOF joint space reaching without relying on analytical inverse kinematics.
Hardware mirroring streams policy actions over CycloneDDS to physical G1 arm motors.

MuJoCo Simulation · 3D Goal Reaching

MuJoCo Reinforcement Learning Policy — PPO agent solving 7-joint inverse kinematics in real time to reach a 3D goal target.

01Training Setup — PPO in MuJoCo

24-D observation space encompasses joint angles, velocities, end-effector pose, and target deltas.
Incremental joint deltas bounded to ±0.05 rad prevent jerk and ensure smooth physical execution.

02Sim-to-Real Deployment — DDS Streaming

Policy inference executes at 25 Hz against the live simulation state.
Streaming mode broadcasts LowCmd_ packets over rt/arm_sdk to actuate the real robot.

Sim-to-Real Deployment · Real Robot Arm Mirror

Physical Arm Mirroring — Real Unitree G1 arm mirroring MuJoCo policy joint trajectories over low-latency DDS.

03Reward Design — Dense Shaping

Dense reward shaping combines Euclidean goal distance with regularizers for upright wrist pitch.
Self-collision avoidance penalty prevents damaging contact with the robot torso.