Spatio-Temporal Motion Retargeting
for Quadruped Robots

This work presents a motion retargeting approach for legged robots, aimed at transferring the dynamic and agile movements to robots from source motions. In particular, we guide the imitation learning procedures by transferring motions from source to target, effectively bridging the morphological disparities while ensuring the physical feasibility of the target system. In the first stage, we focus on motion retargeting at the kinematic level by generating kinematically feasible whole-body motions from keypoint trajectories. Following this, we refine the motion at the dynamic level by adjusting it in the temporal domain while adhering to physical constraints. This process facilitates policy training via reinforcement learning, enabling precise and robust motion tracking. We demonstrate that our approach successfully transforms noisy motion sources, such as hand-held camera videos, into robot-specific motions that align with the morphology and physical properties of the target robots. Moreover, we demonstrate terrain-aware motion retargeting to perform BackFlip on top of a box. We successfully deployed these skills to four robots with different dimensions and physical properties in the real world through hardware experiments.

Q: What distinguishes STMR from previous works on motion retargeting?
A: Spatio-Temporal Motion Retargeting (STMR) lets quadruped robots imitate agile motions from videos or handcrafted animations, even when base motion is missing. It reconstructs the full-body movement—including the base and legs—from only joint poses and contact booleans, enabling motion retargeting directly from video. STMR then refines the temporal aspect of the motion to ensure it is physically feasible for the robot, especially crucial for dynamic movements with flight phases, such as hoping or back flipping. This allows robots to perform complex motions in the real world without manual tuning or per-motion optimization.