Robotics & Embodied AI · entry 03/05

Learning to move

Robot skills come from three wells — human demonstrations, simulated experience, and pretrained models that read the internet — and the frontier is how hard the three can be combined.

Imitation: buy the behavior

The most reliable path to a robot skill is a person demonstrating it — teleoperating the arm through fifty towel-folds while the system records observations and actions, then supervised learning on the pairs. Behavior cloning is simple and startlingly effective, with one classic flaw: the demonstrator never shows how to recover from states they never entered, so small errors compound and drift the robot into unfamiliar territory where it has no idea what to do. Fixes are practical — collect corrections in exactly the states the policy visits, add noise during demos so recovery gets demonstrated — and modern diffusion-based policies handle the multi-modality of human demonstrations (three valid ways to reach; don't average them into a fourth that hits the table).

Simulation: manufacture the experience

Simulators offer what reality can't: millions of episodes, no broken hardware, resets for free — RL's favorite conditions. The catch is the sim-to-real gap: policies exploit the simulator's convenient physics and fail on real friction. The workhorse answer is domain randomization — vary masses, friction, lighting, latency across millions of runs until the policy stops trusting any one physics and becomes robust to the family that contains reality. Locomotion made it standard practice: quadrupeds and bipeds now train for the equivalent of years in simulation and walk off the loading dock in an afternoon. Manipulation transfers less cleanly — contact is exactly where simulators are weakest.

Priors: read the internet first

The newest well is pretrained knowledge. Vision-language-action models start from a VLM that already knows what objects are, what "put the apple in the bowl" means, and roughly how tasks decompose — then fine-tune on robot trajectories so that knowledge grounds into motor commands. The bet is that semantic understanding is the expensive part and the internet already paid for it; early evidence is genuinely encouraging (instructions generalize to unseen objects), and cross-robot datasets pool demonstrations across embodiments — with transfer that is real but uneven, better on some task families than others.

Failure mode

Counting demos instead of covering states. Teams report "trained on 500 demonstrations" as if volume were the metric, but fifty demos spanning lighting, positions, occlusions, and recoveries beat five hundred takes of the same sunny-day grasp. Coverage of the state space — especially the embarrassing corners — is what generalization is made of; a policy is only as brave as its dataset's worst day.