Robotics and embodied AI

In short: Embodied AI connects AI with a body: robots perceive with cameras and sensors, plan and act. New models combine language, vision and motion in one network.

In more detail: The world is harder than text: mistakes have physical consequences, data is scarce and real time is needed.

In detail

• Classic pipeline: perceive → map/localise (SLAM) → plan → control (PID, model predictive control) → motors.
• Learning: imitation learning from demonstrations, reinforcement learning in simulation, then transfer to reality (sim-to-real).
• Vision-language-action models (VLA): one model takes a camera image and an instruction (“clear the table”) and outputs movements (RT-2, OpenVLA, π0).
• Simulation: Isaac Sim, MuJoCo, Gazebo. Domain randomisation makes models more robust.
• Obstacles: little real data, safety, hardware cost, long chains of tasks.
• Applications: warehouses, industry, vacuum cleaners, drones, autonomous driving.
• Autonomous driving: sensor fusion of camera, radar, lidar, levels 0 to 5 of automation.

See also: Reinforcement learning, Computer vision: tasks and methods