Processing sensory input enables robots to adjust their model of the world around them. In the case of a vacuum-cleaner robot, the furniture in the room may change day to day, so the robot must be able to adapt to chaotic environments.
Let’s say you own a futuristic housemaid robot, which comes with a few basic skills but also with the ability to learn new skills from human demonstrations. Maybe you’d like to teach it how to fold clothes.
Teaching a robot how to accomplish a new task is a tricky problem. Some immediate questions come to mind:
- Should the robot simply mimic a human’s sequence of actions? Such a process is referred to as imitation learning.
- How do a robot’s arms and joints match up to human poses? This dilemma is often referred to as the correspondence problem.
In this chapter, you’re going to model a task from human demonstrations while avoiding both imitation learning and the correspondence problem. Lucky you! You’ll achieve this task by studying a way to rank states of the world with a utility function, which takes a state and returns a real value representing its desirability. You’ll not only steer clear of imitation as a measure of success, but also bypass the complications of mapping a robot’s set of actions to that of a human (the correspondence problem).