19 Utility landscape

 

This chapter covers

  • Implementing a neural network for ranking
  • Image embedding with VGG16
  • Visualizing utility

Processing sensory input enables robots to adjust their model of the world around them. In the case of a vacuum-cleaner robot, the furniture in the room may change day to day, so the robot must be able to adapt to chaotic environments.

Let’s say you own a futuristic housemaid robot, which comes with a few basic skills but also with the ability to learn new skills from human demonstrations. Maybe you’d like to teach it how to fold clothes.

Teaching a robot how to accomplish a new task is a tricky problem. Some immediate questions come to mind:

  • Should the robot simply mimic a human’s sequence of actions? Such a process is referred to as imitation learning.
  • How do a robot’s arms and joints match up to human poses? This dilemma is often referred to as the correspondence problem.

In this chapter, you’re going to model a task from human demonstrations while avoiding both imitation learning and the correspondence problem. Lucky you! You’ll achieve this task by studying a way to rank states of the world with a utility function, which takes a state and returns a real value representing its desirability. You’ll not only steer clear of imitation as a measure of success, but also bypass the complications of mapping a robot’s set of actions to that of a human (the correspondence problem).

19.1 Preference model

19.2 Image embedding

19.3 Ranking images

Summary

What’s next