For any machine learning task, you should start with a simple but functional system and build out more sophisticated components as you go. This guideline applies to most technology: ship the minimum viable product (MVP) and then iterate on that product. The feedback you get from what you ship first will tell you which pieces are the most important to build out next.
This chapter is dedicated to building your first human-in-the-loop machine learning MVP. We will build on this system as this book progresses, allowing you to learn about the different components that are needed to build more sophisticated data annotation interfaces, active learning algorithms, and evaluation strategies.
Sometimes, a simple system is enough. Suppose that you work at a media company, and your job is to tag news articles according to their topic. You already have topics such as sports, politics, and entertainment. Natural disasters have been in the news lately, and your boss has asked you to annotate the relevant past news articles as disaster-related to allow better search for this new tag. You don’t have months to build out an optimal system; you want to get an MVP out as quickly as possible.