This chapter covers
- Describing the purpose of a datastore in an orchestration system
- Defining the requirements for our persistent datastore
- Defining the Store interface
- Introducing BoltDB
- Implementing the persistent datastore using the Store interface
- Discussing the special concerns that exist for the manager’s datastore
The fundamental unit of our orchestration system is the task. Up until now, we have been keeping track of this fundamental unit by storing it in Go’s built-in map type. Both our worker and our manager store their respective tasks in a map. This strategy has served us well, but you may have noticed a major problem with it: any time we restart the worker or the manager, they lose all their tasks. The reason they lose their tasks is that Go’s built-in map is an in-memory data structure and is not persisted to disk.
Similar to how we revisited our earlier decisions around scheduling tasks, we’re now going to return to our decision about how we store them. We’re going to talk briefly about the purpose of a datastore in an orchestration system, and then we’re going to start the process of replacing our previous in-memory map datastore with a persistent one.