front matter
Software is the lifeblood of most industries today and can be a differentiator for those companies that can iterate quickly and find customer value before their competitors. Some of the recent trends that allow large organizations to move fast include the adoption of cloud platforms and microservice architectures. While some of the trends have evolved, one thing has remained constant: when things go wrong, we need to quickly understand where to look to fix the problem. Microservices and ephemeral cloud infrastructure (containers, etc.) exacerbate this problem.
I vividly remember working on a particularly nasty distributed problem for a client a few years back wherein a set of services would communicate with each other to provide some business function, and after six days (almost on the dot!), the set of services would all come crashing down. The resulting outage caused significant revenue loss for this client. The client decided to restart all of the services one by one after four days to avoid the problem.