liveBook · Manning

Chapter 5: Evolutionary observability

What you will learn in chapter 5 of Effective Platform Engineering

The introduction shows how observability data can help an organization understand performance and value, while illustrating the costs of relying on ad hoc spreadsheets to use that data.

After this chapter you will be able to

What it builds on

Section by section

Introduction

The introduction shows how observability data can help an organization understand performance and value, while illustrating the costs of relying on ad hoc spreadsheets to use that data.

Key terms: observability-driven development, telemetry data, portfolio, shadow IT, spreadsheets, data structure

5.1 Why observability matters

The section defines observability as insight into a system’s state and usefulness, then shows how that insight supports faster responses, shared decisions, and platform improvement.

Key terms: observability, internal state, output data, insights, telemetry, shared data, observability dashboards, shadow IT, telemetry debt, evolution

5.1.1 Observability is more than metrics and alerts

The section explains how metrics, logs, and traces provide complementary evidence that can be connected to diagnose a system problem.

Key terms: metrics, logs, traces, telemetry data, insights, alerts, CPU and memory usage, HTTP errors, disk capacity, discrete events

5.1.2 Use cases for observability beyond basic monitoring of applications

The section broadens observability beyond application monitoring by showing how eight kinds of data can inform engineering decisions, business decisions, and customer experience.

Key terms: application observability, end-user experience, customer experience, infrastructure, application, service health, incidents, portfolio, platform

5.1.3 What does good look like?

The section uses three categories of operating states to explain what observability data to collect and how to make more system behavior understandable over time.

Key terms: healthy, meaningful load, operating states, known-knowns, telemetry, queries, alerts, self-healing, known-unknowns, instrumenting

5.1.4 Viewing observability through a single pane of glass

The section explains how a single pane of glass can bring together observability data from different sources and outlines an architecture for collecting, storing, and querying it.

Key terms: single pane of glass (SPOG), correlation, data sources, ingestion APIs, metrics, logs, traces, polling, time-series database, log management system

5.2 Observability as a platform service

This section presents observability as a platform service and identifies the qualities that make such a platform effective.

Key terms: observability platforms, integration capabilities, data aggregation, user-friendly experience, distributed systems

5.2.1 The end-user access experience

This section explains how an observability platform can give end users easy, unified, responsive access to telemetry and configurable observability tools.

Key terms: identity provider, single sign-on, role-based access control, platform-wide aggregation, metrics, logs, traces, holistic view, mean time to recover, unified and responsive experience

5.2.2 Automatic collection of customer data

This section explains how a platform automatically collects application telemetry and shields application teams from the collection infrastructure.

Key terms: telemetry, metrics, logs, traces, scrape service, platform endpoints, collection agent, onboarding, well-known endpoints, backend technologies

5.2.3 Who needs to respond when things need attention?

This section assigns observability responsibilities across development teams, the platform team, and vendors, then offers an exercise for checking application telemetry.

Key terms: shared responsibility model, development team’s responsibility, platform responsibility, vendor responsibility, demo application, observability stack, instrumentation, Metrics, Logs, and Traces, telemetry data

5.3 Observability platform as a separate internal product

The section traces how observability needs change as an organization grows and argues for treating observability as a self-serve internal product that serves more than developers.

Key terms: start phase, unified view, cross-service problems, dedicated monitoring team, Enterprise-style tooling, DevOps initiative, engineering friction, self-serve, preconfigured architecture patterns, product mindset

5.3.1 Architecture of an observability platform

This section traces a sample observability platform architecture from telemetry collection through buffering, processing, storage, and access.

Key terms: external data collection agent, data ingestion service, metrics, logs, traces, message bus, buffering, fault tolerance, data storage service, Prometheus

5.3.2 Should you build or buy?

This section frames the observability build-or-buy decision as a trade-off in total cost, time, and control that should be tested with real systems rather than settled by assumption.

Key terms: time to value, opportunity costs, labor costs, SaaS observability platform, evolutionary architecture, software-defined delivery, domain design, interoperate, vendor lock-in, evolutionary architectural approach

5.3.3 Cross-platform observability

The section explains why observability for an engineering platform should extend beyond individual clusters and include an independent way to detect outages.

Key terms: per-cluster solution, dashboards, monitoring, alerting, aggregate the observability data, developer control plane, dedicated infrastructure, external monitoring service, basic data collection, internal observability tools

5.3.4 Strategies to drive adoption

This section explains how to encourage adoption of an observability platform and presents a build-versus-buy exercise grounded in requirements, staffing, and capability gaps.

Key terms: education and awareness, quick wins, observability champions, incentivize adoption, automated metrics collection, log aggregation, prebuilt dashboards, CI/CD pipelines, definition of “done”, baseline level of observability

5.4 Observability of published service-level indicators, service-level objectives, and service-level agreements

The section explains how customer-relevant service measures, published objectives, and observable results help a platform set expectations and build trust.

Key terms: uptime, resilience, response time, service-level indicator (SLI), service-level objective (SLO), service-level agreement (SLA), published service-level goals, observability system, customer expectations

5.4.1 SLOs as code

This section shows how to publish software-defined SLO dashboards for a platform and invites readers to design an SLO using Prometheus data.

Key terms: SLO dashboard, software-defined, platform user documentation, target SLO, error budget, burn rate charts, warning and critical alerts, SLO, SLI, Prometheus data

5.5 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 4: Governance, compliance, and trust Chapter 6: Building a software-defined engineering platform →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.