chapter two

2 Signal Types

 

This chapter covers

  • What different signal types are
  • When and how to use a certain signal type
  • The costs and benefits of each signal type
  • How to figure out what signals are relevant
  • Determine signal baselines

In the context of observability, signals play a central role: this is the intel we’re basing our decisions on.

As discussed in 1.1, the System under Observation (SuO)–for example, a Kubernetes application or a serverless app based on Lambda functions–emits signals from various sources. You as a human, or equally a piece of software, then uses these signals to make decisions. There is a one-to-many relation between SuO and sources. Your SuO typically involves compute, storage, and network and each of these represents a signal source. We will cover the sources in detail in Chapter 3.

In terms of signals (and sources) we can differentiate between two cases:

  1. Sources, such as code that you own and you can instrument yourself (that is, decide where and what kind of signals to emit).

  2. Sources you don’t own, meaning any signals will be pre-defined. Your task is limited to selecting which signals to consume, in that case. This might be due to:

    • Code that is a dependency of your code (library, package, module).
    • Due to the fact that the component’s source code is not available (for example, a binary file).
    • The component only offers a (networked) API. Any managed service from your cloud provider of choice falls into this category.

2.1 Reference Example

2.2 Assessing Instrumentation Costs

2.3 Logs

2.3.1 Instrumentation

2.3.2 Telemetry

2.3.3 Costs and Benefits

2.3.4 Observability with Logs

2.4 Metrics

2.4.1 Instrumentation

2.4.2 Telemetry

2.4.3 Costs and Benefits

2.4.4 Observability with Metrics

2.5 Traces

2.5.1 Distributed Traces