Instrumentation adds software mechanisms that emit telemetry about a system’s behavior and state. In observability, it determines which signals—logs, metrics, and traces—code exposes for collection and analysis. section 2.0 section 4.0
Choose instrumentation according to the signal, task, and amount of control required:
- Use structured logs to record events for human-oriented investigation; JSON formatting and consistent fields improve usefulness. section 2.3.1 section 2.7
- Use metrics to expose numerical measurements for dashboards and alerting; Prometheus libraries provide a common implementation path. section 2.4.1 section 2.7
- Use traces to show request movement across microservices, including duration and success state; OpenTelemetry emits the spans. section 2.5.1 section 2.7
- Start with auto-instrumentation when supported, then add manual instrumentation for business logic or signals requiring finer control. section 4.3.1 section 4.6
Instrumenting an echo service
A small Go HTTP echo service can log each request, emit a counter labeled by HTTP status, and start an OpenTelemetry span around request handling. The service can then route its signals through agents to destinations such as Grafana, Prometheus, or Jaeger. This end-to-end pattern makes instrumentation an operational workflow rather than isolated logging statements. section 2.1 section 2.3.1 section 2.4.1 section 2.5.1
Instrumentation has ongoing costs: developer effort, application CPU and memory overhead, retained telemetry, and network usage. Manual instrumentation offers control but can reduce code readability and maintainability; auto-instrumentation reduces effort but still affects performance and may not capture business-level behavior. section 1.5.1 section 2.4.3
B2I = LOC_AFTER_INSTRUMENTATION/LOC_BEFORE_INSTRUMENTATION
Use the business-logic-to-instrumentation ratio as a rough regression guide for code growth, not as a measure of instrumentation quality. section 2.2
