Log aggregation
Log aggregation is the collection and consolidation of log records from multiple sources into a common destination for storage, search, or analysis.
Microservices Patterns, Second Edition
Log aggregation is the centralized collection and organization of log records from multiple application components or instances.
- How it is used
- Centralizes service-activity logs to support searching and alerting when diagnosing problems across microservices.
- How deep it goes
- Defines centralized collection, search, and alerting, while deferring detailed pattern treatment.
- Examples
- Uses a request passing through multiple services to show why one instance’s log file is insufficient.
- What you can do after
- Diagnose microservice problems by searching centrally gathered service logs and using alerts.
Read how Microservices Patterns, Second Edition explains log aggregation →
Spring Microservices in Action
A logging pattern that gathers logs from multiple microservices and their instances into a single queryable database.
- How it is used
- Centralizes logs from microservices and their instances to support searching and distributed debugging.
- How deep it goes
- Explains centralized streaming and search, compares platform options, and gives a Papertrail integration example.
- Examples
- Uses Papertrail with Docker and Logspout, and lists ELK, Graylog, Splunk, and Sumo Logic as alternatives.
- What you can do after
- Operate a searchable log store and use correlation IDs to find entries from a distributed transaction.
Read how Spring Microservices in Action explains log aggregation →
Secure APIs
A log aggregator is a system that receives log records from multiple sources for centralized storage, search, and analysis.
- How it is used
- Centralizes API logs for security investigation, attack detection, storage, and analysis.
- How deep it goes
- Explains the stdout-to-router-to-aggregator pipeline and sidecar deployment, but stops short of multi-request attack detection.
- Examples
- Uses stdout streams, Fluentd or Logplex, a sidecar collector, and an order endpoint that excludes credit-card data.
- What you can do after
- Operate/configure a collection pipeline and reduce sensitive-data exposure in request logs.
Effective Platform Engineering
A log aggregation system collects logs from multiple sources and makes them available for centralized storage and querying.
- How it is used
- Supports centralized debugging, tracing, querying, and operational monitoring across distributed services.
- How deep it goes
- Explains storage routing, Loki deployment, distributed backends, and Grafana integration, but stops short of configuration details.
- Examples
- Uses Loki-backed log storage, Grafana dashboards, automated collection, templates, and observability champions as implementation examples.
- What you can do after
- Operate centralized log collection and querying that helps teams identify and resolve distributed-system problems faster.
Read how Effective Platform Engineering explains log aggregation →
ScyllaDB in Action
Log aggregation is the centralized collection and organization of log entries from multiple systems or nodes.
- How it is used
- Centralizes production logs so operators can query and correlate activity across application instances and ScyllaDB nodes.
- How deep it goes
- Defines the practice and names tool choices, but does not detail collection pipelines, schemas, or configuration.
- Examples
- Uses Grafana Loki and correlates logs from ScyllaDB nodes with application errors and traces.
- What you can do after
- Diagnose distributed-system incidents by searching and correlating logs across nodes and applications.
Also covered in
- Logs and Telemetry
- Kafka for Architects
- Think Like a CTO
- Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and Terraform
What to know first
- microservice architectureMicroservice architecture is a software architectural style that structures an application as independently deployable services, typically organized around focused capabilities and communicating through defined interfaces.
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
- monolithic architectureMonolithic architecture is a software architectural style in which an application's major functionality is organized and deployed as a single unit.
- metricA metric is a quantitative measure that assigns values to a property or outcome so it can be assessed, compared, or monitored.
Related concepts
- Papertrailimplemented by · Papertrail is a cloud-based log management service that aggregates and makes application logs searchable.
- Lokiincludes · Loki is a log aggregation system for collecting, storing, and querying log data, commonly indexing labels rather than log content.
- correlation iduses · A correlation ID is an identifier associated with related requests, operations, messages, or records so their activity can be linked across system components.
- Kubernetescontrasts with · Kubernetes is an open-source platform for orchestrating containerized workloads and automating their deployment, scaling, and management.
- loguses · A log is a time-ordered record of events, messages, or activity generated by a system or process.
- auditpart of · An audit is a systematic examination of records, activities, systems, or controls against defined criteria to assess compliance, correctness, or accountability.
Questions about log aggregation
What is log aggregation?
Log aggregation is the collection and consolidation of log records from multiple sources into a common destination for storage, search, or analysis.
Which Manning books explain log aggregation?
Microservices Patterns, Second Edition, Spring Microservices in Action, Secure APIs, Effective Platform Engineering and ScyllaDB in Action.
What should I know before learning log aggregation?
Manning books that teach log aggregation usually introduce microservice architecture, mean, monolithic architecture and metric first.
What is the difference between log aggregation and Kubernetes?
Log aggregation is the collection and consolidation of log records from multiple sources into a common destination for storage, search, or analysis. Kubernetes is an open-source platform for orchestrating containerized workloads and automating their deployment, scaling, and management.
What kinds of log aggregation are there?
Kinds of log aggregation include Loki.
What is log aggregation part of?
Log aggregation is part of audit. An audit is a systematic examination of records, activities, systems, or controls against defined criteria to assess compliance, correctness, or accountability.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




