Log aggregation
Log aggregation is the collection and consolidation of log records from multiple sources into a common destination for storage, search, or analysis.
Secure APIs
A log aggregator is a system that receives log records from multiple sources for centralized storage, search, and analysis.
- How it is used
- Centralizes API logs for security investigation, attack detection, storage, and analysis.
- How deep it goes
- Explains the stdout-to-router-to-aggregator pipeline and sidecar deployment, but stops short of multi-request attack detection.
- Examples
- Uses stdout streams, Fluentd or Logplex, a sidecar collector, and an order endpoint that excludes credit-card data.
- What you can do after
- Operate/configure a collection pipeline and reduce sensitive-data exposure in request logs.
Effective Platform Engineering
A log aggregation system collects logs from multiple sources and makes them available for centralized storage and querying.
- How it is used
- Supports centralized debugging, tracing, querying, and operational monitoring across distributed services.
- How deep it goes
- Explains storage routing, Loki deployment, distributed backends, and Grafana integration, but stops short of configuration details.
- Examples
- Uses Loki-backed log storage, Grafana dashboards, automated collection, templates, and observability champions as implementation examples.
- What you can do after
- Operate centralized log collection and querying that helps teams identify and resolve distributed-system problems faster.
Read how Effective Platform Engineering explains log aggregation →
ScyllaDB in Action
Log aggregation is the centralized collection and organization of log entries from multiple systems or nodes.
- How it is used
- Centralizes production logs so operators can query and correlate activity across application instances and ScyllaDB nodes.
- How deep it goes
- Defines the practice and names tool choices, but does not detail collection pipelines, schemas, or configuration.
- Examples
- Uses Grafana Loki and correlates logs from ScyllaDB nodes with application errors and traces.
- What you can do after
- Diagnose distributed-system incidents by searching and correlating logs across nodes and applications.
Related concepts
- Lokiincludes · Loki is a log aggregation system for collecting, storing, and querying log data, commonly indexing labels rather than log content.
- Kubernetescontrasts with · Kubernetes is an open-source platform for orchestrating and automating the deployment, scaling, and management of containerized workloads across computing clusters.
- loguses · A log is a record of events, activity, or diagnostic information produced during the operation of a system or process, often ordered by time.
- log routerused by · A log router is software that receives, processes, and forwards log records between sources and configured destinations.
- auditpart of · An audit is a systematic examination of records, activities, systems, or controls against defined criteria to assess compliance, correctness, or accountability.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.