Elasticsearch

As explained in Kafka for Architects

Elasticsearch is a distributed search and analytics engine that indexes and queries structured and unstructured data.

Elasticsearch is a distributed search and analytics engine for indexing and querying structured or unstructured data. In centralized logging, applications emit standardized log events to Kafka, which transfers them to Elasticsearch for indexing and analysis. section 5.1.3

Elasticsearch handles large log volumes, indexes new data within seconds, and supports complex searches. Users access indexed events through its REST API or Kibana, which provides SQL-like queries and real-time dashboards. OpenSearch and OpenSearch Dashboards offer analogous alternatives. section 5.1.3

Figure 5.6 Conceptual flow for log data collection: log data is sent from the application to Kafka for processing, then indexed in Elasticsearch, and finally visualized in Kibana.

Follow the conceptual flow from application log events through Kafka and Elasticsearch to Kibana visualization. section 5.1.3

How Kafka for Architects covers it

How deep it goes

Defines the high-level logging flow and user interfaces but stops short of implementation details.

Examples

Uses multi-service application logs, Kafka-to-Elasticsearch transfer, REST queries, and Kibana dashboards.

What it assumes

Assumes applications generate events and teaches how Kafka-mediated collection supports centralized log analysis.

What you can do after

Operate log analysis through Elasticsearch’s REST API or Kibana dashboards after events are indexed.

How it connects

Connects log events with Kafka, Kibana, REST APIs, OpenSearch, and centralized logging.

liveBookmanning.com

Read Kafka for Architects on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Elasticsearch in Action, Second EditionElasticsearch is a distributed search and analytics engine for indexing, querying, and analyzing structured and unstructured data.
• 

  liveBookmanning.com
  Learn Docker in a Month of Lunches, Second EditionElasticsearch is a distributed search and analytics engine for storing, searching, and analyzing structured and unstructured data.
• 

  liveBookmanning.com
  Effective Conversational AIElasticsearch is a distributed search and analytics engine that indexes and retrieves structured and unstructured data.
• 

  liveBookmanning.com
  MongoDB 8.0 in Action, Third EditionElasticsearch is a distributed search and analytics engine for indexing, searching, and analyzing structured and unstructured data.

Compare how 5 books explain Elasticsearch →

Related concepts

• Kibanaused by · Kibana is an open-source web interface for searching, analyzing, visualizing, monitoring, and dashboarding data stored in Elasticsearch.
• ELK stackpart of · The ELK stack is a collection of Elasticsearch, Logstash, and Kibana components for ingesting, processing, searching, analyzing, and visualizing data.
• Luceneuses · Apache Lucene is an open-source Java library for indexing, searching, and retrieving textual and other content.
• document databasekind of · A document database is a nonrelational database that stores and retrieves records as self-contained documents, often with flexible or nested structures.
• fluentdused by · Fluentd is an open-source data collector that gathers, processes, transforms, and routes logs and other machine-generated event data through pluggable integrations.
• search enginekind of · A search engine is software that retrieves and presents information relevant to queries, typically by matching them against indexed content and ranking results.

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.