liveBookmanning.com

What you will learn in Kafka for Architects

Event-driven architecture, logs, microservices, real-time event processing

by Katya Gorshkova
Foreword by Viktor Gamov

Read it on liveBook Get the book at manning.com

Chapter by chapter

Concepts explained in this book

• Apache KafkaAn open-source distributed event-streaming platform for publishing, storing, processing, and subscribing to streams of records.
• batchingBatch processing is the execution of accumulated data-processing work as a group rather than continuously as individual records arrive.
• Order propertyMessage ordering is the preservation or enforcement of a sequence among messages, in Kafka guaranteed only within an individual partition.
• Kafka ConnectKafka Connect is a framework for moving data between Kafka and external systems through configurable connectors rather than custom application code.
• schemaA data schema is a formal definition of the fields, types, and structure of a particular data or message type.
• partitioning keyA message key is a Kafka record value used to determine partition placement and associate records for ordering.
• schema evolutionA schema change is a modification to the structure, fields, types, or format of data exchanged or stored by a system.
• log compactionLog compaction is a Kafka storage process that removes older records for a key while retaining the record with the latest value and preserving record offsets.
• clientA client is an application that communicates with a message broker to send or receive messages.
• consumer groupA shared consumer group is a Kafka consumption mode that permits multiple consumers in a group to receive records from the same partition.
• KRaftKRaft is Kafka's metadata management mode that uses an internal controller quorum instead of ZooKeeper.
• replicationData replication is the process of copying data between systems or storage locations to maintain corresponding datasets.
• event-driven architectureAn event-driven system is a software system whose components react to events rather than relying primarily on direct synchronous interactions.
• configurationA configuration is a set of settings that specifies system behavior, including connection details, selected data, and polling parameters.
• partitionA Kafka partition is an ordered, independently replicated subdivision of a topic that stores records and supports parallel processing.
• backward compatibilityBackward compatibility is a schema compatibility policy in which consumers using the new schema can read messages produced with the previous schema.
• topicA Kafka topic is a named, append-oriented stream of records that Kafka stores and makes available to consumers.
• Kafka StreamsThe Kafka Streams API is the programming interface through which applications define Kafka Streams processing logic.
• producerA producer is an application or component that publishes records to a Kafka topic or other messaging destination.
• statefulnessA stateful system is a system whose behavior or operation depends on data maintained across requests or processing intervals.
• streamingAn event stream is a continuously available sequence of records representing events produced by a system.
• quorumA quorum architecture commits replicated state only after a majority of participating voters acknowledge the update.
• schema registryA schema registry is a service that centrally stores, versions, and validates schemas for structured data exchanged between producers and consumers.
• sink connectorA JDBC sink connector is a Kafka Connect connector that reads records from Kafka and writes them to tables in a target relational database.
• real-time processingReal-time event processing is the handling of events with sufficiently low latency to support immediate or near-immediate reactions.
• exactly-once deliveryExactly-once delivery is a processing guarantee in which each record's effect is applied once despite retries or failures.
• offsetA Kafka offset is a monotonically increasing integer that identifies a record's position within a partition or ordered record stream.
• online learningIncremental learning is a machine-learning technique that updates a model progressively as new data arrives rather than retraining only from scratch.
• eventAn event is a recorded occurrence or state change that can notify other components about something that happened in a system.
• source connectorA source connector is a Kafka Connect connector that reads data from an external source system and writes it to Kafka.
• dead-letter topic patternA dead letter topic is a messaging destination where messages that cannot be successfully processed are redirected for later review and handling.
• data contractA data contract is an agreement specifying the structure and evolution expectations that producers and consumers share for exchanged data.
• monitoringLogging is the recording of structured or textual runtime information about software execution and events.
• heartbeatA Kafka consumer heartbeat is a periodic signal sent to the group coordinator to indicate that the consumer is healthy and active.
• durabilityDurability is the property that acknowledged data survives failures and remains available for later retrieval.
• state storeA local state store is a stateful storage mechanism located on the same node as processing logic for low-latency access to application state.
• connectorA connector is a configured Kafka Connect instance that defines a particular data pipeline between Kafka and an external system.
• FlinkApache Flink is an open-source distributed stream-processing framework for stateful computations over bounded and unbounded data streams.
• metadata logA Raft log is an ordered replicated log used by Kafka controllers in KRaft mode to store changes to cluster metadata.
• dataset splitPartitioning is the distribution of records across separate partitions so data processing and storage can be divided across resources.
• data lossData loss is the permanent or temporary unavailability or destruction of data that a system was expected to preserve.
• acknowledgmentAn acknowledgment is a confirmation that a message or delivery operation has been received or completed.
• retention policyA retention policy is a rule governing how long or under what conditions stored Kafka records remain available.
• self-hostingAn on-premises solution is software or infrastructure installed and operated within the customer’s own facilities.
• HTTP headerA message header is auxiliary metadata attached to a Kafka record independently of its key and value.
• data formatA data format is a defined representation and structure for encoding data exchanged between software components.
• messageA message is a discrete data record transmitted between components through a messaging system.
• nonfunctional requirementA non-functional requirement specifies a quality attribute, constraint, or operational characteristic of a system rather than a business function.
• migrationExtract-transform-load is a data integration process that extracts data from a source, transforms it, and loads it into a target storage system.
• parallelismParallel processing is the simultaneous execution of multiple data-processing tasks to increase processing capacity or reduce elapsed time.
• bounded streamA bounded stream is a finite data sequence with a clearly defined start and end.
• checkpointingCheckpointing is the periodic capture and persistence of streaming application state so it can be restored after failure.
• ExecutionGraphAn ExecutionGraph is Flink's distributed execution representation describing how a job's tasks run across the cluster.
• Flink MLFlink ML is a Flink library for building machine-learning pipelines across batch and streaming data.
• hybrid deploymentA hybrid approach combines on-premises infrastructure with cloud-based services in one solution.
• JobGraphA JobGraph is Flink's execution graph representation that defines task dependencies and execution optimizations for a submitted job.
• keyed stateKeyed state is Flink-managed state associated with specific record keys and accessed or updated during keyed processing.
• stream partitioningStream partitioning is the logical distribution of stream records among parallel processing tasks during execution.
• unbounded streamAn unbounded stream is a continuously arriving sequence of events with no defined end.
• group coordinatorA group coordinator is the Kafka broker responsible for managing a consumer group and coordinating its membership and partition assignments.
• primary nodeA Kafka partition leader is the replica that coordinates writes for that partition and serves as the synchronization reference for follower replicas.
• replication factorA replication factor is the configured number of copies maintained for each Kafka partition.
• validationMessage validation is the checking of a message's structure and content against required validity rules before it is accepted or processed.
• controller quorumA controller quorum is the group of Kafka controllers that coordinates metadata management and reaches consensus in KRaft mode.
• AsyncAPIAsyncAPI is an open specification for defining and documenting asynchronous, event-driven interactions independently of a specific messaging implementation.
• deserializationDeserialization is the process of converting serialized bytes into structured data according to a schema or agreed data format.
• integration testIntegration testing is the practice of verifying that multiple software components communicate and interoperate correctly through their defined contracts and protocols.
• deploymentDeployment is the process of installing and making software available to run in a target environment.
• scalabilityScalability is the ability of a system to maintain or improve its performance as workload or resource demands increase.
• encryptionEncryption is the transformation of readable data into a form accessible only through an appropriate cryptographic key.
• timestampA message timestamp is the time value associated with a Kafka message for determining its retention age.
• AvroAvro is a data serialization system that represents structured records with schemas and a compact binary encoding.
• metadata snapshotA metadata snapshot is a persisted representation of the latest Kafka cluster metadata state reconstructed from the metadata log.
• Kafka client libraryA Kafka client library is a software library that provides APIs and client-side behavior for producing and consuming Kafka records.
• leader electionLeader election is the process of selecting a controller or partition leader to coordinate operations and accept leadership responsibilities.
• controller clusterA KRaft controller cluster is a group of Kafka servers that stores cluster metadata and coordinates broker operations through the Raft consensus protocol.
• consumer group rebalancingConsumer group rebalancing is the redistribution of topic-partition assignments among consumer group members when membership or partition availability changes.
• event catalogAn event catalog is a structured inventory describing the business events available within an integration, including their meanings, participants, ownership, and data requirements.
• partition countA partition count is the configured number of partitions assigned to a Kafka topic.
• Confluent REST ProxyConfluent REST Proxy is a service that exposes Kafka production, consumption, and administration operations through HTTP-based interfaces.
• vendor lock-inVendor lock-in is a condition in which proprietary dependencies, integrations, or switching costs make migration away from a provider difficult or impractical.
• controllerA KRaft controller is a Kafka controller implementation that uses the Raft consensus protocol to manage cluster metadata and coordination.
• functional requirementA functional requirement specifies a capability or behavior that a system must provide.
• serializationSerialization is the process of converting structured data into a transferable representation such as bytes or text.
• streaming applicationA streaming application is an application that processes continuously arriving data streams and can emit transformed or derived streams.
• event sourcingAn event-sourced system persists state changes as an ordered sequence of events from which current state can be reconstructed.
• network topologyA graph topology is a directed processing structure that defines how streaming records traverse processing steps.
• forward compatibilityForward compatibility is a schema compatibility policy in which consumers using an older schema can read messages produced with the new schema.
• data meshA data mesh architecture is a decentralized data architecture in which domain teams own and provide data products under federated governance.
• control planeA service-mesh control plane is the centralized subsystem that configures, deploys, and manages the behavior of sidecar proxies and mesh-wide network policies.
• authorizationAccess control is the enforcement of rules governing which subjects can access particular resources or data attributes.
• append-only systemAn append-only log is a sequential record structure in which new entries are added without modifying prior entries.
• change feedA change feed is an ordered stream of records describing changes made to data in a source system.
• change-data-captureChange data capture is a pattern that observes a source system's change records and emits insertions, updates, and deletions as events.
• GitOpsA GitOps workflow manages infrastructure or application configuration declaratively through version-controlled files and automated reconciliation.
• compressionData compression is the transformation of data into a more compact representation to reduce storage or transmission size.
• ZooKeeperApache ZooKeeper is a distributed coordination service historically used by Kafka to manage cluster metadata and controller coordination.
• mutual TLSMutual TLS is a TLS authentication mechanism in which both communicating parties verify each other's identities with certificates.
• system integrationIntegration is the coordination and interconnection of separate systems so they can exchange data or functionality.
• Kafka protocolThe Kafka binary protocol is the binary request-and-response protocol used by Kafka clients and servers for communication.
• request-response APIA request-response pattern is a communication pattern in which one service sends a request and another service returns a corresponding response.
• securitySecurity is the protection of systems, data, and operations against unauthorized access, misuse, disruption, or compromise.
• subjectA subject is a Schema Registry entity that groups versions of a data schema according to an event type, topic, or both.
• RangeAssignorRangeAssignor is a Kafka consumer partition assignor that orders consumers and assigns each topic's partitions sequentially in contiguous ranges.
• idempotencyIdempotency is the property that repeating an operation produces the same effective result as performing it once.
• ksqlDBksqlDB is a Confluent streaming database system that defines and executes continuous event-processing queries using a SQL-like language over Kafka data.
• min.insync.replicasThe minimum in-sync replicas setting is the configured number of synchronized partition replicas that must acknowledge a write for it to be considered successful.
• RaftRaft is a consensus algorithm that coordinates replicated state among nodes and commits updates after acknowledgment by a quorum.
• appenderAn appender is a logging component that delivers emitted log records to a configured destination.
• bufferingBuffering temporarily stores data between producers and downstream consumers to absorb differences in processing rate or load.
• centralized logging systemA centralized logging system collects log records from multiple applications or services into a common system for storage, search, and analysis.
• ElasticsearchElasticsearch is a distributed search and analytics engine that indexes and queries structured and unstructured data.
• fluentdFluentd is an open-source data collector that gathers, transforms, and routes event and log data.
• Kafka appenderA Kafka appender is a logging component that publishes log records to Kafka.
• KibanaKibana is a visualization and user-interface tool for searching, exploring, and building dashboards over Elasticsearch data.
• resource isolationResource isolation is the separation of compute, storage, or network resources so one workload cannot materially interfere with another.
• acknowledgment strategyAn acknowledgment strategy is a producer-side policy that determines when a Kafka record send is considered successful and what delivery guarantee applies.
• stateless systemA stateless operation is an operation whose output depends only on the current input and not on prior events or retained external state.
• distributed transactionA distributed transaction is a transaction whose operations span multiple independent systems or databases and must satisfy a coordinated all-or-nothing outcome.
• consumer assignment strategyA partition assignment is the allocation of Kafka topic partitions to consumer instances for reading.
• throughputThroughput is the rate at which a system processes or transfers data during a specified period.
• fact eventA fact event is an event that records a state change or occurrence that has already happened rather than requesting an action.
• Kafka clusterA Kafka cluster is a group of broker nodes operating together to provide distributed event storage, processing, and service availability.
• authenticationAuthentication is the process of verifying the identity of a user, service, or client.
• continuous queryA continuous query is a query that remains active and incrementally updates its results as new data arrives.
• KSQL queryA KSQL query is a SQL-based query that continuously processes streams or tables backed by Kafka topics.
• materialized viewA materialized view is a stored, continuously maintained representation of query results that can be read without recomputing the underlying query.
• pull queryA pull query is an on-demand query that retrieves the current state of data and terminates after returning its result.
• delivery guaranteeA delivery guarantee is a defined assurance about whether and how messages are delivered, retained, and processed despite failures or retries.
• consistencyConsistency is the transaction property that preserves defined data rules while moving a system between valid states.
• publish/subscribe modelA publish-subscribe pattern is a messaging pattern in which publishers send messages to subscribers without requiring direct knowledge of the subscribers.
• compacted topicA compacted topic is a Kafka topic configured to retain at least the latest record associated with each non-null key.
• tombstoneA tombstone message is a Kafka record with a non-null key and a null value that marks the keyed record for deletion during compaction.
• source of truthA source of truth is the authoritative system or record from which the canonical value of information is determined.
• data transformationA map operation transforms each input event into a corresponding event in another format or representation.
• access control listAn access control list is a set of permissions specifying which identities may access a resource and what operations they may perform.
• KTableKTable is a Kafka Streams table abstraction that represents the latest value associated with each key and updates as records change.
• schema idA schema identifier is a unique value associated with a registered schema and embedded or referenced in serialized data.
• complexityOperational complexity is the difficulty of provisioning, configuring, securing, monitoring, maintaining, and recovering a system in production.
• resilienceFault tolerance is the ability of a system to continue providing its required behavior despite failures in components or dependencies.
• Apache PulsarApache Pulsar is an open-source distributed messaging and event-streaming platform.
• retryA retry is a configured or performed subsequent attempt to complete an operation after a transient failure.
• under-replicated partitionAn under-replicated partition is a Kafka partition whose available replicas do not currently satisfy its configured replication requirements.
• metadataKafka metadata is information describing cluster or application state, including the distribution of keys among Kafka Streams application instances.
• joinA stream join is a stream-processing operation that combines records from multiple streams or tables when their keys or other join conditions match.
• multiregion configurationA multi-region deployment distributes system components across separate geographic cloud or data-center regions.
• managed serviceA managed service is a provider-operated offering in which the provider administers the underlying software and infrastructure for the customer.
• batch sizeBatch size is a configuration value specifying how many records or units are grouped for collective transmission or processing.
• TLSTLS is a cryptographic protocol that authenticates endpoints and protects data in transit through encryption and integrity checks.
• data sovereigntyData sovereignty is the requirement that data be subject to the laws and jurisdiction of the geographic location where it is stored or processed.
• managed Kafka serviceA managed Kafka service is a hosted Kafka offering in which an external provider operates the Kafka infrastructure.
• partition rebalancingPartition rebalancing is the migration and reassignment of Kafka partitions and their replicas across brokers to redistribute workload or remove capacity.
• data tieringTiered storage is a storage arrangement that places data on different storage tiers according to factors such as age, speed, and cost.
• taskA Kafka Streams task is an independently executable processing unit containing assigned input partitions, state stores, and a graph topology.
• Kafka brokerA Kafka broker is a Kafka server that stores, receives, and serves records for producers and consumers without interpreting their application-level content.
• etl toolAn ETL tool is software for extracting data from sources, transforming it, and loading it into target systems, often through batch or streaming pipelines.
• reference tableA reference table is a relatively stable table of lookup or classification values shared by other data-processing systems.
• snapshot-based replicationSnapshot-based replication copies the current state of a source dataset to another system rather than replaying each individual change.
• testUnit testing is the testing of isolated software units, such as functions or classes, independently from their surrounding components.
• commitIn Kafka, a commit is either confirmation that messages are safely stored or an indication that a consumer has advanced past processed messages, while in databases it finalizes a transaction.
• Kafka transactionA Kafka transaction is an atomic operation that groups record production and related offset handling so their effects can be committed or aborted together.
• state persistenceMessage persistence is the durable storage of messages beyond the lifetime of the process that produced or consumed them.
• uptimeHigh availability is the property of a system remaining operational despite failures of individual components.
• correlation idA correlation ID is an identifier carried with related request and response messages so they can be matched.
• sliding windowA sliding window is a stream-processing time interval defined by the timestamp difference between records being considered for a join or computation.
• deployment pipelineA continuous integration/continuous deployment pipeline is an automated workflow that validates, builds, tests, and delivers software or infrastructure changes.
• delta eventA delta event is an event containing the identifier of a changed entity and only the attributes modified by the change.
• performanceA performance metric measures how effectively and responsively a system performs under a workload.
• poison messageA poison pill message is a malformed or otherwise unprocessable message that repeatedly causes a consumer or processing stage to fail.
• in-sync replicaAn in-sync replica is a Kafka partition replica that has caught up with its leader and is not substantially lagging.
• ZabZooKeeper Atomic Broadcast is the protocol ZooKeeper uses to order and replicate state changes among its servers.
• dual-cluster architectureA dual Kafka cluster is an architecture that operates two Kafka clusters, typically across on-premises and cloud environments, with synchronized data.
• application instanceAn application instance is one running process or deployment instance of a streaming application that executes processing tasks and hosts local state.
• Kubernetes operatorA Kubernetes operator is a controller pattern that automates management of an application or service by reconciling its desired and actual state through the Kubernetes API.
• distributed commit logA distributed commit log is a replicated, append-oriented record of events maintained across multiple nodes.
• transaction logA transaction log is an ordered database record of changes made by committed transactions.
• CQRSCQRS is an architectural pattern that separates models and processing paths for changing state from those for querying state.
• producer idempotenceenable.idempotence is a Kafka producer setting that enables broker-managed deduplication and ordering protection for producer retries.
• RabbitMQRabbitMQ is an open-source message broker that supports asynchronous and synchronous messaging through multiple messaging protocols and patterns.
• microservice architectureMicroservices architecture is an architectural style composed of independently deployable services organized around separate capabilities.
• event correlationEvent correlation is the association of related events or workflows through a shared identifier or contextual relationship.
• event typeAn event type is a reusable specification that bundles predefined characteristics for a class of events or topics.
• offset commit strategyAn offset commit strategy defines when and how a consumer records the positions of processed Kafka records.
• per-partition orderingPer-partition ordering is the sequencing guarantee that Kafka provides for records within an individual partition.
• transformation componentA transformation component is a software component that converts data from one structural or representational format into another.
• ArgumentCaptorArgumentCaptor is a Mockito utility that captures arguments passed to a method so tests can inspect them.
• EmbeddedKafkaBrokerEmbeddedKafkaBroker is a Spring-Kafka test component that starts an in-memory Kafka broker for automated tests.
• TestcontainersTestcontainers is a library for running disposable containerized dependencies during automated tests.
• end-to-end encryptionEnd-to-end encryption is a communication design in which data is encrypted by the producer or sender and decrypted only by the intended consumer or recipient.
• service meshA service mesh is an infrastructure layer that manages service-to-service communication and cross-cutting network concerns independently of service business logic.
• KStreamKStream is the Kafka Streams abstraction representing a continuous stream of records and their transformations.
• SASLSASL is a standardized framework for adding authentication mechanisms to connection-oriented protocols.
• controller failureA controller failure is the condition in which a Kafka controller becomes unavailable or stops participating in controller coordination.
• software upgradeAn upgrade is a change that replaces software or its components with a newer version to improve security, stability, functionality, or compatibility.
• message sizeMessage size is the amount of data contained in a message, typically measured as its serialized size in transit or storage.
• polyglot topicA polyglot topic is a Kafka topic that contains multiple event types with distinct schemas.
• record name strategyRecordNameStrategy is a Schema Registry subject-naming strategy that identifies subjects by fully qualified record name independently of Kafka topics.
• topic name strategyTopicNameStrategy is a Schema Registry subject-naming strategy that groups schema versions by Kafka topic and message component.
• topic record name strategyTopicRecordNameStrategy is a Schema Registry subject-naming strategy that combines topic association with fully qualified record names.
• JDBC source pluginJDBC Source Connector is a Kafka Connect connector implementation that reads records from relational databases through JDBC and publishes them to Kafka.
• mockingA mock is a test double that imitates a dependency and records or verifies interactions with it.
• MockitoMockito is a Java mocking framework for creating and configuring test doubles that substitute for real dependencies.
• TestInputTopicTestInputTopic is a Kafka Streams testing class that provides serialized records as input to a topology under test.
• TestOutputTopicTestOutputTopic is a Kafka Streams testing class that reads and verifies records emitted by a topology under test.
• TopologyTestDriverTopologyTestDriver is a Kafka Streams testing class that executes and verifies a topology without requiring a running Kafka cluster.
• time windowA time window is a bounded interval of event or processing time over which a stream processor groups and evaluates records.
• custom resource definitionA custom resource definition is a Kubernetes API extension that declares a new resource type and its schema.
• infrastructure as codeInfrastructure as code is the practice of defining and managing infrastructure through machine-readable declarative or procedural files.
• driftConfiguration drift is the divergence between a system's actual configuration and its declared desired configuration.
• metadata APIA metadata request is a Kafka client request that retrieves cluster metadata such as partition leadership and broker assignments.
• channelAn AsyncAPI channel is a named address or communication path through which messages are sent or received.
• queueA queue is an ordered messaging destination in which messages wait for delivery to consumers.
• serializerA serializer is a component that converts structured data into a serialized representation such as bytes for transmission or storage.
• administrative APIThe Kafka Admin API is Kafka's programmatic interface for inspecting and modifying administrative resources and cluster configuration.
• ACIDACID is the set of atomicity, consistency, isolation, and durability properties that define conventional transactional behavior.
• data loss toleranceData loss tolerance is the maximum amount of expected or acceptable data loss under failure conditions.
• principalA principal is an identity representing a client, broker, user, or service in a security system.
• gradual migrationGradual migration is the staged transition of functionality, components, or traffic from an older system version to a newer one.
• strangler patternThe strangler fig pattern is an incremental replacement approach in which a new implementation gradually takes over functionality from a legacy system.
• event-carried state transferEvent-carried state transfer is an integration pattern in which events carry the producer's state so consumers can maintain local representations without requesting the state separately.
• round-robin load balancingA round-robin approach assigns successive items cyclically across available destinations to distribute work evenly.
• cloud elasticityElasticity is the ability of a system to dynamically add or remove resources in response to changing workload demand.
• processing nodeA processing node is a topology node that contains stream-processing logic such as filtering, aggregation, or transformation.
• Kafka-specific metricA Kafka-specific metric measures the internal behavior, availability, correctness, or throughput of Kafka components.
• enterprise service busAn enterprise service bus is centralized integration middleware that transports messages and performs intermediary processing such as routing, validation, security, or auditing.
• sidecar processThe sidecar pattern deploys an auxiliary component alongside a primary service component to provide supporting functionality without embedding it in the service.
• smart endpoints, dumb pipesThe smart endpoints, dumb pipes pattern places processing and integration logic in communicating endpoints while transport channels only carry messages.
• timeoutA timeout is a configured interval after which the absence of an expected signal is treated as a failure.
• two-phase commitTwo-phase commit is a distributed transaction protocol in which a coordinator first prepares all participants and then directs them to commit or abort as a group.
• language modelInference is the generation of model predictions from input data using a trained machine-learning model.
• cross-cluster replicationCross-cluster replication is the asynchronous or synchronous copying of Kafka data between separate clusters.
• data governanceData governance is the organizational discipline of defining and enforcing policies for managing data and its use.
• partition imbalancePartition load imbalance is an uneven distribution of records or processing work across partitions.
• candidateCandidate state is the Raft election state in which a controller seeks votes to become the active controller.
• follower controllerA follower controller is a KRaft controller that replicates metadata from the active controller and participates in controller quorum decisions.
• copartitioningCopartitioning is the arrangement of multiple input topics with matching partition counts and partitioning logic so records with corresponding keys are processed together.
• event enrichmentEvent enrichment is the combination of related events to produce a more complete representation of a business object or occurrence.
• GlobalKTableGlobalKTable is a Kafka Streams interface for a globally replicated, immutable, read-only table that is available to every application task.
• selectKeyselectKey is a Kafka Streams operation that assigns a new key to each input record and repartitions the resulting stream when necessary.
• streaming tableA stream-processing table is a view that retains the latest value associated with each key rather than treating every update as an independent event.
• consumer lagLag is the amount by which processing or consumption falls behind the latest available data.
• cold dataCold data is older or infrequently accessed data that can be retained on lower-cost, higher-latency storage.
• hot dataHot data is recently generated or frequently accessed data that benefits from low-latency storage.
• remote log managerA remote log manager is a Kafka component that manages remote log segments, metadata, and transfers between local brokers and remote storage.
• replicaA replica is a copy of a Kafka partition's records maintained on a broker for redundancy and availability.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.