1 Introduction to Apache Pulsar


This chapter covers

  • The evolution of the enterprise messaging system and why Apache Pulsar represents the next step in the evolutionary process.
  • A comparison of Apache Pulsar to existing enterprise messaging systems
  • How Pulsar’s segment-centric storage differs from the partition-centric storage model used in Apache Kafka.
  • Real-world use cases where Pulsar is used for stream processing and why you should consider using Apache Pulsar.

Developed at Yahoo in 2013, Pulsar was first open sourced in 2016, and only 15 months after joining the Apache Software Foundations’ incubation program graduated to Top Level Project status. Apache Pulsar was designed from the ground up to address the gaps in current open-source messaging systems such as multi-tenancy, geo-replication, and strong durability guarantees.

The Apache Pulsar site describes it as a distributed pub-sub messaging system that provides very low publish and end-to-end latency, guaranteed message delivery, zero data loss, and a serverless lightweight computing framework for stream data processing. Apache Pulsar provides the three key capabilities for processing large data sets:

1.1    Enterprise Messaging Systems

1.1.1   Key Capabilities

1.2    Message Consumption Patterns

1.2.1   Publish-Subscribe Messaging

1.2.2   Message Queuing

1.3    The Evolution of Messaging Systems

1.3.1   Generic Messaging System

1.3.2   Message Oriented Middleware

1.3.3   Enterprise Service Bus

1.3.4   Distributed Messaging Systems

1.4    Comparison to Apache Kafka

1.4.1   Multi-Layered Architecture

1.4.2   Message Consumption

1.4.3   Data Durability

1.4.4   Message Acknowledgement

1.4.5   Message Retention

1.5    Why Do I Need Pulsar?

1.5.1   Guaranteed Message Delivery

1.5.2   Infinite Scalability

1.5.3   Resilient to Failure

1.5.4   Support for Millions of Topics

1.5.5   Geo-Replication and Active Failover

1.6    Real World Use Cases

1.6.1   Unified Messaging System

1.6.2   Microservices Platform

1.6.3   Connected Car

1.6.4   Fraud Detection

1.7    Additional Resources

1.8    Summary