1 Introduction to Apache Pulsar

 

This chapter covers

  • Introduction to streaming data concepts
  • How Apache Pulsar can handle all of streaming data needs.
  • Compare Apache Pulsar to other messaging systems such as Apache Kafka.
  • Real-world use cases where Pulsar is currently being used for stream processing.

Developed at Yahoo in 2013, Pulsar was first open sourced in 2016, and only 2 years after joining the Apache Software Foundations’ incubation program graduated to Top Level Project status. Apache Pulsar was designed from the ground up to address the gaps in current open source messaging systems such as multi-tenancy, geo-replication, and strong durability guarantees.

The Apache Pulsar site describes it as a distributed pub-sub messaging system that provides very low publish and end-to-end latency, guaranteed message delivery, zero data loss, and a serverless lightweight computing framework for stream native data processing. Apache Pulsar provides the three key capabilities for processing large data sets in real-time:

1.1   Unified Messaging

 
 
 

1.1.1   Publish-Subscribe Messaging

 
 

1.1.2   Message Queuing

 
 

1.2   Stream Native Processing

 
 

1.2.1   Traditional Batching

 
 
 

1.2.2   Micro-Batching

 
 
 
 

1.2.3   Stream Processing

 
 
 
 

1.3   Scalable Storage

 
 

1.4   Comparison to Apache Kafka

 
 
 
 

1.4.1   Partition-Centric Storage in Kafka

 
 
 

1.4.2   Segment-Centric Storage in Pulsar

 
 
 
 

1.5   Why Do I Need Pulsar?

 
 
 
 

1.5.1   Zero Data Loss

 
 

1.5.2   Guaranteed Message Delivery

 
 
 

1.5.3   Infinite Scalability

 
 
 

1.5.4   Resilient to Failure

 

1.5.5   Support for Millions of Topics

 
 
 

1.5.6   Geo-Replication and Active Failover

 
 

1.5.7   Message Deduplication

 
 
 

1.6   Real World Use Cases

 
 

1.6.1   Unified Messaging System

 
 

1.6.2   Microservices

 
 

1.6.3   Connected Car

 
 

1.7   Additional Resources

 

1.8   Summary

 
 
 
sitemap

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage
test yourself with a liveTest