Chapter 4. Foundational RabbitMQ

Chapter 1 from RabbitMQ in Depth by Gavin M. Roy.

This chapter covers

Whether your application is in the cloud or in your own data center, RabbitMQ is a lightweight and extremely powerful tool for creating distributed software architectures that range from the very simple to the incredibly complex. In this chapter you’ll learn how RabbitMQ, as messaging-oriented middleware, allows tremendous flexibility in how you approach and solve problems. You’ll learn how some companies are using it and about key features that make RabbitMQ one of the most popular message brokers today.

4.1. RabbitMQ’s features and benefits

Open source— Originally developed in a partnership between LShift, LTD, and Cohesive FT as RabbitMQ Technologies, RabbitMQ is now owned by Pivotal Software Inc. and is released under the Mozilla Public License. As an open-source project written in Erlang, RabbitMQ enjoys freedom and flexibility, while leveraging the strength of Pivotal standing behind it as a product. Developers and engineers in the RabbitMQ community are able to contribute enhancements and add-ons, and Pivotal is able to offer commercial support and a stable home for ongoing product maturation.

We’ll explore the features on this list in later chapters, but I’d like to focus right now on the two most foundational features of RabbitMQ: the language it’s programmed in (Erlang), and the model it’s based on (the Advanced Message Queuing model), a specification that defines much of the RabbitMQ lexicon and its behavior.

RabbitMQ has many features and benefits, the most important of which are

4.2. Who’s using RabbitMQ, and how?

Reddit, the popular online community, uses RabbitMQ heavily in the core of their application platform, which serves billions of web pages per month. When a user registers on the site, submits a news post, or votes on a link, a message is published into RabbitMQ for asynchronous processing by consumer applications.

MercadoLibre, the largest e-commerce ecosystem in Latin America, uses RabbitMQ at the heart of their Enterprise Service Bus (ESB) architecture, decoupling their data from tightly coupled applications, allowing for flexible integrations with various components in their application architecture.

4.3. The advantages of loosely coupled architectures

After decoupling the login process from the database updates that were required, I discovered a new level of freedom. Members were able to quickly log in because we were no longer updating the database as part of the authentication process. Instead, a member login message was published containing all of the information needed to update any database, and consumer applications were written that updated each database table independently (figure 1.3). This login message didn’t contain authentication information for the member, but instead, only the information needed to maintain the member’s last login status in our various databases and applications. This allowed us to horizontally scale database writes with more control. By controlling the number of consumer applications writing to a specific database server, we were able to throttle database writes for servers that had started to strain under the load created by new site growth while we worked through their own unique scaling issues.

As I detail the advantages of a messaging-based architecture, it’s important to note that these advantages could also impact the performance of systems like the login architecture described. Any number of problems may impact publisher performance, from networking issues to RabbitMQ throttling message publishers. When such events happen, your application will see degraded performance. In addition to horizontally scaling consumers, it’s wise to plan for horizontal scaling of message brokers to allow for better message throughput and publisher performance.

Summary