New Release: RabbitMQ 3.2.0 with Federated Queues

October 31, 2013 Al Sargent

RabbitMQ3.2.0_HeaderWe are pleased to announce the release of RabbitMQ 3.2.0!

There are a number of enhancements and bug fixes for the server as well as many plugins, including management, LDAP, federation, Web-STOMP, AMQP 1.0, STOMP, MQTT, Java, .NET, and more. These are all noted in the release notes and summarized further below. The most significant new enhancement is for federated queues, but there are also significant advances with policies, authentication failure notification, connection blocks, and consumer priority.

About Federated Queues

There is a problem when messaging systems hit maximum capacity and throughput. Federated queues help solve the problem by increasing capacity.

While the Federation Plugin has existed for some time, federated queues allow you to have the same “logical” queue distributed over many brokers, even distributed across a WAN, and allows for higher capacity than a single queue. In work-queuing scenarios, messages are moved to where they can be consumed. A federated queue links to another queue, called an upstream queue, via AMQP. These queues do not have to be on the same broker or in the same cluster, nor do the upstream queues need to be reconfigured. Here are a few scenarios where federated queues apply:

  1. Generally, work is queued and scaled by connecting more consumers to an existing broker or queue. Companies often start by making sure each broker has more than enough consuming capacity, meaning you do waste some capacity. However, there are still times when this doesn’t solve the problem because the broker is operating at maximum capacity, and the broker itself is the bottleneck, not the consumers. By federating queues, other brokers and their consumers can help out. If there is enough capacity globally, you don’t have to have the original partition perfect in advance. The federated queue will grab unconsumed, “spare” messages from upstream. For example, you might have producers and consumers in multiple locations, and you want NYC to “help out” London or vice versa when one is busier than the other.
  2. While it might be more temporary, you could also add a new broker, federate a queue within the new broker, and add more consumers to the new queue until a more fully load balanced solution can be put in place. The federated queue will help with capacity in this case, but the original queue will still take the majority of the load.
  3. Federated queues can also expand capacity within a single broker. A single queue can only use a single core, and this puts a limit on how many messages/second a queue can handle. When you need more capacity than the core allows, you ultimately have to spread the load across multiple queues. With federated queues, you can basically add queues to a single broker by federating multiple queues within the broker. In doing so, the resulting logical queue gets spread across cores and increases capacity.
  4. Federated queues also become useful when a WAN network link goes down. Let’s say you have producers in multiple locations and the broker is located with all the consumers in one place. If a network link goes down, you can’t really publish to the broker. With queue federation, you could have a local broker in each location.

For more information on federated queues, read the following blog post or documentation. There are also important considerations to be aware of in the upgrade process.

cta-download-rabbitmq

RabbitMQ 3.2.0 Server Enhancements

Here is a list of key enhancements in this release:

  • Support for federated queues
  • Allow alternate and dead-letter exchanges, queue max length, expiry and message TTL to be controlled by policy as well as AMQP arguments
  • Report client authentication errors during connection establishment explicitly using basic.close
  • Inform clients when memory or disk alarms are set or cleared
  • Allow policies to target queues or exchanges or both
  • Make it harder to trigger the disk space alarm with default settings
  • Offer greater control over threshold at which messages are paged to disk
  • Allow missing exchanges & queues to be deleted and unbound without generating an AMQP error
  • Implement consumer priorities [4]
  • Backport OTP process supervision infrastructure improvements
  • Relax type constraints of header exchanges
  • Add support for specifying a SSL verify_fun name in the config file

For more information on RabbitMQ

About the Author

Biography

More Content by Al Sargent
Previous
How Bank Branches are Going to Evolve
How Bank Branches are Going to Evolve

The advent of mobile banking granted consumers the ability to pay bills, deposit cheques and, in the near f...

Next
Pivotal CEO Pays Toronto a Visit
Pivotal CEO Pays Toronto a Visit

Yesterday, we had the pleasure of hosting Pivotal CEO Paul Maritz in Toronto. After spending the last coupl...

Enter curious. Exit smarter.

Register Now