The use of message-oriented systems is becoming increasingly important in modern, distributed IT infrastructure. Where a hard, synchronous coupling via API interfaces would previously have been used you will now find asynchronous interfaces between various systems. The asynchronous connection in particular requires better load balancing and consequently the utilization of the available resources.
In addition, message queuing systems are used, for example, to cross the boundaries between individual security zones.
These days, lightweight and application-independent protocols, such as MQTT or AMQP, are often used. While MQTT can primarily be found in the area of embedded systems and IoT systems, AMQP-based message-oriented middleware is used in a range of areas, especially in the Enterprise environment.
From an organizational perspective, message brokers support the publish-subscribe pattern by replicating this on FIFO storage systems.
RabbitMQ is an Open Source message broker that is being commercially further developed by the company Pivotal Software. Although RabbitMQ is written entirely in “Erlang”, it integrates exceptionally well into Java-based environments. Other Open Source projects such as OpenStack also use RabbitMQ as the internal communication center.
The technological origin from the Erlang environment ensures an extremely high operational stability and availability with the straightforward structure of a cluster. RabbitMQ can deliver extremely high throughput rates, even with complex distribution rules. The message broker is extremely flexible as, besides AMQP in version 0.9.1, it can also broker other protocols, such as MQTT, AMQP 1.0, Stomp, WebStomp, etc. This can be used for translation, i.e. clients that use different protocols to communicate can still be combined in a queue or exchange. Various plug-ins, in some cases provided by the manufacturer, can be used to expand RabbitMQ and tailor it to specific requirements.
User authorizations, TLS connections, and virtual hosts allow a RabbitMQ server or RabbitMQ cluster to be used as a central element of an infrastructure. Various setting options, from a weighting towards speed through to a weighting towards high availability, can be used to adapt RabbitMQ systems to specific requirements. In addition, RabbitMQ offers outstanding capabilities for establishing various cluster structures.
Apache ActiveMQ “Classic”
Apache ActiveMQ, written in Java, fully implements the Java Message Service (JMS) 1.1 specification. ActiveMQ also supports the AMQP protocol in versions 0.9.1 and 1.0, MQTT and STOMP.
Due to its origins in the Java environment and the positioning within the Apache project, it is often used in this area. In the cluster segment, ActiveMQ relies on the support of Apache ZooKeeper. To be able to ensure the desired persistence, ActiveMQ uses KahaDB and databases that can be reached via JDBC.
Apache ActiveMQ Artemis
Apache ActiveMQ Artemis is a new development that displays a convergence to the original Apache ActiveMQ project. The code is based on HornetQ from JBoss. This new implementation is focused on higher performance and the simultaneous adaptation to new paradigms, such as the capability to be used in containers. Artemis supports JMS 1.1 and 2.0, high availability thanks to shared storage and network replication, as well as dynamic clustering.
A FIFO concept is generally not used in the area of stream distribution. Instead, these services normally implement logs in which publishers write and consumers read, and the streaming software discards the entries after a criterion, such as the age or size of the log, is met. The consumer software itself has to determine whether everything has already been read.
Apache Kafka is optimized for high data throughput, and is a typical representative of the streaming fraction. In particular, the consistent distribution of subtasks across individual nodes in the cluster provides a number of options for optimization and performance enhancement. The cluster coordination in a Kafka cluster is taken over by Apache ZooKeeper.
The simplicity of the structure and the deliberate omission of the delivery logic with regard to individual notifications means that this system is able to move and scale extremely large quantities of data in a short space of time. This is complemented by possibilities, such as stream processing, which enables filtering, compaction or convolution, etc., to be performed during a data stream. These capabilities as well as the option, e.g. of docking to a Hadoop, are particularly important in the big data environment.
In the classic messaging segment, Apache Kafka is comparable to systems like RabbitMQ, but shifts a large part of the logic to client systems without implementing protocols such as AMQP. As a result, it is not advisable to use Apache Kafka instead of RabbitMQ or ActiveMQ in this environment.
Do you have any questions on the topic of messaging? Then we are the right partner for you!
Have we piqued your interest?
Our experts will be happy to answer your specific questions and support you in choosing the right open source solution for your project.
Open Source Support Center
Our answers to the most frequently asked questions: