Rabbit MQ
Rabbit MQ
Rabbit MQ is one of the most popular open source message brokers which can run on many operating
systems and cloud environments. RabbitMQ is lightweight and easy to deploy on premises and in the
cloud. It supports multiple messaging protocols. RabbitMQ can be deployed in distributed and federated
configurations to meet high-scale, high-availability requirements.
After login to MQ, we can see the Nets traffic in Overview screen.
Virtual Hosts:
In RabbitMQ virtual hosts are logical groups of entities, they are similar to virtual hosts in Apache or
server blocks in Nginx. Virtual hosts are created using rabbitmqctl or HTTP API and they provide logical
grouping and separation of resources. Every virtual host has a name. When an AMQP 0-9-1 client
connects to RabbitMQ, it specifies a vhost name to connect to. If authentication succeeds and the
username provided was granted permissions to the vhost, connection is established.
Admin interface in the Rabbit MQ will enable to create Users, Virtual Hosts, Policies, Limits and Clusters.
Connections:
In order for a client to interact with Rabbit MQ it must open a connection. Connections should be
closed. Number of currently open client connections and connection opening/closure rates are
important metrics of the system that should be monitored. Connection leak will occur when an
application repeatedly opens connection without closing them or at least closing only some of them.
When we launch consumers, we can see some connections in connections tab.
Below are the available connections after launching consumers. In this, virtual host is connected to
collectingInterface.
Virtual Host: It refers from which it is connected. In the above scenario it is connected from virtual host
collectingInterface.
Name: This is actual IP and Port of the machine.
Username: This refers to connected with which user.
State: This refers to the status of consumer.
Protocol: This is the protocol which MQ library uses.
Channels: This refers to the number of channels. In Rabbit MQ, each connection is divided into channels.
But usually 1 channel is allocated for each consumer. This can be tuned and tested.
Network (From Client/To Client): This refers to the Network speed.
Channels:
A channel only exists in the context of a connection and never on its own. When a connection is closed,
so are all channels on it.
For applications that use multiple threads/processes for processing, it is very common to open a new
channel per thread/process and not share channels between them.
Prefetch value is the important in channels, which will limit each consumer created under the channel. If
we tune this prefetch value to higher than 100, it will take maximum of 100 consumed messages each
time. It is a kind of chunking. If we don’t limit this value, it takes all the messages it can, but it causes
some issues and it can stuck. Because if server is badly configured, it might crash during the number of
messages coming in. This can be limited between 100 to 1000. In development environments it is set to
100. It varies in production. This value is configurable and can be found under cpspCommon (MQ.php).
Each channel consumes a relatively small amount of memory on the client. Depending on client library's
implementation detail it can also use a dedicated thread pool or similar where consumer operations are
dispatched.
Each channel also consumes a relatively small amount of memory on the node the client is connected
to. Since a node usually serves multiple channel connections, the effects of excessive channel usage or
channel leaks will primarily be reflected in RabbitMQ nodes' metrics and not those of clients.
Given both of these factors, limiting the number of channels used per connection is highly
recommended.
Exchanges:
Each consumer is bound to exchange. AMQP is default exchanger for server. Below are the some of the
exchanges used in settlementEngine.
Credit: For publishing refund, we use credit exchange.
Debit: For payments we use debit exchange.
Settlement: For settlement publishing we use settlement exchange.
Settlement.dead: Dead queue for errors. Whenever we want to publish errors about settlement, we
publish this. The dead exchanges that we actually communicate to Nets. For example, we might publish
settlement.create.request.error.v1 message to this settlement dead exchange, the operation people
from Nets will see this message and debug. If this message is valid then they will fix.
Exchange Types: Exchanges controls the routing of messages to queues. There are 4 types of exchanges.
Each exchange type defines a specific routing algorithm which the server uses to determine which
bound queues a published message should be routed to.
1. Direct Exchange: It publish messages directly to consumer. This type routes messages with a
routing key declared by the binding queue. The Direct exchange type is useful when you would
like to distinguish messages published to the same exchange using a simple string identifier.
2. Fanout Exchange: Fanout is publishing messages to whole queue that has multiple consumers.
Fanout is powerful way of publishing messages. The Fanout exchange type is useful for
facilitating the publish-subscribe pattern. When using the fanout exchange type, different
queues can be declared to handle messages in different ways. For instance, a message
indicating a customer order has been placed might be received by one queue whose consumers
fulfill the order, another whose consumers update a read-only history of orders, and yet another
whose consumers record the order for reporting purposes.
3. Topic Exchanges: We can define certain topics. If we refer above screenshot, credit, debit,
settlement etc., some of the defined topics. The Topic exchange type routes messages to
queues whose routing key matches all, or a portion of a routing key. Queues binding to a topic
exchange supply a matching pattern for the server to use when routing the message. This
exchange type is useful for directing messages based on multiple or for routing messages
originating from multiple sources.
When a message arrives with topic credit, we can check queues. Queue names are generally set
with applicationName.topic. We can put own binding from MQ interface. Refer below
screenshot for bindings:
4. Header Exchanges: The Headers exchange type routes messages based upon a matching of
message headers to the expected headers specified by the binding queue. The headers
exchange type is similar to the topic exchange type in that more than one criteria can be
specified as a filter, but the headers exchange differs in that its criteria is expressed in the
message headers as opposed to the routing key, may occur in any order, and may be specified
as matching any or all of the specified headers.
Publishing Messages:
Queues:
Queues in RabbitMQ are ordered collections of messages. Messages are enqueued and dequeued
(consumed) in the FIFO manner, although priority queues, sharded queues and other features may
affect this.
Queues have names so that applications can reference them.Applications may pick queue names or ask
the broker to generate a name for them. Queue names may be up to 255 bytes of UTF-8 characters.
Queue names starting with "amq." are reserved for internal use by the broker. Attempts to declare a
queue with a name that violates this rule will result in a channel-level exception with reply code 403
(ACCESS_REFUSED).
Queues have properties that define how they behave. There is a set of mandatory properties and a map
of optional ones:
Name
Durable (the queue will survive a broker restart)
Exclusive (used by only one connection and the queue will be deleted when that connection
closes)
Auto-delete (queue that has had at least one consumer is deleted when last consumer
unsubscribes)
Arguments (optional; used by plugins and broker-specific features such as message TTL, queue
length limit, etc.)
To purge single queue message, select & open one message and purge those. To purge all queues, we
can execute below command. Script is located at https://git.paytrail.com/snippets/7.