MQTT Version 5 Shared Subscriptions
MQTT message transmissions typically occur between a publisher, a message broker, and a subscriber, and there are two parts: from publisher to broker and from broker to subscriber. For each topic, the broker sends the message to all subscribers to that topic. In this article, we will talk about MQTT shared subscriptions, how they work, and when they are most suitable for your next IoT project.
Introduction
The broker sends a message to each subscribed client in regular MQTT transmissions. If three clients subscribe to a particular topic, the broker sends the message three times.
This works well when each subscriber requires its own copy of a message to display the value of a sensor. Suppose your topic is a data stream that needs processing, and each message should only be processed once. In that case, this arrangement is no longer suitable because all subscribers receive the same messages, and so would carry out duplicate processing.
For the case where the processing involved is lightweight, it may be fine to use a single subscriber to get around this problem. In that instance where processing each message requires significant time, this no longer works, and multiple subscribers are required.
So how do you mitigate this issue? This is where MQTT shared subscriptions come in.
What are MQTT shared subscriptions?
Simply put, MQTT shared subscriptions allow client load balancing, which means that the broker splits the message load equally amongst the subscribed clients for a particular topic.
So if you have two clients sharing an MQTT subscription, a message will only ever reach one of the clients in each transmission.
How do MQTT shared subscriptions work?
In a shared subscription, MQTT brokers alternate message transmissions to each client in the shared subscription, usually in a round-robin manner.
Like regular MQTT transmissions, shared subscriptions also have topic filters and subscription options. However, the topic filters must follow this syntax: $share/<GroupID>/<Topic>.
An MQTT shared subscription consists of three parts:
$share: A prefixed name telling the MQTT server this is a shared subscription.GroupID: A string, without any wildcards (‘/’, ‘+’, and ‘#’), that identifies a subscription group. All clients with the same GroupID and Topic are part of the same shared subscription.Topic: A string – can include wildcards (‘+’ and ‘#’) – that denotes the topic filter to be used. This is equivalent to the topic filter in a non-shared subscription.
Let’s use an example to illustrate how shared subscriptions work. We have a broker that must send many messages to its subscribers daily. At its peak, the broker might send 2000 messages daily to the clients subscribing to topic123. To distribute the load across their clients, we set up two shared subscriptions: GroupA and GroupB. The shared subscription for GroupA looks like this: $share/GroupA/topic123.
With shared subscriptions, messages are shared within a sub-group. If there are multiple groups, the messages will be duplicated to those groups, but they will still be shared within them.
For instance, suppose clients P and Q are subscribed to $share/GroupA/topic123, while clients X and Y are subscribed to $share/GroupB/topic123. In this example, the first message received would go to both P and X, the second would go to Q and Y, and so on.
Shared subscription MQTT use cases
MQTT shared subscriptions can be used in many scenarios, but they are especially useful in projects that need high scalability. From our experience, shared subscriptions work particularly well in these scenarios:
- When you have MQTT clients within your project that are unable to deal with the message load from subscribed topics.
- When your project handles popular topics that have a high message rate, which in turn could lead to a scalability bottleneck.
- When your backend applications ingest MQTT streams, and you need to scale your systems easily.
Recommendations on using MQTT shared subscriptions
Using MQTT shared subscriptions with Quality of Service (QoS) levels
For simplicity, it is recommended that all clients within a given shared subscription group use the same QoS level. This means that all messages will be treated the same across all clients. In addition, it will be easier to debug should any issues arise.
MQTT shared subscription examples for Mosquitto or Pro Mosquitto
Follow these steps to set up an MQTT shared subscription with access control lists (ACL) on Mosquitto or Pro Mosquitto:
- Grant subscribe permissions to the shared subscription by adding a subscribePattern to an ACL for a role in the Management Center. Example: Adding the
$share/#subscribePattern to the ACL enables users of that role to access the shared topics with that pattern. - Grant permissions to your publishers for the specified topic. Add a publishSend type ACL to the publisher role and specify
<your-topic>.
It is recommended to use a stricter ACL. You can change the subscribePattern to include a specific topic or even a specific topic within a specific group.
$share/+/<your-topic>, where <your-topic> is the specific topic you want to publish to: Allows users to access the specified topic from all shared groups.$share/<shared-group>/<your-topic>: Allows users to access a specific topic within a specific shared group.
Wrap up
Shared subscriptions are best used when your project requires high scalability, as they allow for client load balancing in MQTT message deliveries. In a shared subscription, MQTT brokers alternate message deliveries among the clients within a subscription group. A shared subscription is defined by following this syntax: $share/<GroupID>/<Topic>.
Therefore, to avoid additional complexity, ensure that clients within a shared subscription group should have the same QoS level. To set up MQTT shared subscriptions on Mosquitto, you will need to grant permissions to your subscribers and publishers by adding the subscribePattern and publishSend, respectively, to your ACLs on Mosquitto. You can have a stricter ACL by adjusting the subscribePattern to specify a particular topic within a particular group.
To start using MQTT shared subscriptions in your projects, check out the Cedalo MQTT Platform’s Pro Edition for Eclipse Mosquitto™ broker. You can try out the pro features by signing up for a Cedalo MQTT Platform trial. However, if you face any issues while using the open-source version, do reach out to us via our new professional support service.
About the author
Roger has a background in Electronic Engineering academic research focusing on the design of scientific optical imaging sensors, accompanying control and acquisition systems, and other embedded designs.
He has 6 years of Academic teaching responsibility, teaching semiconductor chip design, embedded design, and programming.
Roger started Mosquitto as a hobby project in 2009, which gained rapid popularity and contributed to the widespread adoption of MQTT as an IoT protocol. He also developed an MQTT client library in Python, and both projects became part of the Eclipse Foundation in 2014/15. In 2018, Roger joined Cedalo to develop Mosquitto further, starting with implementing MQTT v5.0 support.
Subscribe for monthly updates
