MQTT basics

MQTT Protocol Guide: Everything You Need to Know

Tizian Prokosch04.09.202316.12.2023

Picture this: you’re away from home, but with just a few taps on your smartphone, you can illuminate your living space and adjust the temperature to create a comfortable atmosphere for when you return. What may seem like magic is made possible by MQTT – a messaging protocol used in many IoT applications. From smartwatches to home security systems and connected cars, MQTT enables seamless communication among these devices, making our lives more convenient and interconnected. Using MQTT in IoT enables real-time and asynchronous device communication, while conserving power and bandwidth.

This article explores the basics of the MQTT protocol – what it is, how it works, its main components, and more. Whether you’re new to MQTT or looking more more insights into its advantages, this article covers all your needs! Let’s start by gaining a better understanding of what MQTT is.

What is MQTT?

MQTT is a messaging protocol that enables communication between devices. Its design is specific for use in environments with limited bandwidth and high latency. MQTT operates on a publish-subscribe model which allows devices to send messages on specific topics, while other devices can subscribe to those topics to receive the messages. 

For example, a temperature sensor can publish messages on the topic, “temperature degrees.” A thermostat can subscribe to the “temperature degrees” topic, allowing it to receive messages on that topic and adjust the temperature settings. The publish-subscribe model creates a one-to-many communication pattern that allows a single publisher to send messages to multiple subscribers effortlessly. 

What is the history behind the MQTT protocol?

In 1999, MQTT was created through the collaborative efforts of Andy Stanford Clark and Arlen Nipper from IBM. Their goal was to develop a messaging protocol that would be lightweight and suitable for devices like sensors and actuators. The creators specifically tailored MQTT to excel in networks that were not always reliable, such as satellite links commonly used in the oil and gas industry at that time. The protocol’s engineering prioritized high efficiency, allowing it to transmit large data volumes even on slower networks.

IBM initially named MQTT as “MQ Telemetry Transport“ when they used it internally. They derived the name from IBM’s MQSeries, where “MQ” represented “Message Queueing.” After its public release, the protocol’s usage quickly expanded beyond telemetry. It was later referred to as MQTT – an acronym that no longer stood for anything. Since then, all subsequent versions have consistently referred to the protocol simply as “MQTT.”

Furthermore, the “Message Queue” component was never fully utilized in the protocol naming. Since the MQTT protocol relies on a publish and subscribe messaging pattern, message queuing in a traditional sense is not observed.

In 2013, MQTT received standardization from the OASIS organization and OASIS took over the maintenance of the protocol from IBM. In 2014 OASIS released the MQTT version 3.1.1, and in 2019 – MQTT version 5.

Today, MQTT is widely recognized as a leading messaging protocol for the IoT infrastructure. Its versatility has made it a popular choice across applications, including smart home automation, industrial automation, asset tracking, environmental monitoring, medical devices, etc.

What are the benefits of the MQTT protocol?

The MQTT protocol in IoT has become extremely popular and widely adopted due to the following benefits:

• Lightweight: The design of MQTT messages are small (the smallest message can be as small as just two bytes), making them easy to transmit over networks. The protocol is binary-based which optimizes the size of the messages and, therefore, consume minimal bandwidth during its transmission. This makes MQTT ideal for applications that often operate within limited network capacity.
• Reliability: MQTT allows to adjust the level of reliability for message delivery to the intended recipients. Therefore, MQTT supports different MQTT Quality of Service (QoS) levels and allows user to specify the importance of message delivery by choosing from QoS levels 0, 1 and 2. QoS 0 does not ensure a message was reliably delivered, whereas 2 ensures that an intended subscriber receives a message exactly once. For other cases, when a network stability comes under question, one can also utilize persistent MQTT sessions. An active MQTT persistent session ensures that even if a client has already established a connection with a broker and subscribed to a specific topic goes offline, the broker will preserve the information the client did not receive since the last active session. This information will be sent to the subscriber when it comes back online.
• Secure Communication: MQTT is very flexible and allows developers to implement extra security measures as part of an MQTT broker functionality like username and password authentication, client certificate authentication, and encryption using Transport Layer Security (TLS), also colloquially referred to as Secure Sockets Layer (SSL). You can also utilize access control lists and other measures to protect your MQTT setup from unauthorized access and ensure its high level of security.
• Scalability: MQTT protocol supports both horizontal and vertical scalability approaches, enabling smooth growth and scaling of your IoT project.
• Simple Integration: The MQTT protocol’s simplicity and availability of various MQTT libraries for different programming languages and platforms make it easy for new IoT developers to integrate it into systems.
• Strong Industry Support: MQTT has gained widespread support throughout the industry, with MQTT brokers and client libraries forming a robust ecosystem of tools and resources. Many providers also feature convenient and ready-to-use cloud or on-premises MQTT solutions. They offer rich features and additional tools that make MQTT even more accessible and easy to work with.

To test our Pro Mosquitto MQTT broker in the cloud or on-premises, sign up for a free trial here!

What are the common use cases for using MQTT?

While smart homes represent a common use case for MQTT, its effectiveness extends well beyond home automation. MQTT is widely applicable in various IoT and machine-to-machine (M2M) scenarios. Several examples of these applications include:

• Industrial Monitoring and Control: Within industrial settings, MQTT plays a crucial role in the real-time monitoring and control of machinery, sensors, and equipment. It facilitates communication between devices on the factory floor enabling data collection, process optimization, and predictive maintenance for improved productivity.
• Telemetry and Remote Sensing: MQTT is suitable for applications that center around sensing and telemetry such as monitoring, weather stations, and agricultural systems. It enables the collection and transmission of data from sensors located in remote areas to servers or cloud platforms. This facilitates decision-making processes driven by data.
• Asset Tracking and Fleet Management: MQTT is vital in asset tracking systems and fleet management solutions by providing real-time location updates for vehicles, containers, and valuable assets. It ensures tracking capabilities alongside route optimization, which enhances the overall efficiency of transportation operations.
• Energy Management and Smart Grids: MQTT plays a significant role in energy management by facilitating communication between smart meters, energy monitoring systems, and utilities. It enables real-time data exchange, which helps manage load balancers and implement demand response mechanisms. These capabilities can help optimize energy consumption, contributing to the development of smart grids.
• Healthcare: In the healthcare industry, MQTT guarantees reliable communication for a range of applications, including patient monitoring, telemedicine, and health tracking devices. It enables the transmission of vital signs, patient data, and alerts between devices and healthcare providers. This exchange elevates patient care and contributes to improved health outcomes.
• Smart Cities: MQTTs capabilities also extend to city applications such as street lighting, waste management systems, traffic monitoring solutions, and environmental monitoring technologies. MQTT contributes to city infrastructure development while optimizing resource usage by enabling data exchange and control in these areas. Ultimately, the goal is to enhance the quality of life for urban residents.

What is the underlying principle of MQTT?

The MQTT publish-subscribe or pub-sub model is a messaging pattern involving clients (devices that establish a connection with the broker) that send or receive messages on topics. The clients do not need to be aware of the other clients sending or receiving messages. The MQTT communication involves at least three parties:

• Publishers: Devices or applications that send messages containing data to specific topics without knowledge of the recipients (subscribers).
• Subscribers: Devices or applications that express interest in receiving data published to specific topics by “subscribing” to these topics. 
• MQTT Broker: The centerpiece that receives messages from publishers and delivers them to the subscribers based on their respective subscriptions. It employs a process known as decoupling (which separates senders from the receivers), to ensure message distribution without requiring direct interaction between publishers and subscribers. For a more detailed description of the MQTT broker’s role, refer to our MQTT connection beginners guide.

The Pub-Sub model in action

Let’s consider a household furnished with devices such as temperature sensors, light controllers and smart switches. These devices require communication to enable automation and coordination. MQTT facilitates this through the publish-subscribe model:

• Temperature Sensor (Publisher): This sensor gauges the room temperature and publishes the data to the MQTT broker using the example topic home/living_room/temperature.
• Light Controller (Subscriber): The light controller subscribes to the topic home/living_room/temperature, allowing it to adjust the lighting in response to changes in room temperature.
• Smart Switch (Subscriber): The smart switch also subscribes to the topic in order to control the air conditioning based on the room temperature.
• The role of the MQTT broker is to transmit temperature data to both the controller and smart switch.

Whenever the sensor detects a change in temperature, it sends out the updated value to the broker. The network then forwards this information to both the controller and smart switch, which triggers the following actions:

• The light controller adjusts either the intensity or color of the lighting based on the room’s temperature.
• The smart switch turns the air conditioning system on or off to maintain desired temperature levels.

The devices engage in real-time communication to achieve the desired outcome.

What are the main MQTT elements?

In the section above, we already touched on what MQTT clients and an MQTT broker are. In this segment, we will dive deeper into each of the involved parties, introduce new MQTT system elements such as an MQTT packet, MQTT messages, and the MQTT broker web interface and elaborate on their roles and interactions with each other.

1. An MQTT client can, for example, be a device (sensors, smartphones, web apps, and other devices) that establishes a connection to an MQTT broker.

Previously we defined two roles that MQTT clients can have:

• A publisher – an MQTT client that publishes a message on a topic.
• A subscriber – an MQTT client that subscribes to a topic to receive all messages posted to it.

A single MQTT client can simultaneously be a publisher and a subscriber.

To establish the connection, clients use an MQTT library, a software component that handles the MQTT protocol implementation. MQTT libraries provide an interface between clients and the MQTT broker, allowing the clients to communicate with the broker.

2. MQTT broker: At the core of the MQTT system is the MQTT broker, which serves as a central message hub. It receives messages from publishers and routes them to subscribers based on their topic subscriptions. When planning on launching an MQTT project, it is imperative that you select a quality MQTT broker. Therefore, we recommend exploring the benefits of the Pro Edition of Mosquitto. It offers high availability and is accessible both in the cloud and on-premises.

To experience the capabilities of the Pro Mosquitto MQTT broker, sign up for a free trial and test basic MQTT broker scenarios. 

3. MQTT Packets: MQTT relies on packets to enable communication between the client and the broker. Here are just a few examples of packets:

• CONNECT: An MQTT client uses this packet to establish a connection with the broker.
• PUBLISH: The client employs this packet to send a message to a topic.
• SUBSCRIBE: Clients use this packet to join one or more topics.
• UNSUBSCRIBE: Clients can utilize this packet to leave one or more topics.
• DISCONNECT: The client can terminate the connection with the broker using this packet.

These examples do not encompass the full spectrum of MQTT packets. Refer to the blog post on MQTT packets for a list of the different packets involved in MQTT communication.

4. MQTT Messages: Note that the MQTT PUBLISH packet includes a payload (sometimes called a message) that contains custom data sent to a broker by a publisher. The broker then relays it to interested subscribers. Such messages are versatile and can contain various types of data — from sensor readings and commands to vital status updates, making them central to MQTT’s communication model.

5. A web user interface: Pro Mosquito offers Management Center (MMC), which is a graphical interface that connects to a broker as an MQTT client and helps manage your MQTT broker instances. With this, you have complete control over your MQTT broker setup.

How does the MQTT protocol function?

MQTT Connection

To start the communication, an MQTT client must first establish a connection with the broker by sending a CONNECT request.

This packet must include three values: 

• clientID 
• cleanSession
• and keepAlive

But can include optional details such as username, password, lastWillTopic, lastWillQos, lastWillMessage, and lastWillRetain. The example below shows the temperature sensor that wants to connect to an MQTT broker. Here is what the CONNECT request looks like:

Client ID: "client-1"
Clean Session: true
Keep Alive: 120 


Optional Information:
Username: "user-1"
Password: "cedalo"
Last Will Topic: "/sensor/status"
Last Will QoS: 2
Last Will Message: "Disconnected"
Last Will Retain: false

Once the sensor establishes a connection, it gains the ability to share or receive messages. Read this article to learn more about how to establish MQTT connections.

MQTT Topic

An MQTT broker uses topics as filters to deliver MQTT messages from publishers to subscribers. MQTT topics are case-sensitive simple strings, which are subject to a hierarchical structure and can consist of one or more levels separated by forward slashes (/). 

For instance:

• home
• home/temperature/
• home/temperature/living_room
• home/temperature/kitchen
• home/temperature/bedroom

In some cases, you may need to publish or subscribe to several topics that share a defined pattern. This is where MQTT wildcards come in.

MQTT wildcards are special characters that allow you to achieve this goal. There are two wildcards one can use: + and #.

The single-level wildcard (+) corresponds to a level in the topic hierarchy.

• So home/+/living_room would match both home/temperature/living_room and home/humidity/living_room MQTT topics and allow publishing or subscribing to both of them.

Conversely, the multi-level wildcard (#) accounts for any number of levels in the topic hierarchy. Note: place the # wildcard at the end of the topic.

• For example location/# would match location/city/new_york, location/city/paris, and location/country/usa.

Check out this guide to grasp the essence of MQTT topics and Wildcards and how they work.

MQTT Subscribe

To stay updated on specific topics, an MQTT client needs to subscribe to them. After establishing the connection between a subscriber and an MQTT broker, the former sends a SUBSCRIBE packet to the broker specifying a unique packetID set by the client library, the topic it wants to receive messages from, and a QoS level to subscribe to. The broker then keeps track of subscribers for each topic.

Expanding on our example, let’s say a thermostat device subscribes to the  home/temperature/living_room topic. Here, we illustrate what our SUBSCRIBE packet will look like:

Packet ID: 0014 Topic Subscription: "home/temperature/living_room"
QoS: 1 

This ensures that whenever the temperature sensor updates its values, the broker relays that message to the thermostat.

MQTT Publish

When a device intends to share information with others, it “publishes” a message onto a designated topic. The Publish packet includes the packetID, topicName, the desired Quality of Service (QoS) level for message delivery, the retainFlag (if set to “true”, then this message will be retained, read more about MQTT retained messages), Payload (a message), dupFlag (if set to “true”, then a duplicated message was resent), etc.

Now, whenever the temperature sensor sends a PUBLISH packet, it can look like the following:

PacketID: 0015
topicName: "home/temperature/living_room"
qos: 1
retainFlag: false
Payload: The current temperature is 22.5 degrees Celsius.
dupFlag: false

The broker’s responsibility now is to guarantee that all subscribers who are interested in the home/temperature/living_room topic receive this message and can access the temperature information. 

Check out the MQTT Subscribe and Publish article for a more detailed explanation of publishing a message and subscribing to a topic.

What are the standard MQTT ports, and why?

• Port 1883: Default port for non-secure MQTT communication.
• Port 8883: Default port for secure MQTT communication using TLS/SSL encryption.

The choice of port 1883 was based on historical and conventional factors during its development in the late 1990s. At the time, port 1883 was was vacant, which meant that the MQTT protocol could utilize this port without conflicting with other well-known services. As MQTT gained popularity, port 1883 became the default in many MQTT implementations. It simplified the process for developers to connect devices and applications using this standardized port.

For secure communication using TLS/SSL, MQTT utilizes port 8883. This is often referred to as MQTT over TLS (MQTT/SSL). The reasoning behind port 8883 is likely due to its memorability and similar conventions as port 1883. Port 8883 was also vacant and, same as 1883, fell into the range of high ports (above 1023) available for custom services. 

While it is possible to customize MQTT port numbers if needed, such modifications require explicit configuration of both the MQTT client and broker. Sticking to the standard port numbers is recommended to ensure smooth interoperability and hassle-free communication across various MQTT implementations.

What are the MQTT versions?

The two primary versions of the MQTT protocol are MQTT v3.1.1 and v5.0. In 2013, IBM introduced MQTT v3.1 to the OASIS specification body, stipulating the acceptance of only minor modifications to the specification. OASIS later took on the maintenance responsibility and released version 3.1.1 in 2014. Subsequently, a more advanced iteration emerged in 2019 with the release of MQTT version 5.. 

The table below summarizes the comparison between MQTT versions 3.1.1 and 5:

The Pro Edition for Mosquitto MQTT broker fully supports all the features and specifications defined in both MQTT v3.1.1 and MQTT v5 protocols. If you are starting a new IoT application, consider opting for MQTT v5, it offers enhanced security and expanded capabilities. However, if you are currently using MQTT v3.1 for an existing IoT application, it is still effective, and there is no immediate need to switch to MQTT v5.

What does MQTT over WebSockets entail

MQTT over WebSockets is an adaptation of the MQTT protocol to work over WebSocket connections. WebSockets provide a full-duplex (two-way) communication channel over a single, long-lived connection and are fully compatible with web browsers and web-centric environments. 

However, browsers cannot use MQTT connections directly. For such cases, some brokers like Mosquitto allow you to open a WebSocket listener, which can accept MQTT packets when a connection is established. In this way, it is possible to communicate with the broker and its clients directly from the browser. This feature is typically referred to as MQTT over WebSockets, and you can read more about it in the WebSockets for MQTT guide.

Is MQTT considered a secure protocol?

MQTT is designed for efficient and reliable messaging but does not inherently provide comprehensive security features. However, it can leverage existing security mechanisms, such as Transport Layer Security (TLS) encryption. 

Enabling TLS encryption secures the communication between MQTT clients and brokers so that the data exchanged remains confidential, authenticated, and protected from eavesdropping or tampering.

TLS over MQTT (or sometimes called MQTTS) communications typically use port 8883 and the “mqtts://” protocol prefix. For more details on configuring and enabling TLS on the Mosquitto broker, refer to this article.

Alternative security measures such as client certificate authentication, Access Control Lists (ACLs), and dynamic security are possible. However, you must implement them on the broker since they fall outside the scope of the standard MQTT protocol.

Is MQTT a reliable protocol?

Many regard MQTT as a reliable protocol primarily due to its Quality of Service (QoS) options. MQTT offers three QoS levels, each with reliability and message delivery guarantees. These levels allow MQTT to accommodate application needs and ensure the delivery of intended messages.

Three QoS levels in MQTT

QoS 0 (At most once)This QoS level ensures that a message is sent at most once without any confirmation. The publisher sends the message to the broker once and does not wait for any acknowledgment. Therefore, QoS 0 is suitable for applications where occasional data loss is acceptable, such as routine status updates that do not influence the result of the process.

QoS 1 (At least once): This QoS level ensures that a message is delivered at least once. To start off, the publisher (data sender) sends a message to the broker (data receiver), which then forwards it to the subscribers (data receiver). Once the broker receives the message, it sends an acknowledgement (PUBACK packet) back to the sender. If the acknowledgment gets lost, the publisher (data sender) retransmits the message until it receives acknowledgment from the MQTT Broker (data receiver). The same process happens whenever the broker retransmits the message to the QoS1 subscribers on the topic. In this way, QoS 1 is beneficial for applications where receiving duplicate messages is acceptable.

QoS 2 (Exactly once): This level ensures that messages are delivered exactly once. The data sender (either a publisher or MQTT broker) sends the PUBLISH packet and waits for an acknowledgment. The data receiver (either a MQTT broker or subscriber) answers the sender with a PUBREC acknowledgment packet to confirm that the message was received. After that, the sender sends a publish release PUBREL packet to the receiver to confirm the receipt of the PUBREC packet. Once the receiver gets the PUBREL packet, it answers with another confirmation – the publish complete PUBCOMP packet – and then starts processing the original data packet.

As you can see, this level introduces a four-way handshake between a sender and a receiver. The recipient keeps the message in a separate queue on its side and does not process it further until the sender responds with PUBREL to indicate that it got the receiver’s acknowledgment (PUBREC).

This ensures that both the receiver and sender know that the receiver has successfully received the message and is ready to process it. The packet will only be processed after this mutual understanding between the sender and receiver is established through PUBREC and PUBREL. 

The core principle of QoS2 is to ensure that both sides of the communication are on the same page regarding the status of the packet before processing it. In contrast, QoS1 processes the packet only for the receiver right when it establishes this knowledge. If the receiver’s acknowledgement is then lost in transit, the sender would have no way to ensure data delivery and would have to resend the packet, leading to duplicates. QoS 2 guarantees message delivery to subscribers exactly once. This level of service is particularly suitable for applications that cannot afford any loss or duplication of messages.

Selecting the appropriate QoS level will depend on the application’s requirements. Our MQTT QoS blog explains the different levels and packet types involved in the communication.

Is MQTT compatible with RESTful principles?

MQTT is not directly compatible with RESTful principles. They both represent two distinct communication paradigms, each with its unique characteristics and applications. 

REST (Representational State Transfer) is not a protocol but rather an architectural style that governs the design of networked applications. It defines some core principles, such as statelessness, request-response communication model, uniform interface, resources, etc., which are typically not found in MQTT. 

RESTful systems typically communicate via HTTP using HTTP methods such as GET, POST, PUT, and DELETE to perform operations on resources identified by URLs. The difference between RESTful services and those that use MQTT lies in the nature of their communication. RESTful communication is stateless with a request-response model, while MQTT is stateful and relies on the publish-subscribe (pub-sub) communication model.

MQTT and RESTful principles serve different purposes and employ distinct communication styles. However, specific middleware or integration solutions can bridge the gap between their interfaces. For instance, Pro Mosquitto utilizes both MQTT API and REST API to enable efficient communication and management of MQTT-based systems. The Mosquitto Management Center (MMC) provides a REST API for accessing the Mosquitto broker.

Closing thoughts

Now that you’ve grasped MQTT protocol basics, including what it is, how it functions, and its components, it’s time to consider your MQTT project requirements. Sign up for a free trial of the Pro Edition for Eclipse Mosquitto MQTT broker to test its capabilities firsthand.

Additionally, should you continue to use the open-source version of Mosquitto and encounter any roadblocks, our professional support service is available to provide expert guidance and troubleshooting assistance.

About the author

Tizian Prokosch

Solutions Engineer and Scrum Master

Tizian is a dedicated team member of Cedalo and has embraced the opportunity to wear many hats, driven by an unwavering passion for technology. His journey has been dynamic, allowing him to craft a multifaceted expertise.

His foundation in IT, developed during his studies, has been further refined through hands-on experience in various roles.

As a Quality Manager, Tizian ensures the functionality and usability of Cedalo's products. As a Solutions Engineer, he eagerly dives into technical inquiries, guiding their clients toward success. As a Scrum Master, he leads their agile team, fostering collaboration and effective communication.