Back to all

How the MQTT client and broker connection works

You encounter MQTT connection more often than you think. 

Developed by IBM in 1999, MQTT, or MQ Telemetry Transport, is the lightweight messaging protocol that enables communication between remote devices, such as smartphones and web apps, sensors, and other systems. 

In addition, one can see examples of successful MQTT implementation in many industries — automotive, logistics, manufacturing, smart home, consumer products, transportation, and many others.

To perform data transfer between devices, MQTT runs over TCP/IP network protocol and uses the publish-subscribe pattern to send out messages. However, there is much more to it than that. Keep reading this article to understand how to establish the MQTT connection, learn more about the CONNECT and CONNACK data packets, and what roles a client and a broker play in this process. 

The MQTT client and its role in the MQTT connection

To put it simply, an MQTT client is a device that operates an MQTT library and establishes a connection to an MQTT broker (if you wish, you can jump to the broker explanation here). 

It is worth mentioning that MQTT libraries are available for most common programming languages, such as C, C++, C#, Go, iOS, Java, JavaScript, and .NET, among others. There is no need to worry about this part. Your client will most likely satisfy this requirement.

Eventually, all devices have roles in the publish-subscribe pattern that MQTT utilizes to perform data transfer. Check out the MQTT Subscribe and Publish article to learn how to publish a message and subscribe to a topic.

In fact, there are two roles that MQTT clients can have. They are:

  • A publisher – an MQTT client that publishes a message on a topic.
  • A subscriber – an MQTT client that subscribes to a topic to receive all messages that are posted to it.

Below you can find some examples of MQTT clients from different industries:

  • Automotive industry: a car door sends a message with the status “open” / “closed .” The receiver is the GUI inside the car.
  • IoT/Smart home: a fridge temperature sensor is the publisher posting the following information: “-7”. The receiver is the operating system of the fridge.

It is worth mentioning that a single MQTT client can have multiple roles. In other words, it can be a publisher and a subscriber simultaneously.

The MQTT broker and its role in the MQTT connection

An MQTT broker is the centerpiece and orchestrator of the MQTT infrastructure, responsible for establishing and maintaining secure connections with clients and routing messages between them.

To perform messaging, an MQTT broker has to complete the following steps:

  1. Receive messages from clients.
  2. Check if these clients have sufficient rights to publish messages on these topics.
  3. Queue the incoming messages for Quality of Service 1 & 2.
  4. Based on access rights, determine the subscribers for the listed topics.
  5. Queue the outgoing messages for Quality of Service 1 & 2.
  6. Dispatch them to subscribers.
The role of MQTT broker in messaging scheme

Effectively, an MQTT broker performs decoupling, ensuring that all messages get delivered to the proper recipients without establishing direct contact between them. In other words, a publishing and a subscribing client never get to talk to each other directly.

Due to the nature of the broker and its core task, its quality determines the reliability of the MQTT connection. Therefore, when choosing an MQTT broker for your project, we recommend you go with Pro Edition for Mosquitto, which offers high availability and is available both in the cloud and on-premises. 

We have zero bureaucracy at our company. We are open and transparent about everything we do, so I encourage you to sign up for a free 14-day trial to test basic MQTT broker scenarios without spending weeks on this task.
Furthermore, it is very reliable and secure (read more on how to enable MQTT TLS), which makes it ideal for commercial use.

How to enable the MQTT client and broker connection

At first, a client must send a CONNECT request to establish an MQTT connection.

MQTT Client’s CONNECT request

The MQTT CONNECT request consists of the three must-have values: clientID, cleanSession, and keepAlive; and the optional ones, such as username, password, lastWillTopic, lastWillQos, lastWillMessage, and lastWillRetain.

clientID 		“client-1”
cleanSession		true
keepAlive		120

Optional information:
username		“tom”
password		“cedalo”
lastWillTopic		“/tom/will”
lastWillQos		2
lastWillMessage		“connection disabled”
lastWillRetain		false

Let’s look at these fields and how they make the MQTT client and broker connection work.

CONNECT request fields

  • clientID

The clientId (for example, “client-1”) identifies each MQTT client connecting to an MQTT broker. If the clientId value is blank, an MQTT broker will still generate a unique parameter for this client to identify it.

Note: The clientId value must be unique. For instance, if a new client connects to a broker and another client session already exists with the same clientId, the old session will be terminated and taken over by a new one. The old client, whose session was terminated, publishes a lastWillMessage (if it is indicated in the CONNECT packet a client sent to the broker to establish the MQTT connection).

Clients that want to maintain persistent MQTT sessions (these sessions keep messages that a broker did not forward during the period while a client was offline) with a broker require the clientId value. Instead, for non-persistent MQTT sessions, a broker does not need to know a clientId unless it requires authentication. In this case, a client sends a blank clientID to a broker, and the latter creates a unique ID while a session is open. 

What is an MQTT persistent session?

An active MQTT persistent session (with the cleanSession having the “false” value) stipulates that all information that a client did not receive from a broker since the last active connection will be preserved on the broker and delivered to a client once the connection is restored. In a persistent session, a client will receive messages from the subscriptions it had during the previous session and will not have to resubscribe to all topics again.

Note: You should not use automatically generated IDs for persistent sessions (to know what the clientId is and be able to reconnect to the previous persistent session if a client loses connection). 
  • cleanSession

This parameter shows whether a client wants to establish a persistent session with a broker or not. 

Therefore, the cleanSession parameter can have two values: “true” or “false”:

“True” means a client doesn’t want to establish a persistent session (a broker will not save unsent messages if the connection is interrupted). In addition, all previous persistent sessions will get dismissed.

“False” means a client wants to establish a persistent session.

  • keepAlive

The MQTT keepAlive parameter identifies the maximum interval in seconds (e.g., 120, as I show in this article’s example) when a client maintains the MQTT connection but does not transmit any data.

After the specified in the keepAlive parameter period is over, a client sends a PING request (PINGREQ), and a broker has to respond with a PING Response (PINGRESP). If no response followed the PING request, the broker must wait for another half of the set in the keepAlive parameter time (150% of the keepAlive parameter time), and then disconnect.

PINGREQ & PINGRESP data packets flow scheme

The client should behave similarly. Since it sends a packet every keepAlive interval at the latest, it also gets an answer once per interval. If there is no answer from the broker, the client knows that there is no active connection. As a result, the client terminates the current connection and establishes a new one.

PINGREQ did not arrive scheme

In other words, for either a client or a broker, it is always sufficient to close the own side of the connection. Whether or not the other side notices and confirms this action is then dealt with within the network stack at the operating system level (with corresponding timeouts).

Read the Paho MQTT Python client article to learn how to set it up and discover the differences of keepAlive parameter for different MQTT versions.

  • username and password

A client can also send their username and password as part of the CONNECT packet to a broker. In other words, if you don’t enable authorization, you don’t need to send username and password values.

The username should be an UTF-8 Encoded String, and an MQTT broker can use it for authentication and authorization purposes.

The password value, in turn, is binary data and can convey any credential information.

It is essential to understand that if the CONNECT request is not secure (through encryption or hashing), you send the password in plain text. In this case the traffic might become available to external parties.

To make your MQTT connection secure, read our ultimate guide on enabling TLS for a Pro Mosquitto MQTT broker.

Note: MQTT v5 allows sending a password without a username. MQTT v3 did not support this functionality.

LastWill Flag: even though you do not see the LastWill flag field as part of the CONNECT packet, it still exists and may have its value set to either “true” or “false.” Thus, if the request contains the lastWillMessage parameter, then the lastWill flag is “true,” and this particular client wants to use the lastWill functionality. If there is no lastWillMessage parameter in the CONNECT packet, then the value is “false”, and the client does not need this functionality.

  • lastWillMessage

The lastWillMessage value is a message (this can be a string, e.g., “hello“ or even a JSON file) a broker sends when the MQTT connection abruptly terminates without sending a DISCONNECT packet. The lastWill notifies connected clients when this happens. To learn more about the lastWillMessage parameter, check out our docs article.

How lastWillMessage works scheme
  • lastWillTopic

The lastWillTopic states the topic (or several of them) to which the lastWillMessage should be published. In other words, all subscribers of this particular topic will get the lastWillMessage once the client goes offline.

  • lastWillQoS

The lastWillQoS parameter provides information about the Quality of Service (QoS) level to use when publishing the lastWillMessge. 

The lastWillQoS value can be either  “0”, “1”, or “2”, where:

  • “0” simply forwards the message once (there is no guarantee that the message was received by the receiver),
  • “1” ensures the delivery of the message at least once (the message can be sent more than once),
  • “2” provides the delivery of the message exactly once (the message can be sent once only).

Check out our documentation to learn more about QoS.

  • lastWillRetain

The lastWillRetain parameter specifies whether the lastWillMessage should be retained or not. If the value is “false,” an MQTT broker publishes the lastWillMessage as a non-retained one. If the value is “true,”  then the lastWillMessage is published as a retained one.

  • A non-retained lastWillMessage means that new subscribers to the lastWillTopic will not get the lastWillMessage from the disconnected client.
  • A retained lastWillMessage means that new subscribers to the lastWillTopic will get the lastWillMessage from the disconnected client. 

After the client has reconnected successfully, the new CONNECT package will overwrite the lastWill command.

MQTT Broker’s CONNACK Response

Once a broker receives the CONNECT request from a client, it validates the packet to make certain that it satisfies the format requirements. The broker can also run additional tests to verify that this MQTT connection can be established (including the necessary authentication and authorization checks).

How MQTT connection works overview

Then, based on the findings, the MQTT broker has to respond to a client with the CONNACK packet, sending over the following information:

sessionPresent		true
returnCode		0
  • sessionPresent

This field lets a client know whether there has been a previous persistent session that it could re-connect to or it should start a new one. 

If the cleanSession’s value in a client’s CONNECT message is “true”, then the sessionPresent value in the CONNACK response will be “false.” It means that a client and broker should establish a new session and, at the same time, discard any existing one.

Suppose the cleanSession’s value in a client’s CONNECT message is “false” and a previous session was running for this device with this particular clientID. In that case, the MQTT broker resumes the session based on the existing one.

In case, if the cleanSession’s value in a client’s CONNECT message is “false” and there is no session running for this device with this particular clientID, then the MQTT broker establishes a new session from scratch.
And an even more interesting case is when a client does not have any cleanSession value in a client’s CONNECT message and, in response, gets a sessionPresent value in the CONNACK packet with the “true”, then it has to disconnect. However, if the clients still wants establishing an MQTT connection, it should send another CONNECT message with the cleanSession value set to “true”.

  • returnCode

Afterward, an MQTT broker sends a 0 returnCode to confirm that the MQTT client and broker connection is successful.

If the MQTT connection is not successful (the returnCode is not 0), the broker thereafter sends back the CONNACK response with sessionPresent value set to “false”.

Note: There are different return codes that a broker sends to a client for MQTT v.3.1.1 and v.5.0.

Below you may find some of the returnCode values that an MQTT broker can send back to a client for MQTT v.5.0:

ValueHexReason Code nameDescription
00x00SuccessThe connection is accepted.
1280x80Unspecified errorThe MQTT broker does not want to disclose the cause for the error, or none of the other reason codes apply.
1290x81Malformed PacketThe information in the CONNECT packet could not be correctly read.
1300x82Protocol ErrorThe information in the CONNECT packet does not observe the requirements of this specification.
1310x83Implementation specific errorThe CONNECT packet is valid but is not accepted by this MQTT broker.
1320x84Unsupported Protocol VersionThe MQTT broker does not support the MQTT protocol’s version that the client requested.
1330x85Client Identifier not validThe clientID is valid but it is not allowed by the MQTT broker.
returnCode list for MQTT v.5.0

Please check out this document to see more returnCode values for MQTT v.5.0.

In addition, below you may find some of the returnCode values that an MQTT broker can send back to a client for MQTT v.3.1.1

ValueReturn Code ResponseDescription
00x00 Connection AcceptedConnection accepted
10x01 Connection Refused, unacceptable protocol versionThe MQTT broker does not support the MQTT protocol’s version that the client requested.
20x02 Connection Refused, identifier rejectedThe clientID is valid UTF-8 but it is not allowed by the MQTT broker.
30x03 Connection Refused, Server unavailableThe MQTT connection has been established. However, the MQTT service is not available.
40x04 Connection Refused, bad user name or passwordThe information in the username or password has errors.
50x05 Connection Refused, not authorizedThe client does not have access rights to connect to the broker (is not authorized to do so).
returnCode list for MQTT v.3.1.1

In other words, if an MQTT broker transmits any returnCode value except for 0, it must close the connection straight away.

Takeaway points

To sum up, I hope this article has helped you understand how to establish the MQTT client and broker connection and answered most of your questions. 

Stay tuned, and subscribe to our newsletter. In the meantime, our team is already working on the next article to dive deeper into MQTT technology and help you flawlessly set up your IoT project. 

All in all, we recommend using the Pro Edition for Eclipse Mosquitto MQTT broker for your project. Try it out by signing up for a 14-day free trial here.

Newsletters icon

Subscribe for monthly updates