Dev guides

How to Use the Paho MQTT Client in Python with Examples

Andreas Schiffler28.11.202328.11.2023

Like the Eclipse Mosquitto MQTT broker, Eclipse Paho is an open-source project. It provides MQTT clients with various programming languages and is the de facto standard for MQTT client functionality.

It is also worth mentioning that the creators of the project named it “Paho,” meaning “communicate with everyone,” to reference its multi-language capability (ref. Eclipse wiki).

In this article, I will demonstrate how to use the Paho Python client with the Pro Edition for Eclipse Mosquitto MQTT broker. You do not require any additional prerequisites – you can simply test it in your internet browser.

Here’s a video version of this tutorial.

If you prefer a written tutorial, continue reading and find an example code snippet at the end of the tutorial.

Set up the Paho MQTT Python client

Since I want to use the Python implementation of the Paho MQTT client, a Python execution environment needs to be in place. There are different approaches to setting this up.

You can start a ready-to-use environment by clicking this link to follow the approach I want to share. Doing so will launch the Jupyter Notebook in your internet browser complete with a Python MQTT example code. I will guide you through each step of the process in the following sections.

The use of this environment is straightforward. Click “play,” as shown below, and the selected Python code lines (marked with the blue vertical bar on the left) will run in your browser window.

Another nice feature of the Jupyter Python environment is that when pressing ‘ctrl+I’ for Windows or ‘command+I’ for Mac (or right-clicking ‘Show Contextual Help’), you can get information about the specific function call depending on your cursor location.

If you decide to prepare or use your Python environment, install the Paho MQTT client using the Python package tool. Check out this reference for more information.

pip install paho-mqtt

As the picture above shows, the imported and used SSL package ( import ssl ) is always part of the Python default environment installation.

Get your ready-to-use and secure MQTT broker

Of course, you will also need an MQTT broker for this Paho MQTT Python setup. Sign up for a 14-day free trial and verify your email to receive an email with detailed instructions, as shown here. Then, you will get a free version of a professional and secure Eclipse Mosquitto MQTT broker with an intuitive WebUI setup in minutes.

Create the Python MQTT client object

After following the steps in the previous sections, you now have a Python environment and an MQTT broker set up. In addition, at least one client should have been created with the following credentials, as shown here:

Client id: “myPy”

Username: “user”

Password: “password”

Now, let’s create a client object named client in Python.

import paho.mqtt.client as mqtt
import ssl

version = '5' # or '3' 
mytransport = 'websockets' # or 'tcp'

if version == '5':
    client = mqtt.Client(client_id="myPy",
                         transport=mytransport,
                         protocol=mqtt.MQTTv5)
if version == '3':
    client = mqtt.Client(client_id="myPy",
                         transport=mytransport,
                         protocol=mqtt.MQTTv311,
                         clean_session=True)

The next two sections will provide an additional explanation for the code snippet above.

MQTT protocol version and the introduction of expiry intervals

MQTT communication is a standardized protocol that currently has two versions: 3.1.1 and 5.0. During the Python MQTT configuration process, we can set the MQTT version to use by specifying the protocol parameter and choosing either the MQTTv311 or MQTTv5 value from the Paho library (as shown in the code above). 

It is worth mentioning that version 5.0 was introduced in 2018 and released in March 2019, so it is yet to be part of the ISO standard.

However, the MQTT protocol underwent several improvements from version 3.1.1 to 5.0. One of them is how the protocol handles session expiry. This aspect can be advantageous for TLS encryption because it saves traffic overhead by reusing the session data when deploying MQTT version 5.0. Refer to the MQTT TLS article for more details.

Suppose  the default clean_session flag is set to true (see the code snippet above) in version 3.1.1. In that case, the data of every session (including outstanding messages, subscriptions, and TLS handshakes) is cleared if the keepalive period (the default value is 60 seconds) is up. This happens when there is no communication between a client and a broker. Check out this blog post to learn more about the CONNECT and CONNACK parameters required for establishing the MQTT connection.

With the MQTT version 5.0, the clean_session flag is split into two flags:

• A clean start flag, indicating that the session should start without using existing session data
• And a SessionExpiryInterval flag that defines how long to retain the session after a disconnect (see later in “Configure callbacks and connect a client to your broker” section).

Setting the clean_start flag to 1 and SessionExpiryInterval to 0 is equivalent to setting clean_session to 1 in MQTT v3.1.1.

The MessageExpiryInterval, which is explained in the “Publish messages with the Paho MQTT client in Python” section, enables the use of an expiry interval for the so-called Last Will messages published with the retain flag set to 1 (“last will”). After the MessageExpiryInterval, the lastWillMessage will be dismissed. You can learn more about the lastWill parameter implementation for the Mosquitto broker here. Moreover, the Pro Edition of Eclipse Mosquitto fully supports the MessageExpiryInterval, which has been available since version 5.0. It allows the implementation of application-specific settings and boosts functionality.

Examples of “last will” usage

If you would like examples of “last will” usage, I will happily give you several.

Imagine that your IoT infrastructure includes thousands of connected IoT clients like cars, trucks, or other assets with a high probability of going offline for minutes or even weeks. In such cases, messages (e.g., about traffic jams) must have a short expiry interval. After a client reconnects, it may take up to several weeks (after going offline) to deliver other notices or updates.

SessionExpiryInterval is a helpful setting to choose between reducing traffic overhead and increasing information security. Imagine an IoT device with a small footprint in compute performance, energy consumption, and traffic volume. This device may have an unstable or interrupted network connection.

With a very short SessionExpiryInterval on every reconnection, you may need to execute the TLS handshake and certificate exchange. However, this would be the most secure setting to prevent issues such as data tampering or leaks.

For further improvements in the protocol, see the reference here. The Paho MQTT Python client supports the usage of all named properties.

Choosing an MQTT transport layer

As the second step for creating a client object, one has to choose a transport layer.

Pro Edition for Eclipse Mosquitto supports WebSocket and TCP transport layers, an advantage of hosted MQTT brokers. The former uses the HTTP protocol and has the following benefits: it is firewall-friendly, and you can utilize it with almost any internet connection. However, the TCP protocol performs better because the communication uses a lower layer – the TCP layer.

For this demo, I used a publicly available Python environment from mybinder.org. This means you have to use WebSockets to communicate with the MQTT broker.

To test our Pro Mosquitto MQTT broker, sign up for a free 14-day trial here!

Configure client authentication

To enable TLS for an MQTT broker and to access the broker, you must set two parameters for the client object in Python: the username/password and the TLS settings.

To do this, please check out the code snippet provided below:

client.username_pw_set("user", "password")
client.tls_set(certfile=None,
               keyfile=None,
               cert_reqs=ssl.CERT_REQUIRED)

The first method calls the username_pw_set parameter. It is straightforward and sets the username and password you chose in the “Get your ready-to-use and secure MQTT broker” step above.

The second method calls tls_set and ensures that the communication is encrypted. Setting the parameter cert_reqs to ssl.CERT_REQUIRED ensures that the connection is only established if the broker can provide a valid server certificate. This is because we are using the hosted MQTT broker version from Cedalo.

If your device does not support the CA from the Cedalo server, you can download the selfsigned .pem file here. Use client.tls_set(ca_certs="<path>/pemfile.pem") to set the correct CA path. 

As an alternative to username/password, you have the option of using certificate-based authentication and authorization (if your broker supports this). In this case, you must add the parameters certfile and keyfile to the path of the corresponding files (client certificate and private key). For more information, refer to the certificate-based authentication and authorization section of the MQTT TLS configuration guide. 

Configure callbacks and connect a client to your broker

Now, you are almost ready to connect to the broker. However, you must first define at least one callback that handles incoming messages on subscribed topics. To do this, the client library has the option to define callback functions. In this example, the callback functions are defined in a separate file named mycallbacks.

The code to help you set the essential callbacks is as follows:

import mycallbacks
client.on_message = mycallbacks.on_message;
client.on_connect = mycallbacks.on_connect;
client.on_publish = mycallbacks.on_publish;
client.on_subscribe = mycallbacks.on_subscribe;

The callback function on_message is defined as follows and publishes the information about the received message. The other optional callbacks in this example print out verbose information.

def on_message(client, userdata, message, properties=None):
    print(" Received message " + str(message.payload)
        + " on topic '" + message.topic
        + "' with QoS " + str(message.qos))

Now, you are ready to connect to the broker. Please note that the connect method call also differs depending on the MQTT protocol version. Please refer to the section ‘MQTT protocol versions’ above for details. Nevertheless, one has to set the communication port according to the chosen transport layer. For WebSocket – use port 443, and for TCP – use port 8883. Once you establish a successful connection to the broker, the prepared example will print out the callback message  (see the picture with the area marked in red).

broker = 'YOUR-BROKER-ADDRESS' # eg. choosen-name-xxxx.cedalo.cloud
myport = 443
if version == '5':
    from paho.mqtt.properties import Properties
    from paho.mqtt.packettypes import PacketTypes 
    properties=Properties(PacketTypes.CONNECT)
    properties.SessionExpiryInterval=30*60 # in seconds
    client.connect(broker,
                   port=myport,
                   clean_start=mqtt.MQTT_CLEAN_START_FIRST_ONLY,
                   properties=properties,
                   keepalive=60);

elif version == '3':
    client.connect(broker,port=myport,keepalive=60);

client.loop_start();

Until now, the code will run without any doubt. However, after trying to connect, possible issues may arise. Here is a small table to help you troubleshoot potential problems.

It is recommended to restart the kernel if you experience further issues within the web-based Python environment. To do so, select  kernel in the menu then ->Restart kernel and start the code execution again from the top.

Finally, the client method call loop_start() ensures the connection keeps the “online” status. Thus, you can publish messages and subscribe to topics, as shown in the following sections. Check out the MQTT subscribe blog post to learn how it works and to see relevant Mosquitto examples.

To test our Pro Mosquitto MQTT broker, sign up for a free 14-day trial here!

Subscribe to topics with the Paho MQTT client

Having completed the steps above, you can now easily subscribe to topics on the broker to which the client has access. To enable the subscription, call the client method _subscribe_ to subscribe to the required topic and set the  _quality of service_ optional parameter. Check out the MQTT QoS article to learn more about it.

mytopic = 'topic/important'
client.subscribe(mytopic,2);

Let’s use the same topic to publish messages so you can see how to receive messages.

Publish messages with the Paho MQTT client in Python

To publish messages on topics, you need to call the method to publish on the specific topic. You must also define the payload and the optional quality of service parameter. The following code lines show the case for MQTT version 5.0, which includes the option to specify the MessageExpiryInterval as mentioned in the section “Create the Python MQTT client object” above.

from paho.mqtt.properties import Properties
from paho.mqtt.packettypes import PacketTypes 
properties=Properties(PacketTypes.PUBLISH)
properties.MessageExpiryInterval=30 # in seconds
    
client.publish(mytopic,'Cedalo Mosquitto is awesome',2,properties=properties);

Here are some sample outputs from the browser-based Python environment.

Thanks to the connected callback functions, you will see the printout for publishing and receiving. The time difference is marked red in the figure above. In this case, the value is around ten milliseconds – indicating a round trip time for the messages.

In conclusion, any client implementation will only use one connection instance, reuse it, and if not needed anymore, disconnect it by calling disconnect().

Wrap up

In this article, I used the Eclipse Paho MQTT client for Python to publish and receive messages from the Mosquitto MQTT broker. 

In addition, I showed and explained the core differences between MQTT protocol versions 3.1.1 and 5.0 as part of the prepared Python environment. Furthermore, I have used the Management Center as a GUI to administer client access and monitor the Pro Edition for Mosquitto MQTT broker. 

Eventually, you can use the code snippets on GitHub for your upcoming projects utilizing the Python Paho client and Mosquitto.

An example connection could look like this:

import paho.mqtt.client as mqtt
import ssl
from datetime import datetime as dt
def on_connect(client, userdata, flags, reason_code, properties=None):
    client.subscribe(topic="RXB")
def on_message(client, userdata, message, properties=None):
    print(
        f"{dt.now()} Received message {message.payload} on topic '{message.topic}' with QoS {message.qos}"
    )
def on_subscribe(client, userdata, mid, qos, properties=None):
    print(f"{dt.now()} Subscribed with QoS {qos}")
client = mqtt.Client(client_id="clientid", protocol=mqtt.MQTTv311, clean_session=True)
client.on_connect = on_connect
client.on_message = on_message
client.on_subscribe = on_subscribe
client.username_pw_set(username="username", password="password")
client.tls_set(ca_certs="cacerts/isrgrootx1.pem")
client.connect(host="example.cedalo.cloud", port=8883, keepalive=60)
client.loop_forever()

About the author

Andreas Schiffler

Research Professor at the Technical University of Wuerzburg-Schweinfurt

Dr. Andreas Schiffler is a research professor at the Technical University of Wuerzburg-Schweinfurt in the field of production and data technology in mechanical engineering. In addition to research topics related to 3D metal printing, Dr. Schiffler developed a Kubernetes cluster for the practice-oriented basics of IoT and Industry 4.0 as part of the student training. Before joining the university, he worked in different product development-related positions for Siemens AG and Schaeffler AG.

His hobbies and private activities are mixed to share practical knowledge on topics like home automation driven by MQTT protocol or using open-source CNC controllers for machine tools.