Python

How to use the Paho MQTT client in Python (with examples)

Andreas Schiffler23.08.202223.09.2022

Like the Eclipse Mosquitto MQTT broker, the Eclipse Paho MQTT Python client is an open-source project. Because the Eclipse Paho MQTT client is available in various programming languages, it is the de facto standard for MQTT client functionality.

It is also worth mentioning that the multi-language capability of the Paho MQTT client is the reason why its creators chose “Paho,” meaning “communicate with everyone” (ref. Eclipse wiki), as the name for this project.

In the following sections of this article, I will show you how to use the Paho client with the Pro Edition for Eclipse Mosquitto MQTT broker. You do not require any additional prerequisites. Simply test it in your internet browser.

Set up the Paho MQTT Python client

Since we want to use the Python implementation of the Paho MQTT client, we need to have a Python execution environment in place. There are different approaches to setting this up. 

To follow the approach I want to share, you can start a ready-to-use environment by clicking this link. Doing so will launch the Jupyter Notebook in your internet browser complete with a Python MQTT example code. We will go through it step-by-step in the following sections. 

The use of this environment is straightforward. Click “play” (see the image below), and the selected Python code lines (= code marked with the blue vertical bar on the left) will run in your browser window. 

Another nice feature of the Jupyter Python environment is that when pressing ‘ctrl+I’ for Windows or ‘command+I’ for Mac (or right-clicking ‘Show Contextual Help’), you can get information about the specific function call depending on your cursor location.

If you decide to prepare or use your Python environment, install the Paho MQTT client. To do so, use the Python package tool. Check out this reference for more information.

pip install paho-mqtt

As seen in the picture above, the imported and used SSL package ( import ssl ) is always part of the Python default environment installation.

Get your ready-to-use and secure MQTT broker

Of course, we will also need an MQTT broker for our Paho MQTT Python setup. Sign up for a 14-day free trial and, within a few minutes, get a free version of a professional and secure Eclipse Mosquitto MQTT broker with an intuitive WebUI setup. Once you sign up and verify your email, you will receive an email with detailed instructions, as shown here.

Create the Python MQTT client object

Following the steps in the previous sections, we now have a Python environment and an MQTT broker set up. In addition, we have created at least one client with the following credentials:

Client id: “myPy”

Username: “user”

Password: “password” as shown step-by-step here.

Now let’s create a client object named client in Python.

import paho.mqtt.client as mqtt
import ssl

version = '5' # or '3' 
mytransport = 'websockets' # or 'tcp'

if version == '5':
    client = mqtt.Client(client_id="myPy",
                         transport=mytransport,
                         protocol=mqtt.MQTTv5)
if version == '3':
    client = mqtt.Client(client_id="myPy",
                         transport=mytransport,
                         protocol=mqtt.MQTTv311,
                         clean_session=True)

In the two sections below, I will provide an additional explanation of the above code snippet.

MQTT protocol version and the introduction of expiry intervals

MQTT communication is standardized. Currently, it has two versions: Version 3.1.1 and 5.0. During the Python MQTT configuration process, we can set the MQTT version to use by specifying the protocol parameter and choosing either the MQTTv311 or the MQTTv5 value from the Paho library (as shown in the code above). 

It is worth mentioning that the protocol version 5.0 was introduced in 2018 and released in March 2019. It is not yet part of the ISO standard. 

However, the MQTT protocol underwent several improvements from version 3.1.1 to 5.0. One of them is how the protocol handles session expiry. This aspect can be advantageous for TLS encryption because it saves overhead traffic by reusing the session data when deploying MQTT version 5.0. Refer to the MQTT TLS article for more details.

In version 3.1.1, with the default clean_session flag set to true (see the code snippet above), the data of every session (including outstanding messages, subscriptions, and TLS handshakes) is cleared if the keepalive period (the default value is 60 seconds) is up. This happens when there is no communication between a client and a broker. Check out this blog post to learn more about the CONNECT and CONNACK parameters required for establishing the MQTT connection.

With the MQTT version 5.0, the clean_session flag is split into a clean start flag, indicating that the session should start without using existing session data, and a SessionExpiryInterval flag (see later in “Configure callbacks and connect a client to your broker” section), which defines how long to retain the session after a disconnect. Setting the clean_start flag to 1 and SessionExpiryInterval to 0 is equivalent to setting clean_session to 1 in MQTT v3.1.1. 

The MessageExpiryInterval (see “Publish messages with the Paho MQTT client in Python” section) allows using an expiry interval for so-called Last Will messages published with the retain flag set to 1 (“last will”). After the MessageExpiryInterval, the lastWillMessage will be dismissed (read here about the lastWill parameter implementation for the Mosquitto broker). Also, the MessageExpiryInterval, which has been available since version 5.0, is fully supported by Pro Edition of Eclipse Mosquitto. It allows the implementation of application-specific settings and boosts functionality. 

Examples of “last will” usage

If you would like examples of “last will” usage, I will happily give you several.

Imagine that your IoT infrastructure includes thousands of connected IoT clients like cars, trucks, or other assets with a high probability of going offline for minutes or even weeks. In cases like this, messages (e.g., about traffic jams) must have a short expiry interval. Other notices or updates must still be delivered after a client re-connects, which can take up to several weeks after going offline.

SessionExpiryInterval is a helpful setting to choose between reducing traffic overhead and increasing information security. Imagine an IoT device with a small footprint in compute performance, energy consumption, and traffic volume. This device may have an unstable or interrupted network connection.

With a very short SessionExpiryInterval on every reconnection, the TLS handshake and certificate exchange may have to be executed. However, this would be the most secure setting to prevent issues such as data tampering or leaks.

For further improvements in the protocol, see the reference here. The Paho MQTT Python client supports the usage of all named properties.

Choosing an MQTT transport layer

As the second step for creating a client object, one has to choose a transport layer.

Pro Edition for Eclipse Mosquitto supports WebSocket and TCP transport layers, an advantage of hosted MQTT brokers. The former uses the HTTP protocol and has the following benefits: it is firewall-friendly, and one can utilize it with almost any internet connection. The TCP protocol performs better because the communication uses a lower layer – the TCP layer. 

For this demo, where we use a publicly available Python environment from mybinder.org, we have to use WebSockets to communicate with the MQTT broker.

Configure client authentication

To enable TLS for an MQTT broker and access the broker, we need to set two parameters for the client object in Python: the username/password and the TLS settings.

To do it, please check out the following snippet of code:

client.username_pw_set("user", "password")
client.tls_set(certfile=None,
               keyfile=None,
               cert_reqs=ssl.CERT_REQUIRED)

The first method calls the username_pw_set parameter. It is straightforward and sets the username and password you chose in the “Get your ready-to-use and secure MQTT broker” step above.

The second method calls tls_set and ensures that the communication is encrypted. Setting the parameter cert_reqs to ssl.CERT_REQUIRED ensures that the connection is only established if the broker can provide a valid server certificate. This is because we are using the hosted MQTT broker version from Cedalo. 

Optionally and as an alternative to username/password, you can use certificate-based authentication and authorization (if your broker supports this). In this case, you must add the parameters certfile and keyfile to the path of the corresponding files (client certificate and private key). For more information, refer to the certificate-based authentication and authorization section of the MQTT TLS configuration guide. 

Configure callbacks and connect a client to your broker

Now we are almost ready to connect to the broker. However, we first need to define at least one callback that handles incoming messages on subscribed topics. To do this, the client library has the option to define callback functions. In this example, the callback functions are defined in a separate file named mycallbacks.

The code to help you set the essential callbacks is as follows:

import mycallbacks
client.on_message = mycallbacks.on_message;
client.on_connect = mycallbacks.on_connect;
client.on_publish = mycallbacks.on_publish;
client.on_subscribe = mycallbacks.on_subscribe;

The callback function on_message is defined as follows and publishes the information about the received message. The other optional callbacks in this example print out verbose information.

def on_message(client, userdata, message,tmp=None):
    print(" Received message " + str(message.payload)
        + " on topic '" + message.topic
        + "' with QoS " + str(message.qos))

Now we are ready to connect to the broker. Please note that the connect method call also differs depending on the MQTT protocol version. Please refer to the section ‘MQTT protocol versions’ above for details. Nevertheless, one has to set the communication port according to the chosen transport layer. For WebSocket – use port 443, and for TCP – use port 8883. The prepared example will print out the callback message after a successful connection to the broker (see the picture with an area marked in red).

broker = 'YOUR-BROKER-ADDRESS' # eg. choosen-name-xxxx.cedalo.cloud
myport = 443
if version == '5':
    from paho.mqtt.properties import Properties
    from paho.mqtt.packettypes import PacketTypes 
    properties=Properties(PacketTypes.CONNECT)
    properties.SessionExpiryInterval=30*60 # in seconds
    client.connect(broker,
                   port=myport,
                   clean_start=mqtt.MQTT_CLEAN_START_FIRST_ONLY,
                   properties=properties,
                   keepalive=60);

if version == '3':
    client.connect(broker,port=myport,keepalive=60);

client.loop_start();

Up till now, the code will run without any doubt. Now, after trying to connect, possible issues may occur. Here is a small table to help you troubleshoot potential problems.

Another recommendation will be to restart the kernel if you have further issues within the web-based Python environment. To do so, click in the menu kernel->Restart kernel and start the code execution again from the top.

Finally, the client method call loop_start() ensures that the connection keeps the “online” status. Thus, you can publish messages and subscribe to topics, as shown in the following sections. Check out the MQTT subscribe blog post to learn how it works, as well as see relevant Mosquitto examples.

Subscribe to topics with the Paho MQTT client

Having completed the above steps, we can now easily subscribe to topics on the broker to which the client has access. To enable the subscription, we call the client method _subscribe_ to subscribe to the required topic and set the  _quality of service_ optional parameter.

mytopic = 'topic/important'
client.subscribe(mytopic,2);

We will use the same topic to publish messages so we can see how we receive our messages.

Publish messages with the Paho MQTT client in Python

To publish messages on topics, we need to call the method to publish on the specific topic and define the payload and the optional quality of service parameter. The following code lines show the use for MQTT version 5.0, which includes the option to specify the MessageExpiryInterval as mentioned in the section “Create the Python MQTT client object” above.

from paho.mqtt.properties import Properties
from paho.mqtt.packettypes import PacketTypes 
properties=Properties(PacketTypes.PUBLISH)
properties.MessageExpiryInterval=30 # in seconds
    
client.publish(mytopic,'Cedalo Mosquitto is awesome',2,properties=properties);

Here are some sample outputs from the browser-based Python environment.

Thanks to the connected callback functions, you will see the printout for publishing and receiving. The time difference – marked in red on the figure above. In this case, the value is around ten milliseconds – indicates a round trip time for the messages.

Finally, any client implementation will only use one connection instance, reuse it, and if not needed anymore, disconnect it by calling disconnect().

Wrap up

In this article, we used the Eclipse Paho MQTT client for Python to publish and receive messages from the Mosquitto MQTT broker. 

In addition, I showed and explained the core differences between MQTT protocol versions 3.1.1 and 5.0 as part of the prepared Python environment. Furthermore, you have used the Management Center as a GUI to administer client access and monitor the Pro Edition for Mosquitto MQTT broker. 

Eventually, you can use the code snippets on GitHub for your upcoming projects utilizing the Python Paho client and Mosquitto.