Connected devices

How to Use MQTT on ESP8266 Module: Detailed Guide

Andreas Schiffler28.10.202221.06.2024

The rising number of small Internet of Things (IoT) devices is driving the development of highly integrated, cost and energy-efficient microcontrollers (µC) or systems on a chip (SoC). The Chinese manufacturer, Espressif, offers the ESP-Series as SoC which is very popular for IoT applications. Numerous suppliers provide ready-to-use applications or development boards. For instance, the ESP8266 NodeMCU is one possible module to use the ESP SoC. The board includes a Wi-Fi antenna and USB connection to power and flash the SoC. In addition, a Wi-Fi module with support for the 802.11 b/g/n standards is part of the highly integrated SoC.

With digital inputs, outputs, and digital bus interfaces, smart IoT applications with their web interfaces are possible. You can find a possible application in the project here. As you can see, the ESP8266 grabs actual power values from solar microinverters and publishes them on MQTT topics for subsequent analysis, visualization, or control via home automation systems.

Software needed to use MQTT on ESP8266

In principle, for µC and SoC, there is no operating system like Windows or Linux where you can run and use, for example, the Paho client libraries to connect to MQTT brokers. Instead, you must create a firmware that the device will flash and then realize the application. The ESP8266 SoC was released in 2015, and for software development, the manufacturer offers the Espressif IoT Development Framework (esp-idf).

However, for beginners, coding with this framework might not be straightforward. Alternatively, one can use the popular Arduino framework, which consists of software libraries and an integrated development environment (IDE). Moreover, it includes support for ESP8266. Lastly, one can also use the VSCode IDE from Microsoft with the platformIO extension. See the following table as a reference:

Name	SDK / Libraries	IDE
ESP-IDF	Espressif Systems https://github.com/espressif/esp-idf	Eclipse, VSCode, Common CMake IDEs
Arduino	Arduino https://github.com/arduino/library-registry	Arduino 2.0 (main file extension *.ino)
PlatformIO	Same as for Arduino	VSCode + PlatformIO Extention

For this tutorial, we use the last one because this option combines the advantages of simple and beginner-friendly Arduino libraries and the powerful VSCode IDE, which can run as a desktop or browser-based application.

In addition, you can use the prepared and ready-to-code environment provided here and the sample code in your internet browser (free GitHub account needed). To use and test the application on an ESP8266 module, you need to adjust the code with your credentials, build the firmware with the command pio run in the IDE, and finally flash the created firmware onto the physical device. You can find more details in the code repository.

Example Application for MQTT on ESP8266

Altogether, the following sections explain a basic program enabling the ESP8266 to connect securely against the Pro Mosquitto to publish and subscribe to topics.

The sample application will have a simple functionality. In brief, every time a publisher posts a message on a specific time, the ESP8266 will post the actual time in synchronization via NTP on another topic.

Requirements for the MQTT Broker

To make use of this functionality, you will need the following:

• A running MQTT broker and the address (ipv4 or DNS name).
• Credentials (username/password) to publish and subscribe to topics.
• The X509 certificate of the broker (essential if the connection is over public networks and I recommend to secure it by TLS transport).

Pro Edition for Eclipse Mosquitto can easily fulfill all the above requirements. Also, see the tutorial here on how to set it up. Sign up for a 14-day free trial and, within a few minutes, get a free version of a professional and secure Pro Mosquitto with an intuitive WebUI setup.

Configure the C++ Application

You can find the sample code in the public GitHub repository here. In the src folder, you can find all the code in one main.cpp file. In the first part, you need to configure some variables. All in all, the credentials and other settings are hard-coded in this basic sample application.

 1| //#define MQTT_TLS 
 2| //#define MQTT_TLS_VERIFY
 3| const char* ssid = "YOUR-WIFI";
 4| const char* password = "YOUR-WIFI-PSK";
 5| const char* mqtt_server = "BROKER";
 6| const uint16_t mqtt_server_port = 1883; 
 7| const char* mqttUser = "user";
 8| const char* mqttPassword = "pass";
 9| const char* mqttTopicIn = "esp-8266-in";
10| const char* mqttTopicOut = "esp-8266-out";

The defines in lines 1 and 2 that you can see in the code snippet above will be part of the later section, Connect via TLS. The network connection of the ESP8266 module uses the built-in Wi-Fi. Therefore, you need to configure the station id (line 3) and the pre-shared key (line 4) of the Wi-Fi, which the ESP8266 shall use.

The variables in lines 5 to 8 define the access to the MQTT broker. The value for mqtt_server can either be an ipv4 address or a DNS name. The port (line 6) is, in most cases, should be 1883 for unencrypted MQTT protocol transport and 8883 for encrypted. Note: The provided Adruino MQTT libraries do not support transport via WebSockets.

Of course, you can choose the definition for the topics (lines 9, 10). However, to avoid issues, I recommend picking different ones.

ESP8266 MQTT Code Structure and Functions

Using the Arduino framework and libraries will require at least two functions – setup and loop in the main code file. And, as the names state, the setup is executed after startup once. The loop function is called periodically every time the last loop call finishes.

Also, you will need a few global variables:

• Includes

#include "Arduino.h"
#include "ESP8266WiFi.h"
#include "PubSubClient.h"
#include "NTPClient.h"
#include "WiFiUdp.h"
#include "certificate.h"

• Global variables

11| WiFiClient wifiClient;
12| WiFiUDP ntpUDP;
13| NTPClient timeClient(ntpUDP);
14| PubSubClient mqttClient(wifiClient);

• Setup function

15| void setup() {
16|   Serial.begin(115200);
17|   setup_wifi();
18|   mqttClient.setServer(mqtt_server, mqtt_server_port);
19|   mqttClient.setCallback(callback);
20| }

• Loop function

21| void loop() {
22|   if (!mqttClient.connected()) {
23|     connect();
24|   }
25|   mqttClient.loop();
26|   timeClient.update();
27| }

In the setup function that you see above, the major function calls are the connection setup for the MQTT broker (line 18): mqttClient.setServer(mqtt_server, mqtt_server_port) and the definition of the callback function (line 19): mqttClient.setCallback(callback)

In this case, the latter defines the function, which is called every time the client receives a new message on a subscribed topic from the broker.

In the loop function, the function call mqttClient.loop() (line 25) proceeds the MQTT client communication with the MQTT broker. Thus, to ensure that the local time on ESP8266 is in sync, the function call on line 26 runs to check and correct possible time drifts between the ESP8266 module and NTP servers. You can also check out our article on how to set up a Paho MQTT Python client securely, connect it to an MQTT broker, publish messages on topics and subscribe to them.

To ensure there is an active connection between the client and the MQTT broker, in lines 22, 23 the connection state is checked in every loop. Therefore, if the connection to the MQTT broker is lost or not yet established, the function connect() is called.

You can see the example of the connect() function below:

27| void connect() {
28|   while (!mqttClient.connected()) {
29|     Serial.print("Attempting MQTT connection...");
30|     String mqttClientId = "";
31|     if (mqttClient.connect(mqttClientId.c_str(), mqttUser, mqttPassword)) {
32|       Serial.println("connected");
33|       mqttClient.subscribe(mqttTopicIn);
34|     } else {
35|       Serial.print("failed, rc=");
36|       Serial.print(mqttClient.state());
37|       Serial.println(" will try again in 5 seconds");
38|       delay(5000);
39|     }
40|   }
41| }

As a result, the logic that I implemented in the connect function ensures that connection attempts happen every five seconds if no connection is possible.

The MQTT Callback Function on ESP8266

In the sample application below, you can see the callback function that handles the MQTT communication events and implements the main functionality:

42| void callback(char* topic, byte* payload, unsigned int length) {
43|   Serial.print("Message arrived on topic: '");
44|   Serial.print(topic);
45|   Serial.print("' with payload: ");
46|   for (unsigned int i = 0; i < length; i++) {
47|     Serial.print((char)payload[i]);
48|   }
49|   Serial.println();
50|   String myCurrentTime = timeClient.getFormattedTime();
51|   mqttClient.publish(mqttTopicOut,("Time: " + myCurrentTime).c_str());
52| }

During the connect procedure, the ESP8266 subscribes to the topic ‘esp-8266-in’ (see lines 33 and 9). Now every time an MQTT client publishes a message on this topic, the broker sends the message to the ESP8266, and then the callback function is called.

As you can see in lines 43 to 48 above, the content of the received message is sent out on the serial interface just as information.

As a demonstration of functionality in line 51, the current time on the ESP8266 module is published on the topic ‘eps-8266-out’ (topic name definition, see line 10). You can find the code here as a complete reference.

Configure the Application to use TLS on ESP8266 for MQTT

In the previous sections, I configured the application to not use an encrypted transport for the MQTT payloads. To enable encrypted transport via TLS, you need to uncomment at least line 1 to create a definition named MQTT_TLS. Leaving line 2 commented means that no certificate verification is going to happen. That is why, any MQTT broker identity can be accepted.

 1| #define MQTT_TLS 
 2| //#define MQTT_TLS_VERIFY

As a result, the change in the code will do two small actions. Firstly, the global variable wifiClient is replaced by the secured version WiFiClientSecure wifiClient, ensuring that every transport is now encrypted using TLS.

The second action lies in line 69 in the setup_wifi function (see below). With the method called wifiClient.setInsecure(), we force the client to accept every certificate provided by the broker (server) during the connection parameter negotiation. The name sounds confusing but, in the end, TLS encrypts the transport with the only drawback that every identity, e.g., invalid or self-signed X509 certificates, is accepted.

53| void setup_wifi() {
54|   delay(10);
55|   Serial.println();
56|   Serial.print("Connecting to ");
57|   Serial.println(ssid);
58|   WiFi.begin(ssid, password);
59|   while (WiFi.status() != WL_CONNECTED) {
60|     delay(500);
61|     Serial.print(".");
62|   }
63|   timeClient.begin();
64| #ifdef MQTT_TLS
65|   #ifdef MQTT_TLS_VERIFY
66|     X509List *cert = new X509List(CERT);
67|     wifiClient.setTrustAnchors(cert);
68|   #else
69|     wifiClient.setInsecure();
70|   #endif
71| #endif
72|   Serial.println("WiFi connected");
73| }

Enable Certificate Verification on ESP8266 for MQTT

To begin with, to secure the configuration for the application of the ESP8266 module with the MQTT broker, one should enable the verification of the broker identity. To enable this uncomment, line 2 in the configuration part of the code looks like the following:

 1| #define MQTT_TLS 
 2| #define MQTT_TLS_VERIFY

If the define MQTT_TLS_VERIFY exists in the code, then the certificate’s validity for TLS encryption is checked. You can see in the setup_wifi function above, in lines 66 and 67, that trusted anchors in that case are set for the wifiClient.

In PC- based environments, there are root certificates, which are – as the name states – the roots of certificate trust paths. Therefore, by default, every communication which uses valid certificates from the root certificates is trusted. The following figure shows an example trust path for the website www.cedalo.com.

In the ESP8266, we need to store the public information of the trusted certificates so that the identity of the MQTT broker can be verified. In PC-based environments, storing hundreds of root certificates is no issue, but in an ESP8266 one, the storage (flash) has its limited. Consequently, you need to choose specific certificates to limit the need for storage.

Thus, the easiest option is to store the MQTT broker certificate or the certificate of the issuing CA. If you use the recommended trial setup for Pro edition for Eclipse Mosquitto, the corresponding certificate to trust is the Let’s Encrypt R3 intermediate certificate. At the same time, the sample application stores and provides it by default.

To check or change the correct certificate, which is used as a trust anchor in the ESP8266, view or edit the content in the include/certificate.h file, in the project folder. Also, be careful when changing the lines between BEGIN CERTIFICATE and END CERTIFICATE.

// (re)place the trusted X509 certificate for the tls connection below
// default below: Let's Encrypt R3
const char mqtt_broker_cert[] PROGMEM = R"EOF(
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
…
nLRbwHOoq7hHwg==
-----END CERTIFICATE-----
)EOF";

After any change in the main.cpp or certificate.h project files, start a recompilation and the creation of the firmware by using the command pio run in the terminal of the browser-based IDE or by clicking ‘build’ (if you use a local VSCode installation).

Possible Issues and Troubleshooting

Finally, it can be challenging to realize and debug applications created for SoC or µC successfully. That’s because for debugging, in most cases, you might need additional hardware, and access to, e.g., logs files or similar helpful things, which is limited or not possible due to a missing operating system environment on the ESP8266. The following table provides support and can help if any issues occur. Also, refer to the readme section in the sample code repository here, which shows how to monitor the serial output of the ESP8266.

Output / Behavior	Possible reason / Fix
The code compilation fails. (IDE output “build failed”)	Check the compiler’s error messages, which show line numbers and files where the error occurs. Try to fix the code.
The creation of the firmware fails. (IDE output “build failed”)	The most common is here missing libraries or wrong paths. Here in the sample application, this error will be improbable.
In the serial output of the ESP8266, the Wi-Fi connection failed	Check the PSK, Wi-Fi SSID and if the network is reachable and has sufficient signal strength (refer to the config section in the main code)
In the serial output of the ESP8266, the MQTT connection failed	Check the username and password (if needed and set it on the MQTT broker). Is the address correct? Can the local (Wi-Fi) network reach the broker on the defined port? Also, try first to connect w/o using TLS.
MQTT connection works only w/o using TLS	Have you set the port correctly (1883 vs. 8883)? Are the ports reachable (firewall settings in local network)? Does the broker support TLS? (check with PC-based clients) If TLS verification is enabled, check that the trusted certificate corresponds to the MQTT broker trust path. In case of doubts, use the broker certificate itself in the file certificate.h.