MQTT Explained with Mosquitto: Topics, QoS, Retained Messages and LWT

MQTT is the protocol behind a large share of IoT products, from smart plugs to industrial sensors. It is small enough for an ESP32, yet it gives you push-based two-way messaging that HTTP cannot. To use it you need a broker: a server that accepts connections from clients and routes each published message to every client subscribed to that topic.

Eclipse Mosquitto is a free, open-source broker that runs comfortably on a Raspberry Pi or a Windows laptop. In this tutorial you will install it, lock it down with usernames and passwords, and use its command-line clients to explore every MQTT feature you will need for the ESP32 tutorials that follow.

Core MQTT Concepts

Term	Meaning	Example
Client	Any program connected to the broker; can publish, subscribe or both	ESP32 node, Node-RED, mosquitto_sub
Topic	A UTF-8 string with levels separated by /	home/livingroom/temp
Payload	The message body; the broker does not inspect it	{"temp_c":27.9}
Client ID	Unique name per connection; a duplicate kicks the older client off	esp32-livingroom
QoS	Delivery guarantee level 0, 1 or 2	QoS 1 for alarms
Retained	Broker stores the last message on a topic and sends it to new subscribers	Current relay state
Last Will (LWT)	Message the broker publishes if a client disconnects without saying goodbye	home/node1/status = offline
Keep alive	Interval within which the client must send something, or the broker drops it	15 seconds

Topic wildcards

Subscribers can use two wildcards. The + wildcard matches exactly one level, so home/+/temp matches home/kitchen/temp and home/bedroom/temp. The # wildcard matches all remaining levels and must be last, so home/# matches everything under home. Publishers can never use wildcards. Avoid leading slashes and spaces in topic names; home/kitchen/temp and /home/kitchen/temp are different topics.

QoS levels

QoS 0 (at most once): sent once, no acknowledgement. Fine for frequent sensor readings where the next sample replaces a lost one.
QoS 1 (at least once): the receiver acknowledges with PUBACK; the sender retries until it arrives. Duplicates are possible, so commands should be idempotent, such as "ON" rather than "TOGGLE".
QoS 2 (exactly once): a four-step handshake prevents duplicates. Slowest, rarely needed for hobby projects.

QoS applies separately to each hop: publisher to broker and broker to subscriber. The subscriber receives at the lower of the publish QoS and its own subscription QoS.

Installing Mosquitto

On Raspberry Pi OS or Ubuntu, install the broker and the client tools from the package manager. On Windows, download the installer from mosquitto.org; it installs a Windows service and the same command-line clients in the installation folder.

Install Mosquitto on Raspberry Pi OS / Ubuntu and create a user

sudo apt update
sudo apt install -y mosquitto mosquitto-clients
sudo systemctl enable mosquitto

# Create a password file with a first user (prompts for the password)
sudo mosquitto_passwd -c /etc/mosquitto/passwd iotuser
# Add more users later WITHOUT -c, or the file is overwritten
sudo mosquitto_passwd /etc/mosquitto/passwd nodered

# The broker runs as the mosquitto user, so it must be able to read the file
sudo chown mosquitto:mosquitto /etc/mosquitto/passwd
sudo chmod 600 /etc/mosquitto/passwd

Since version 2.0, Mosquitto only listens on the local machine and refuses anonymous clients unless you configure it otherwise. Create a configuration file that opens port 1883 to your network and requires a password.

/etc/mosquitto/conf.d/local.conf

# Listen on all network interfaces, port 1883 (plain MQTT, LAN only)
listener 1883

# Every client must log in with a username and password
allow_anonymous false
password_file /etc/mosquitto/passwd

# Keep retained messages and QoS 1/2 sessions across broker restarts
persistence true

Restart the broker and check that it is listening

sudo systemctl restart mosquitto
sudo systemctl status mosquitto --no-pager
# Find the Pi's IP address for your ESP32 sketches
hostname -I

Hands-On Experiments

Open two terminals. In the commands below, replace 192.168.1.50 with your broker's IP address and secret with your password.

Wildcards, retained messages and Last Will with the Mosquitto clients

# Terminal 1: subscribe to everything under home/, -v prints the topic too
mosquitto_sub -h 192.168.1.50 -u iotuser -P secret -t 'home/#' -v

# Terminal 2: normal publish
mosquitto_pub -h 192.168.1.50 -u iotuser -P secret -t home/kitchen/temp -m 28.1

# Retained publish: stop and restart terminal 1, the value appears at once
mosquitto_pub -h 192.168.1.50 -u iotuser -P secret -t home/fan/state -m ON -r

# Clear a retained message by publishing an empty retained payload
mosquitto_pub -h 192.168.1.50 -u iotuser -P secret -t home/fan/state -r -n

# Last Will: this client registers a will, then waits for input.
# Close the terminal window (do not press Ctrl+C) and watch terminal 1.
mosquitto_sub -h 192.168.1.50 -u iotuser -P secret -t dummy -k 10 \
  --will-topic home/node1/status --will-payload offline --will-retain

home/kitchen/temp 28.1
home/fan/state ON
home/node1/status offline

The Last Will appears once the broker notices the missing client, after roughly one and a half keep-alive periods. On an ESP32 you register the will during connect and publish a retained "online" message right after, so dashboards always know which nodes are alive.

Designing a Topic Tree

Choose a consistent structure before you have twenty devices. A pattern that works well is site/room/device/measurement, with separate topics for state and commands so a device never receives its own state messages.

home/kitchen/node1/temp        device publishes, QoS 0
home/kitchen/node1/status      online/offline, retained, LWT
home/kitchen/relay1/state      ON/OFF, retained, device publishes
home/kitchen/relay1/set        ON/OFF, app publishes, device subscribes

With this layout a dashboard can subscribe to home/+/+/status to list every device's health, or to home/kitchen/# to see one room. Keep payloads simple: plain values like ON for commands, and compact JSON for readings that carry several values together.

Watching the broker itself

Mosquitto publishes statistics under the special $SYS/ topic tree, such as the number of connected clients and messages received. These topics are not matched by #, so subscribe explicitly with mosquitto_sub -t '$SYS/broker/clients/connected' -u iotuser -P secret to check how many devices are online.

Troubleshooting

Connection refused: not authorised means a wrong username or password, or the broker cannot read the password file. Check ownership and run sudo journalctl -u mosquitto for details.
Clients on other machines cannot connect but localhost works: the listener 1883 line is missing, or a firewall blocks the port. On Windows allow port 1883 in Windows Defender Firewall.
A device keeps disconnecting every few seconds: two clients are using the same client ID and kick each other off.
Old values appear on subscribe: a retained message is stored on that topic. Clear it with an empty retained publish.

Summary

Mosquitto gives you a private MQTT broker in a few minutes. Topics and wildcards organise messages, QoS controls delivery, retained messages hold current state, and Last Will reports dead devices. With the broker running and password-protected, the next tutorial connects an ESP32 to it as a real sensor publisher.