The last tutorial explained that MQTT clients connect to a broker, and these clients can publish messages to the broker or receive messages from a broker. Clients can be both publishers and recipients of messages. To organize messages, clients must send messages to a topic, such as "home/living-room/temperature", and clients must subscribe to that same topic if they are to receive those messages. The broker's job is to route messages from publishers to subscribers, making sure that subscribers only receive messages on the desired topic.
This tutorial covers:
- Connections, subscriptions, and the keep alive "heartbeat"
- Publication and QoS levels
- Downgraded QoS Levels
- How to choose between the QoS levels
To get a detailed understanding of the lifecycle of a MQTT connection and a message publication, we will perform variants of an experiment we performed in the last tutorial.
Connections, Subscriptions, and "Heartbeat"
For this experiment, we will need two Terminal windows:
Window #1 will be running the Mosquitto broker, started with this command:
mosquitto -v
The -v flag tells the broker to be verbose in output.
Window #2 will be running a subscribing client. After Mosquitto has started, enter the following command in the second window:
mosquitto_sub -d -h localhost -i Subscriber1 -t home/living-room/temperature
Here's an explanation of the flags used when starting the mosquitto_sub client:
- -d debug
- -h hostname (in this case, localhost)
- -i client ID (Subscriber1)
- -t topic (home/living-room/temperature)
Leave these running for a few minutes, then press CTRL-C in Window #2 to stop the subscribing client.
The broker window will display something like the following:
pi@raspberrypi:~ $ mosquitto -v 1490306591: mosquitto version 1.4.10 (build date 2017-02-19 02:05:51+0000) starting 1490306591: Using default config. 1490306591: Opening ipv6 listen socket on port 1883. 1490306591: Opening ipv4 listen socket on port 1883. 1490306760: New connection from ::1 on port 1883. 1490306760: New client connected from ::1 as Subscriber1 (c1, k60). 1490306760: Sending CONNACK to Subscriber1 (0, 0) 1490306760: Received SUBSCRIBE from Subscriber1 1490306760: home/living-room/temperature (QoS 0) 1490306760: Subscriber1 0 home/living-room/temperature 1490306760: Sending SUBACK to Subscriber1 1490306820: Received PINGREQ from Subscriber1 1490306820: Sending PINGRESP to Subscriber1 1490306880: Received PINGREQ from Subscriber1 1490306880: Sending PINGRESP to Subscriber1 1490306896: Socket error on client Subscriber1, disconnecting.
Meanwhile in Window #2, we see the following:
pi@raspberrypi:~ $ mosquitto_sub -d -h localhost -i Subscriber1 -t home/living-room/temperature Client Subscriber1 sending CONNECT Client Subscriber1 received CONNACK Client Subscriber1 sending SUBSCRIBE (Mid: 1, Topic: home/living-room/temperature, QoS: 0) Client Subscriber1 received SUBACK Subscribed (mid: 1): 0 Client Subscriber1 sending PINGREQ Client Subscriber1 received PINGRESP Client Subscriber1 sending PINGREQ Client Subscriber1 received PINGRESP ^C
The communication back and forth is in the form of a series of "command messages".
When mosquitto_sub is started, it first sends a CONNECT command message to the broker. The broker responds with a CONNACK message to acknowledge that it has indeed connected to the client. Next, the client sends a SUBSCRIBE message, along with the topic. The broker responds with SUBACK.
And that's how a client connects to a broker!
After the connection is established, the client periodically sends pings to the broker to ensure that the broker hasn't crashed. The client sends PINGREQ, and the broker sends-back PINGRESP. This is MQTT's "keep alive" functionality, essentially a MQTT "heartbeat".
The entire process can be shown as follows, with the vertical axis representing time, increasing as we read from top to bottom:
In summary, we have seen the following command messages being exchanged:
CONNECT - client requests a connection to the broker
CONNACK - broker acknowledges that the connection was either accepted or refused
SUBSCRIBE - subscribe to a topic (subscribe messages use QoS 1)
SUBACK - subscription acknowledgement
PINGREQ - the client is asking if the broker is alive
PINGRESP - the broker sends a PINGRESP message back to the client indicating that the broker is indeed alive
There are three additional command messages that can be seen when unsubscribing or disconnecting. Unfortunately, there does not seem to be a way to get mosquitto_sub to generate these messages:
UNSUBSCRIBE - client tells the broker that it wants to unsubscribe from a topic
UNSUBACK - broker acknowledges receipt of the UNSUBSCRIBE message
DISCONNECT - the client sends this to the broker when it is about to close the TCP/IP connection.
Publication and QoS Levels
In the last tutorial, we discussed the Quality of Service (QoS) levels. Each QoS level is an agreement between sender and receiver guaranteeing receipt of the message. There are three of them, and can be roughly described as follows:
QoS 0 - message is received at most once
QoS 1 - message is received at least once
QoS 2 - message is received exactly once
Each connection from client to broker or broker to client can have its own QoS. Thus, it is possible for connections to have different QoS levels. Let us first look at the situation where the QoS levels are the same going from publisher to broker and from broker to subscriber.
For QoS 1 and QoS 2, there is a confirmation in addition to publishing data (two steps for QoS 1 and four steps for QoS 2). The sender and/or the recipient must retain the data until this confirmation is complete.
To investigate the process of sending and receiving messages, we will set-up three Terminal windows: one for the broker, one for the subscribed client, and one for the publishing client.
QoS 0 is "fire and forget". Let's try an experiment, running these commands in three separate windows in the following order:
Window 1: mosquitto -v
Window 2: mosquitto_sub -d -h localhost -i Subscriber1 -t home/living-room/temperature
Window 3: mosquitto_pub -d -h localhost -i Publisher1 -t home/living-room/temperature -m "Hello, QoS 0"
There is one additional flag in the last command:
- -m is the message to be sent
The clients mosquitto_sub and mosquitto_pub default to QoS 0, so there is no need to specify that in the command line.
Window 1:
pi@raspberrypi:~ $ mosquitto -v 1490312451: mosquitto version 1.4.10 (build date 2017-02-19 02:05:51+0000) starting 1490312451: Using default config. 1490312451: Opening ipv6 listen socket on port 1883. 1490312451: Opening ipv4 listen socket on port 1883. 1490312454: New connection from ::1 on port 1883. 1490312454: New client connected from ::1 as Subscriber1 (c1, k60). 1490312454: Sending CONNACK to Subscriber1 (0, 0) 1490312454: Received SUBSCRIBE from Subscriber1 1490312454: home/living-room/temperature (QoS 0) 1490312454: Subscriber1 0 home/living-room/temperature 1490312454: Sending SUBACK to Subscriber1 1490312459: New connection from ::1 on port 1883. 1490312459: New client connected from ::1 as Publisher1 (c1, k60). 1490312459: Sending CONNACK to Publisher1 (0, 0) 1490312459: Received PUBLISH from Publisher1 (d0, q0, r0, m0, 'home/living-room/temperature', ... (12 bytes)) 1490312459: Socket error on client Publisher1, disconnecting. 1490312459: Sending PUBLISH to Subscriber1 (d0, q0, r0, m0, 'home/living-room/temperature', ... (12 bytes)) 1490312514: Received PINGREQ from Subscriber1 1490312514: Sending PINGRESP to Subscriber1 1490312574: Received PINGREQ from Subscriber1 1490312574: Sending PINGRESP to Subscriber1
Window 2:
pi@raspberrypi:~ $ mosquitto_sub -d -h localhost -i Subscriber1 -t home/living-room/temperature Client Subscriber1 sending CONNECT Client Subscriber1 received CONNACK Client Subscriber1 sending SUBSCRIBE (Mid: 1, Topic: home/living-room/temperature, QoS: 0) Client Subscriber1 received SUBACK Subscribed (mid: 1): 0 Client Subscriber1 received PUBLISH (d0, q0, r0, m0, 'home/living-room/temperature', ... (12 bytes)) Hello, QoS 0 Client Subscriber1 sending PINGREQ Client Subscriber1 received PINGRESP Client Subscriber1 sending PINGREQ Client Subscriber1 received PINGRESP
Window 3:
pi@raspberrypi:~ $ mosquitto_pub -d -h localhost -i Publisher1 -t home/living-room/temperature -m "Hello, QoS 0" Client Publisher1 sending CONNECT Client Publisher1 received CONNACK Client Publisher1 sending PUBLISH (d0, q0, r0, m1, 'home/living-room/temperature', ... (12 bytes)) Client Publisher1 sending DISCONNECT pi@raspberrypi:~ $
The publisher first sends the CONNECT command message, then receives CONNACK. Next, it sends a PUBLISH command message, and then it sends DISCONNECT message. Here's what the procell looks like, focusing on just the PUBLISH messages.
The broker receives the PUBLISH message, then sends the PUBLISH message on to the subscriber.
Once the sender (either Publisher1 or the broker) sends the message, there is no reason to retain it. Truly, QoS 0 is fire and forget!
Now let us try a QoS 1 message...
Window 1: mosquitto -v
Window 2: mosquitto_sub -d -h localhost -q 1 -i Subscriber1 -t home/living-room/temperature
Window 3: mosquitto_pub -d -h localhost -q 1 -i Publisher1 -t home/living-room/temperature -m "Hello, QoS 1"
There is an additional flag in the last two commands:
- -q is the QoS level
Window 1:
pi@raspberrypi:~ $ mosquitto -v 1490312729: mosquitto version 1.4.10 (build date 2017-02-19 02:05:51+0000) starting 1490312729: Using default config. 1490312729: Opening ipv6 listen socket on port 1883. 1490312729: Opening ipv4 listen socket on port 1883. 1490312742: New connection from ::1 on port 1883. 1490312742: New client connected from ::1 as Subscriber1 (c1, k60). 1490312742: Sending CONNACK to Subscriber1 (0, 0) 1490312742: Received SUBSCRIBE from Subscriber1 1490312742: home/living-room/temperature (QoS 1) 1490312742: Subscriber1 1 home/living-room/temperature 1490312742: Sending SUBACK to Subscriber1 1490312760: New connection from ::1 on port 1883. 1490312760: New client connected from ::1 as Publisher1 (c1, k60). 1490312760: Sending CONNACK to Publisher1 (0, 0) 1490312760: Received PUBLISH from Publisher1 (d0, q1, r0, m1, 'home/living-room/temperature', ... (12 bytes)) 1490312760: Sending PUBACK to Publisher1 (Mid: 1) 1490312760: Sending PUBLISH to Subscriber1 (d0, q1, r0, m1, 'home/living-room/temperature', ... (12 bytes)) 1490312760: Received PUBACK from Subscriber1 (Mid: 1) 1490312760: Received DISCONNECT from Publisher1 1490312760: Client Publisher1 disconnected. 1490312820: Received PINGREQ from Subscriber1 1490312820: Sending PINGRESP to Subscriber1
Window 2:
pi@raspberrypi:~ $ mosquitto_sub -d -h localhost -q 1 -i Subscriber1 -t home/living-room/temperature Client Subscriber1 sending CONNECT Client Subscriber1 received CONNACK Client Subscriber1 sending SUBSCRIBE (Mid: 1, Topic: home/living-room/temperature, QoS: 1) Client Subscriber1 received SUBACK Subscribed (mid: 1): 1 Client Subscriber1 received PUBLISH (d0, q1, r0, m1, 'home/living-room/temperature', ... (12 bytes)) Client Subscriber1 sending PUBACK (Mid: 1) Hello, QoS 1 Client Subscriber1 sending PINGREQ Client Subscriber1 received PINGRESP
Window 3:
pi@raspberrypi:~ $ mosquitto_pub -d -h localhost -q 1 -i Publisher1 -t home/living-room/temperature -m "Hello, QoS 1" Client Publisher1 sending CONNECT Client Publisher1 received CONNACK Client Publisher1 sending PUBLISH (d0, q1, r0, m1, 'home/living-room/temperature', ... (12 bytes)) Client Publisher1 received PUBACK (Mid: 1) Client Publisher1 sending DISCONNECT
Publisher1 sends a PUBLISH message. The broker receives the message, then sends a PUBACK message to Publisher1. Publisher1 receives the PUBACK message, letting Publisher1 know that the recipient (broker) has received the message. The exact same thing happens when the message is sent from the broker to Subscriber1.
The sender of the message (either Publisher1 -- QoS 1 --> Broker or Broker -- QoS 1 --> Subscriber1) must retain the message until the PUBACK message is received. So, when Publisher1 receives PUBACK, it is safe to delete the message.
How does the sender know how to match PUBLISH and PUBACK commands? Each message has a packet identifier, so the sender knows when to delete a message when the packet ID of the PUBLISH message matches the packet ID of the PUBACK message.
Finally, let's see the steps involved in QoS 2 messages
Window 1: mosquitto -v
Window 2: mosquitto_sub -d -h localhost -q 2 -i Subscriber1 -t home/living-room/temperature
Window 3: mosquitto_pub -d -h localhost -q 2 -i Publisher1 -t home/living-room/temperature -m "Hello, QoS 2"
Window 1:
pi@raspberrypi:~ $ mosquitto -v 1490312914: mosquitto version 1.4.10 (build date 2017-02-19 02:05:51+0000) starting 1490312914: Using default config. 1490312914: Opening ipv6 listen socket on port 1883. 1490312914: Opening ipv4 listen socket on port 1883. 1490312927: New connection from ::1 on port 1883. 1490312927: New client connected from ::1 as Subscriber1 (c1, k60). 1490312927: Sending CONNACK to Subscriber1 (0, 0) 1490312927: Received SUBSCRIBE from Subscriber1 1490312927: home/living-room/temperature (QoS 2) 1490312927: Subscriber1 2 home/living-room/temperature 1490312927: Sending SUBACK to Subscriber1 1490312955: New connection from ::1 on port 1883. 1490312955: New client connected from ::1 as Publisher1 (c1, k60). 1490312955: Sending CONNACK to Publisher1 (0, 0) 1490312955: Received PUBLISH from Publisher1 (d0, q2, r0, m1, 'home/living-room/temperature', ... (12 bytes)) 1490312955: Sending PUBREC to Publisher1 (Mid: 1) 1490312955: Received PUBREL from Publisher1 (Mid: 1) 1490312955: Sending PUBCOMP to Publisher1 (Mid: 1) 1490312955: Sending PUBLISH to Subscriber1 (d0, q2, r0, m1, 'home/living-room/temperature', ... (12 bytes)) 1490312955: Received DISCONNECT from Publisher1 1490312955: Client Publisher1 disconnected. 1490312955: Received PUBREC from Subscriber1 (Mid: 1) 1490312955: Sending PUBREL to Subscriber1 (Mid: 1) 1490312955: Received PUBCOMP from Subscriber1 (Mid: 1) 1490313015: Received PINGREQ from Subscriber1 1490313015: Sending PINGRESP to Subscriber1
Window 2:
pi@raspberrypi:~ $ mosquitto_sub -d -h localhost -q 2 -i Subscriber1 -t home/living-room/temperature Client Subscriber1 sending CONNECT Client Subscriber1 received CONNACK Client Subscriber1 sending SUBSCRIBE (Mid: 1, Topic: home/living-room/temperature, QoS: 2) Client Subscriber1 received SUBACK Subscribed (mid: 1): 2 Client Subscriber1 received PUBLISH (d0, q2, r0, m1, 'home/living-room/temperature', ... (12 bytes)) Client Subscriber1 sending PUBREC (Mid: 1) Client Subscriber1 received PUBREL (Mid: 1) Hello, QoS 2 Client Subscriber1 sending PUBCOMP (Mid: 1) Client Subscriber1 sending PINGREQ Client Subscriber1 received PINGRESP
Window 3:
pi@raspberrypi:~ $ mosquitto_pub -d -h localhost -q 2 -i Publisher1 -t home/living-room/temperature -m "Hello, QoS 2" Client Publisher1 sending CONNECT Client Publisher1 received CONNACK Client Publisher1 sending PUBLISH (d0, q2, r0, m1, 'home/living-room/temperature', ... (12 bytes)) Client Publisher1 received PUBREC (Mid: 1) Client Publisher1 sending PUBREL (Mid: 1) Client Publisher1 received PUBCOMP (Mid: 1) Client Publisher1 sending DISCONNECT
In QoS 2, publication and confirmation is a four step process. Publisher1 sends the PUBLISH message to the broker, which then sends PUBREC message back. Publisher1 receives PUBREC, and responds with PUBREL. The broker receives that PUBREL, and responds with PUBCOMP. Publisher1 receives PUBCOMP, thus ending the publish and confirmation process. The exact process is repeated when the message is sent from broker to Subscriber1.
Here is a complete list of the command messages involved in publishing messages under the various QoS levels:
PUBLISH - sent by a publisher when it publishes a message
PUBACK - sent in response to a PUBLISH message with QoS level 1
PUBREC - sent in response to a PUBLISH message with QoS level 2. This is the second message in QoS 2 protocol flow.
PUBREL - assured publish release, sent in response to a PUBREC message. Third message in the QoS 2 protocol flow
PUBCOMP - assured publish complete. Sent in response to a PUBREL message. It is the fourth and last message in the QoS 2 protocol flow.
This gives us the steps involved in sending the messages with each of the QoS levels.
Downgraded QoS Levels
As mentioned above, it is possible for the publishing client and the subscribed client connect to the broker with different QoS levels. For example, suppose Publisher1 - QoS 2 -> Broker, and Broker - QoS 0 -> Subscriber1. In that case, the QoS is said to be downgraded, meaning that a message sent to the broker in QoS 2 will then be sent to Subscriber1 as QoS 0.
Let's try an example of that:
Window 1: mosquitto -v
Window 2: mosquitto_sub -d -h localhost -q 0 -i Subscriber1 -t home/living-room/temperature
Window 3: mosquitto_pub -d -h localhost -q 2 -i Publisher1 -t home/living-room/temperature -m "Hello, mixed QoS level"
Window #1:
mosquitto -v 1490369141: mosquitto version 1.4.10 (build date 2017-02-19 02:05:51+0000) starting 1490369141: Using default config. 1490369141: Opening ipv6 listen socket on port 1883. 1490369141: Opening ipv4 listen socket on port 1883. 1490369167: New connection from ::1 on port 1883. 1490369167: New client connected from ::1 as Subscriber1 (c1, k60). 1490369167: Sending CONNACK to Subscriber1 (0, 0) 1490369167: Received SUBSCRIBE from Subscriber1 1490369167: home/living-room/temperature (QoS 0) 1490369167: Subscriber1 0 home/living-room/temperature 1490369167: Sending SUBACK to Subscriber1 1490369201: New connection from ::1 on port 1883. 1490369201: New client connected from ::1 as Publisher1 (c1, k60). 1490369201: Sending CONNACK to Publisher1 (0, 0) 1490369201: Received PUBLISH from Publisher1 (d0, q2, r0, m1, 'home/living-room/temperature', ... (22 bytes)) 1490369201: Sending PUBREC to Publisher1 (Mid: 1) 1490369201: Received PUBREL from Publisher1 (Mid: 1) 1490369201: Sending PUBCOMP to Publisher1 (Mid: 1) 1490369201: Sending PUBLISH to Subscriber1 (d0, q0, r0, m0, 'home/living-room/temperature', ... (22 bytes)) 1490369201: Received DISCONNECT from Publisher1 1490369201: Client Publisher1 disconnected. 1490369226: Received PINGREQ from Subscriber1 1490369226: Sending PINGRESP to Subscriber1
Window #2:
mosquitto_sub -d -h localhost -q 0 -i Subscriber1 -t home/living-room/temperature Client Subscriber1 sending CONNECT Client Subscriber1 received CONNACK Client Subscriber1 sending SUBSCRIBE (Mid: 1, Topic: home/living-room/temperature, QoS: 0) Client Subscriber1 received SUBACK Subscribed (mid: 1): 0 Client Subscriber1 received PUBLISH (d0, q0, r0, m0, 'home/living-room/temperature', ... (22 bytes)) Hello, mixed QoS level Client Subscriber1 sending PINGREQ Client Subscriber1 received PINGRESP
Window #3:
mosquitto_pub -d -h localhost -q 2 -i Publisher1 -t home/living-room/temperature -m "Hello, mixed QoS level" Client Publisher1 sending CONNECT Client Publisher1 received CONNACK Client Publisher1 sending PUBLISH (d0, q2, r0, m1, 'home/living-room/temperature', ... (22 bytes)) Client Publisher1 received PUBREC (Mid: 1) Client Publisher1 sending PUBREL (Mid: 1) Client Publisher1 received PUBCOMP (Mid: 1) Client Publisher1 sending DISCONNECT
As can be seen, Publisher1 and the broker exchange the following command messages: PUBLISH, PUBREC, PUBREL, PUBCOMP since they are connected using QoS 2.
Then, the broker sends PUBLISH to Subscriber1, and Subscriber1 doesn't send a response (nor is broker expecting a response). This is because they are connected using QoS 0. This is how the QoS level is downgraded.
Choosing Between QoS Levels
Here are some notes on when to choose each QoS level:
Use QoS 0 when:
- The connection between sender and receiver is very stable
- Occasional data loss is acceptable
Use QoS 1 when:
- Every message must be received
- Recipient of these messages can handle duplicates - either because there is special-purpose code to detect duplicates, or the operations performed by each message are idempotent
Use QoS 2 when:
- All messages must be received exactly once
- The extra time needed to perform the 4-step publish and confirm process is acceptable
No comments:
Post a Comment