Waterstream version 1.3.0 introduces MQTT version 5.0 support, alongside multiple improvements – including better metadata, easier debugging, more lightweight communication, scaling out reads capability, and much more.
Let’s take a look what Waterstream users can gain from switching to MQTT version 5.0:
Message properties
Somewhat similar to HTTP headers, message properties contain metadata, such as broker feature support, content type, error messages, etc. There are also free-form user properties that can contain application-specific metadata. A lot of other features depend on the message properties.
Improved session expiration control with the Clean Start flag and the Session Expiry Interval property
Previously, if you wanted to start a fresh persistent session, you had to connect with Clean Session flag set to true, disconnect, and then connect again with Clean Session set to false, because the same flag determined both whether the session should be persistent and whether it should be fresh. Now, these responsibilities are separated between the Clean Start flag (which replaces Clean Session) and the Session Expiry Interval property.
Reason codes on all ACK messages
CONNACK, PUBACK, PUBREC, PUBREL, PUBCOMP, SUBACK, UNSUBACK, DISCONNECT. The previous version had more limited return codes in some messages (such as CONNACK and SUBACK), the reason codes now more precisely indicate the reason for the error. Note that numeric values for the CONNACK v 5.0 reason code differs from the v 3.1.1 return codes.
Clients are now notified why the server has closed the connection
In case of technical difficulties or protocol violation by the client, the broker now sends DISCONNECT with a corresponding reason code instead of just dropping the connection.
Payload metadata
Whenever a client sends a PUBLISH message, the following properties are persisted together with the message (under the hood using Kafka message headers):
- Payload Format Indicator (binary or text)
- Content Type (free-form string – for example, MIME type)
- User Properties (free-form string key-value pairs, the key may be repeated)
- Response Topic (name of the topic where the client expects the response to this message)
- Correlation Data (arbitrary bytes array used for tracking the message)
The client that receives the PUBLISH message as a result of the subscription gets these properties as well, with the same values sent by the publishing client.
Scaling out the reading with the shared subscriptions
This feature is somewhat similar to Kafka consumer groups. Multiple clients may subscribe to the pattern of the special format: $share/{ShareName}/{filter} and as long as their ShareName and filter are the same, they’ll get the messages distributed among the clients so that each message gets delivered to one client. To make the distribution of the messages work properly the Kafka topic that stores the MQTT messages must have at least as many partitions as the number of Waterstream nodes you have – clients connected to different nodes rely on Kafka partitions for load-balancing. Clients connected to the single Waterstream node get balanced by Waterstream itself, they don’t rely on Kafka partitions.
Relation between subscriptions and messages with Subscription ID
The SUBSCRIBE packet may include an ID property which is included in the PUBLISH messages delivered as a result of this subscription. For example, clients may use it to invoke the proper callbacks upon receiving the message.
Reduce traffic for long topic names by using Topic Alias
The side that sends the PUBLISH message may decide to replace the topic name with a numeric code. If there are a lot of messages on a topic with a long name this can substantially reduce the traffic. If the client decides to use an alias it can just set the Topic Alias property. Waterstream can be configured to use the alias in messages it sends to clients – use the MQTT_ALIASED_TOPIC_PATTERN environment variable to specify the pattern of topics to which the alias may apply, and MQTT_ALISED_TOPIC_MIN_LENGTH to specify the minimal topic length to which the alias applies.
Server and client may tell each other about their limits and the supported features
For example, Maximum Packet Size, Receive Maximum (how many unconfirmed messages the server can hold), Topic Alias Maximum (the maximum value for topic alias), Shared Subscriptions Available (if shared subscriptions are available – always 1 for Waterstream), etc.
Fine-tune subscription with Subscription Options.
- The No Local option can turn off the sending back of messages to the same client that sent them ( this helps to prevent message loops in bridge mode).
- Retain As Published lets you know the original Retain flag, even if the message was delivered as a part of normal workflow, not as a retained message.
- Retain Handling option – determines whether the retained messages should be sent at all.
Last Will delay
It is now possible to skip unnecessary Last Will if the client reconnects shortly after going down.
Assigned Client Identifier can now be reported back to the client
If the client chooses to let the server generate Client ID, the server now reports generated ID back to the client, thus making it possible to re-connect and keep the session.
We wish you a happy journey into the world of MQTT 5 with Waterstream and all of these exciting new features!
