MQTT with Intermittent Networks Connections
MQTT is a very common protocol in IoT scenarios. It’s a lightweight protocol that is widely supported in most corners of this domain. In this article, we take a closer look at how to cache MQTT messages in the the event of a connectivity outage. We will use Avassa’s built-in pub/sub bus, Volga, as the cache.
Fear not, little IoT devices! MQTT can be cached at the edge
We often see setups with data-producing devices, commonly IoT devices, among our user base. In the event of a flakey connection between the data producer and the MQTT broker, it’s up to the client to buffer messages. Well, that’s much easier said than done when dealing with small embedded clients. But fear not, it can be done. Let’s dig into it.
How to cache MQTT messages in case of network outage
Let’s start by looking at an overview of an often seen MQTT structure. The diagram below depicts a common setup where IoT devices send messages to a broker in the cloud.
If for some reason, the connection to the broker goes down, messages are typically lost. This can happen for several reasons, e.g. the site is disconnected from the Internet/Cloud. In certain cases this may be fine, messages are lost and when the connection is restored, new messages will be passed. On the other hand, if messages can’t be lost, it’s now up to the IoT devices to buffer messages until the connection is restored. This puts the burden of buffering onto each and every device.
The idea here is to instead put an MQTT bridge at the site itself. The bridge caches MQTT messages and passes them on to the Cloud MQTT broker when the connection is available.
In case of connectivity loss, messages are cached locally and can be forwarded to the broker when the connection is restored.
The bridge is really simple, a broker that the IoT devices connect to, and messages are stored in the Avassa built-in pub/sub bus, i.e. Volga, topic. A Volga consumer picks messages from Volga and sends them to the cloud broker using an MQTT client.
As we have discussed in the article, if messages can not be lost, some sort of caching needs to happen at the local site. In some cases, it may be possible to cache in the IoT devices, but in many cases, this will be hard. Entering: our MQTT bridge/cache solution. A solution like this can also be further developed, by providing data refinement in before the MQTT messages are passed to the broker. Filtering and de-duplication are a few that comes to mind.
Keep reading: