Deploying HiveMQ Edge in multiple sites
Edge computing and IoT have a happy marriage. IoT focuses on devices at the edge and data collection. Edge computing enables local processing of that data. There is a range of protocols that are relevant like Modbus and OPC UA. HiveMQ provides a containerized bridge for most relevant IoT protocols.
In this post, I’ll look into how you can deploy HiveMQ Edge across multiple sites using the Avassa platform.
IoT at the Edge – how to get the data out
HiveMQ takes various protocols as input and bridges that to MQTT. This data is sometimes processed locally at the edge and/or forwarded to the cloud.
The steps we will show in this blog post are:
- How to deploy the containerized HiveMQ edge to a large number of edge sites
- How to configure that to collect edge local data and forward that to the cloud
HiveMQ Edge has a function to forward local topics to a remote MQTT Broker, they call this an “MQTT Bridge”. The screenshot below shows what it looks like in the HiveMQ Edge UI when bridging is configured to forward topics to the cloud/enterprise MQTT broker at 192.168.0.20:11883.
To simulate data coming in, we will use a simulated edge device, also built into HiveMQ Edge.
The rest of the post shows how you can deploy HiveMQ Edge at scale using your Avassa system.
Deploy HiveMQ to hundreds or thousands of sites
In the Control Tower, we first store the credentials to the cloud/enterprise MQTT broker in a vault. We also tie this vault to the deployment of HiveMQ Edge.
Next we define the HiveMQ container application, the main sections are:
- variables: pick up the Enterprise MQTT broker credentials from the distributed secret
- config-map: listen for data on the local port 1883, the standardized MQTT port. In this demo, 192.168.0.20:11883 is my Enterprise MQTT broker, i.e. to where we want to forward topics.
- containers: this will pick the HiveMQ container from the Docker Hub and pass the config-map as a mounted XML file.
name: hivemq-edge
version: "1.0"
services:
- name: hivemq-edge
mode: replicated
replicas: 1
variables:
# Get Enterprise MQTT broker credentials from Strongbox
- name: MQTT_USERNAME
value-from-vault-secret:
vault: hivemq-edge
secret: credentials
key: username
- name: MQTT_PASSWORD
value-from-vault-secret:
vault: hivemq-edge
secret: credentialsjjlj
key: password
volumes:
- name: log
ephemeral-volume:
size: 20MB
file-ownership: 1001:1001
- name: cfg
config-map:
items:
- name: default.conf
data: |
<?xml version="1.0"?>
<hivemq xmlns:xsi="<http://www.w3.org/2001/XMLSchema-instance>">
<!-- Add mqtt listener -->
<mqtt-listeners>
<tcp-listener>
<port>1883</port>
<bind-address>0.0.0.0</bind-address>
</tcp-listener>
</mqtt-listeners>
<!-- Create a simulation that publish on a topic called
simulation -->
<protocol-adapters>
<simulation>
<id>test-simulation-server</id>
<pollingIntervalMillis>5000</pollingIntervalMillis>
<subscriptions>
<subscription>
<destination>simulation</destination>
<qos>1</qos>
</subscription>
</subscriptions>
</simulation>
</protocol-adapters>
<mqtt-bridges>
<mqtt-bridge>
<!-- Make sure each bridge uses a unique name -->
<id>${SYS_SITE}-remote-mqtt-broker</id>
<!-- forward to broker with authentication -->
<remote-broker>
<host>192.168.0.20</host>
<port>11883</port>
<authentication>
<mqtt-simple-authentication>
<username>${MQTT_USERNAME}</username>
<password>${MQTT_PASSWORD}</password>
</mqtt-simple-authentication>
</authentication>
</remote-broker>
<forwarded-topics>
<forwarded-topic>
<!-- Only forward topics from the simulation -->
<filters>
<mqtt-topic-filter>simulation/#</mqtt-topic-filter>
</filters>
<!-- prefix each topic with the name of the site -->
<destination>${SYS_SITE}/{#}</destination>
</forwarded-topic>
</forwarded-topics>
</mqtt-bridge>
</mqtt-bridges>
<admin-api>
<enabled>true</enabled>
</admin-api>
</hivemq>
network:
ingress-ip-per-instance:
protocols:
- name: tcp
# 1883: MQTT
# 2442: MQTT-SN
# 8080: HiveMQ Edge WebUI
port-ranges: "1883,2442,8080"
outbound-access:
allow-all: true
containers:
- name: hivemq-edge
image: registry-1.docker.io/hivemq/hivemq-edge
mounts:
# Mount the config file in the container
- volume-name: cfg
files:
- name: default.conf
mount-path: /opt/hivemq/conf/config.xml
# Mount the log volume in the container
- volume-name: log
mount-path: /opt/hivemq/log
For a reference on the HiveMQ Edge configuration, please see https://github.com/hivemq/hivemq-edge/wiki.
On my Mosquitto MQTT server at 192.168.0.20, I can shortly see the following logs:
1693396113: Received PUBLISH from gbg-remote-mqtt-broker (d0, q0, r0, m0, 'gbg/simulation', ... (54 bytes))
1693396113: Received PUBLISH from sthlm-remote-mqtt-broker (d0, q0, r0, m0, 'sthlm/simulation', ... (54 bytes))
1693396114: Received PUBLISH from jkpg-remote-mqtt-broker (d0, q0, r0, m0, 'jkpg/simulation', ... (54 bytes))
This indicates the HiveMQ Edge is up and running on all sites, posting to unique topics.
With very few steps, we can bring up HiveMQ Edge at an arbitrary number of sites and have them connect to a central MQTT broker in the cloud or anywhere else.
In this post, we have shown how Avassa can be your platform for deploying edge IoT applications including handling relevant IoT protocols.