August 2024: Feature releases & highlights

Site admin UI, Webhooks for pub/sub topics

The August release features include the ability to add functionality to your edge sites using the Avassa APIs. We provide two applications that add support for a site-local Web UI and Webhooks.

• Source code for site UI: You might want to provide an edge site local Web UI for the users on the shop floor, in the retail store, in the truck, etc. This can then be used to manage the edge applications locally when the connection to the central Control Tower is down. We provide a rich code example that you can use as a base for your specific needs.
• Webhooks for topics: The Avassa built-in pub/sub bus has valuable topics like alerts and audit logs. We now provide a way to integrate those into your favorite tool, like Chat or Splunk, using Webhooks.

Since the features are deployed as applications, the descriptions below are twofold: They explain and illustrate application specification snippets that make the application run and its end-purpose functionality.

Source Code for an Edge Site UI

The Avassa edge sites are fully autonomous; all control functionality executes at the site. This means that restarts and fail-over scenarios are not dependent on connection to the central component. Container images, secrets, and configurations are cached at the site and replicated amongst the edge hosts.

During normal operations, operational activities and configuration changes, such as application upgrades, are driven from the Control Tower. But when connectivity is down, you must be able to perform tasks locally at the site. You can perform read-only operations such as checking application status, reading logs, and performing application restarts. You can also perform local upgrades and configuration changes. These will be reconciled in a controlled way when the connection is re-established. The changes are kept local as long as no central actions are taken; any later central changes will overwrite the edge local changes. The local changes are clearly shown in the Control Tower UI. Read more about this in a previous release highlights. Local changes can be performed using the Avassa command line tool supctl and the edge local rest API. Read more on managing offline sites in our documentation.

However, there could be many reasons why you would like to provide a simple operational UI at the edge site. Users at the edge sites usually are not advanced-level IT staff, which requires a simple and limited UI to target specific needs. On top of that, it is most likely a combination of Avassa functions like checking logs, application state, restarts, and selected functions in the specific edge application that you want to expose in such a UI.

🆕 To support these needs, we have published a rich source code example illustrating the most commonly used operations at the edge site.

The example prioritizes clarity when using the Avassa REST edge API over advanced styling and client code. With this example as a base, you can quickly build and style your specific site UI.

The site UI source code is available at https://gitlab.com/avassa-public/site-admin-ui

You can quickly test to deploy the raw example to a site with a prebuilt container using the provided application specification:

name: site-admin-ui
services:
  - name: site-admin
    mode: replicated
    replicas: 1
    volumes:
      - name: cert
        vault-secret:
          vault: site-admin-ui
          secret: cert
          file-mode: "400"
          file-ownership: 1000:1000
    share-pid-namespace: false
    containers:
      - name: site-admin
        mounts:
          - volume-name: cert
            mount-path: /home/node/app/certs
            mode: read-only
        container-log-size: 100 MB
        container-log-archive: false
        image: registry.gitlab.com/avassa-public/site-admin-ui/site-admin-ui
        env:
          API_CA_CERT: ${SYS_API_CA_CERT}
        on-mounted-file-change:
          restart: true
    network:
      ingress-ip-per-instance:
        protocols:
          - name: tcp
            port-ranges: "10000"
        access:
          allow-all: true
on-mutable-variable-change: restart-service-instance

The application specification illustrates the use of certificates and ingress networking, which is elaborated on below.

The Avassa built-in CA is used to generate certificates:

name: site-admin-ui
ttl: 3y
cert-key-type: ecdsa
auto-renew: 
  renew-threshold: 30d
distribute:
  deployments:
    - site-admin-ui

This cert is then provided to the application through the secrets manager:

name: cert
auto-cert:
	issuing-ca: site-admin-ui
	ttl: 30d
	truncate-ttl: false
	host: site-admin-ui
	cert-type: server
allow-image-access:

And https is enabled when initiating the web server:

const httpsAgent = new https.Agent({
	ca: process.env.API_CA_CERT,
})

Since the site UI is local to the edge, web clients need to be able to reach the node.js web server and this is achieved by the ingress network configuration, tcp/10000. This assumes you have enabled ingress on the corresponding sites.

A final note on deploying the edge site UI: What about users and authentication at the edge site? The most straightforward way is to use Avassa’s built-in user management features. Any local users you create in the Control Tower are distributed to the edge sites. This means that you can authenticate locally without a central connection. (Note: the initial user you got when registering for Avassa in the cloud account is an external user and can not be used since it assumes connectivity to the OATH server.)

In the screenshot below, you can see that the site UI application is running at home-site. By navigating to the service instance, you can see the ingress address for the web application:

By pointing your browser to https://192.168.66.3:10000/ you will get to the web UI. After (site local) login, you can for example, view the topics at the site:

Once you get the example to run at sites, you can reuse the source code and build what you want. There are many example in the source code on how to use the REST API in order to: list applications, restart, look at logs, view topics etc. (The not secure notice is because a self-signed certificate is used).

The repository contains a Docker file as a starting point for your build.

Webhooks for topics

The Avassa Edge Platform provides an edge native pub/sub bus. The Avassa Edge Platform publishes a number of topics, and application developers can create their own topics. A topic is unique to a site and can reside on either the Control Tower or edge sites. The Control Tower provides relevant topics from a central point of view, such as deployment events, authentication logs, and alerts propagated from sites. Each site has topics related to that particular site, like scheduler events, host metrics, and container logs. You can easily list topics and view their contents. (See more details on local/parent below)

Control Tower topics:

Site-specific topics:

For more information on topics, see the reference guide. You can publish and subscribe / view the topics through Control Tower Web UI, the command line supctl, the REST API and Web Socket API. We also provide a tutorial on how to programmatically access the topics.

Many log and alerting clients and tools offer a Webhook URL for lightweight integration.

🆕 We have now added an application that lets you subscribe to topics from Avassa using Webhooks.

The capability is provided as a stand-alone application that you deploy to the sites from which you want to subscribe. It is available at https://gitlab.com/avassa-public/volga-webhooks

This is ready to be deployed using the provided example application specification:

name: volga-Webhooks-app
services:
  - name: volga-Webhooks-srv
    containers:
      - name: volga-Webhooks
        image: registry.gitlab.com/avassa-public/volga-webhooks
        approle: volga-Webhooks
        env:
          API_CA_CERT: "${SYS_API_CA_CERT}"
          APPROLE_SECRET_ID: "${SYS_APPROLE_SECRET_ID}"
        mounts:
          - volume-name: volga-Webhooks-conf
            files:
              - name: volga-Webhooks.conf
                mount-path: /volga-Webhooks/volga-Webhooks.conf
    network:
      outbound-access:
        allow-all: true
    variables:
      - name: TOKEN
        value-from-vault-secret:
          vault: volga-Webhooks
          secret: myWebhook
          key: token
    volumes:
		  - name: volga-Webhooks-conf
		  ...

The container section points to the prebuilt container Webhook application. To access Avassa APIs, users and applications must be authenticated. The env section above provides the API certificate and the approle secret id, to authenticate as an application. Approles correspond to user authentication but only for applications. You can read more on approles in the developer tutorial.

Then a volume volga-Webhooks-conf is mounted for the configuration data, controlling the Webhook’s behavior, see more below. The network section is important since it needs to allow for outgoing network connections, the actual Webhooks URLs.

That was the plumbing, now lets continue with the fun part, how to configure the Webhooks.

name: volga-Webhooks-app
		...

    volumes:
      - name: volga-Webhooks-conf
        config-map:
          items:
            - name: volga-Webhooks.conf
              data: |
                Webhooks:
                  - name: simple-Webhook
                    topic: my-topic
                    url: "<https://hooks.myWebhook.com/asdfghjkl>"
                    data-template: "{\"text\": \"{{ msg['payload'] }}\"}"
                  - name: more-interesting-Webhook
                    location: local
                    topic: system:audit-trail-log
                    url: "<https://hooks.myWebhook.com/asdfghjkl>"
                    headers:
                      - key: Authorization
                        value: Bearer ${TOKEN}
                    data-template: "{\"text\": \"User {{ user }} accessed {{ path }} at {{ time }}\"}"
                    variables:
                      - name: user
                        path: "payload.user"
                      - name: path
                        path: "payload.path"
                      - name: time
                        path: "payload.occurred-at"
    mode: replicated
    replicas: 1

The configuration is a one-to-one mapping between Avassa topics and Webhook URLs. The simplest possible configuration is just a topic name and a URL, which will result in the payload of each message on the topic being posted to the URL. If the Webhook requires a particular message format, a Jinja2 template can render the message. Headers can also be specified. A regular expression can be provided to filter incoming messages on the server side. Read more in the README file in the repository.

The location variable can be set to local or parent. Local means to listen to topics on the site where the application is deployed. You can use parent to listen to Control Tower topics from a site since there is no way to deploy your application to run in the Control Tower.

Sun RPC 1984, Corba 1991, XML RPC 1998, Webhooks 2007… read the original blog post from Jeff Lindsay on the idea of web hooks.