November 2022: Feature releases & highlights

The November releases include several important new features, but one is simply outstanding: the enhanced edge site cluster management. We have removed any manual management activities to manage hosts coming and going on the site.

• It just works!
• User interface for searching Control Tower pub/sub bus topics
• Container log archiving
• Maintenance windows for Edge Enforcer upgrades
• Tabular output in the Avassa command line
• Tolkien’s words of wisdom

It just works!

Edge sites are not static; planned and unplanned things happen. Irrespective of Murphy or the local IT team touching the hosts, you would like the site to work without manual system management activities.

We have hardened the local site cluster management to run autonomously without manual intervention when hosts are removed and added. The local cluster will automatically reassign RAFT members, move topics on the Avassa pub/sub bus and other tasks you have to deal with manually.

💡 The edge site just works!

User interface for searching Control Tower pub/sub bus topics

The Avassa system generates valuable metrics and logs using the built-in pub/sub topics. The data covers hosts, sites, the Avassa platform, and your applications. You can access this over the APIs and the Avassa command line. We have a dedicated tutorial for using the supctl command line and searching logs/metrics. Lately, we have enhanced the UI support for this. In our previous release highlights, we did show how you can search on a specific site. We have now added support in the UI for searching the central logs and metrics stored in Control Tower.

We have deliberately made design decisions on what is kept on the sites or distributed to Control Tower. The container logs, for example, are kept on the sites rather than sent to Control Tower. You can send a distributed query to all sites, but let’s move on to logs and metrics in Control Tower:

As you can see in the screenshot above, there is a new menu item to the left Topics. By selecting that you can search topics managed by the control tower. Below follows a brief introduction to the topics. More details in our topic documentation.

Generated and stored in Control Tower

• connection-status: a new event every time the connection status to a site is changed
• deployment-events: any events related to changes in deployments. This will in turn result in scheduler-events on the sites.
• logs: a system log with relevant trouble-shooting events

Generated on sites and distributed to Control Tower

• notifications: alerts related to applications across all sites
• site-alerts: for each site, Avassa monitors host resources and state in order to alert if there is an issue. These are available on the edge site but also distributed to Control Tower.
• audit-trail-log: Control Tower has a union of all audit trail logs from the sites.
• unauthenticated-audit-trail-log: log of unauthenticated events like login attempt

All these Control Tower topics can now easily be searched and inspected in the User Interface.

Container log archiving

Container logs are stored on the site in a topic per running container. It contains all output from standard output and standard error from the container. The corresponding topics are named:

system:container-logs****:****APPLICATIONNAME.SERVICENAME-IX.CONTAINERNAME

where IX is a numeric index, one per service instance (replica).

If a container is rescheduled to another host in the cluster a new topic is created with the same name. This resulted in you could not view the container log from the previous host. We have now introduced archiving of pub/sub bus topics. In this case, the log from the previous host is archived with a date and time suffix at the topic name like:

system:container-logs:popcorn.popcorn-service-1.popper-container.20221125T130615Z

By default, archived logs are retained for 14 days, this can of course be overridden per application.

Maintenance windows for Edge Enforcer upgrades

Avassa automatically manages the upgrades of the Edge Enforcer on all sites. the Control Tower fetches new production release when available and requests all sites to upgrade themselves. Running container applications that have been orchestrated by Avassa are not affected by the upgrade. But during the upgrade (less than a minute typically) the Avassa User Interface, Avassa APIs and the Avassa DNS will be unreachable for that site. You can now control maintenance windows per site to define when the upgrade shall occur.

Maintenance windows are configurable by the site provider and indicates the time period where the upgrade is allowed. And it can be controlled per site. Sites without any maintenance window will be immediately upgraded.

The first step is to create a profile:

$ supctl create system site-profiles <<EOF
name: sweden
maintenance-windows:
  - days-of-week: Monday, Tuesday, Wednesday, Thursday, Friday
    start-time: 02:00
    timezone: site-local
    duration: 3h
  - days-of-week: Saturday, Sunday
    start-time: 00:00
    timezone: site-local
    duration: 24h
EOF

The above profile allows updates 2 AM during weekdays and anytime during weekends.

You can now associate the profile to the sites where you want to control Edge Enforcer upgrades

$ supctl merge system sites stockholm-sergel <<EOF
site-profiles:
  - sweden
EOF
$ supctl show system sites stockholm-sergel --fields site-profiles
site-profiles:
  - sweden

Tabular output in the Avassa command line

Tabular output in the Avassa command line, supctl, is a useful option to display repeated data. For example, if you would like to see a site status with some selected fields:

$ supctl --table show system sites stockholm-sture --fields type,location,domain
type   latitude            longitude           domain                                
----   -----------------   -----------------   --------------------------------------
edge   59.33898823699348   18.06780218371588   stockholm-sture.wallan.edge.avassa.dev

The tabular view uses the field as column header, skipping the name structure before. Look at the following example, here you have two fields name referring to tenant name and application name:

$ supctl show system sites stockholm-sture allocations --fields 'tenants/[name,applications/name]'                 
tenants:
  - name: tenant1
    applications:
      - name: popper
  - name: tenant2
    applications:
      - name: popcorn

If you present this in tabular output you will loose the name context

% supctl -T show system sites stockholm-sture allocations --fields 'tenants/[name,applications/name]'                 
name      name   
-------   -------
tenant1   popper 
tenant2   popcorn

We have added the feature to rename fields according to the following example. When doing a field selection you can rename the field by using an equal sign and the desired name

 --fields 'tenants/[name=tenant,applications/name=application]'

And a complete example looks like below:

% supctl -T show system sites stockholm-sture allocations --fields 'tenants/[name=tenant,applications/name=application]'
tenant    application
-------   -----------
tenant1   popper     
tenant2   popcorn

💡 This works as well for YAML and JSON output.

Thats all for November. Our vision and roadmap is clear: make life at the edge easier!

“You step into the road, and if you don’t keep your feet, there is no knowing where you might be swept off to.”  

J.R.R. Tolkien