How to place applications based on edge hardware devices

The Avassa System supports mounting devices into applications. Devices in this context are (typically) hardware that is attached to a host in an Avassa site. Examples are USB devices, e.g. video cameras, temperature sensors etc. The only requirement is that the device is discoverable by udev, the Linux device manager.

The principle for making this happen is the following:

• As an application developer you require devices by matching against device labels in your application specification. (Step 2 below)
• As a site provider you configure device discovery rules on the site that generates device labels. (Step 3 below)

In this article, we will walk through how to set up device labels that matches (virtual) cameras attached to a Raspberry Pi. To simulate attached video cameras, we will use video4linux. Video4linux comes with a kernel module (v4l2loopback) that creates video devices.

Step 1: Preparing the simulated devices

This section has nothing to do with the Avassa system as such. This section describes a way to get simulated camera devices on your hosts.

First thing we’ll do is to initiate two cameras that we’ll name entry camera and kitchen camera

$ sudo modprobe v4l2loopback video_nr=3,4 card_label="entry camera","kitchen camera"

Now we can use udevadm to inspect these two:

$ udevadm info -ap /devices/virtual/video4linux/video3

  looking at device '/devices/virtual/video4linux/video3':
    KERNEL=="video3"
    SUBSYSTEM=="video4linux"
    DRIVER==""
    ATTR{buffers}=="2"
    ATTR{dev_debug}=="0"
    ATTR{format}==""
    ATTR{index}=="0"
    ATTR{max_openers}=="10"
    ATTR{name}=="entry camera"
    ATTR{power/control}=="auto"
    ATTR{power/runtime_active_time}=="0"
    ATTR{power/runtime_status}=="unsupported"
    ATTR{power/runtime_suspended_time}=="0"

$ udevadm info -ap /devices/virtual/video4linux/video4

  looking at device '/devices/virtual/video4linux/video4':
    KERNEL=="video4"
    SUBSYSTEM=="video4linux"
    DRIVER==""
    ATTR{buffers}=="2"
    ATTR{dev_debug}=="0"
    ATTR{format}==""
    ATTR{index}=="0"
    ATTR{max_openers}=="10"
    ATTR{name}=="kitchen camera"
    ATTR{power/control}=="auto"
    ATTR{power/runtime_active_time}=="0"
    ATTR{power/runtime_status}=="unsupported"
    ATTR{power/runtime_suspended_time}=="0"

Step 2: Creating device labels

Now we act as the site provider that set up device rules on the site.

Looking at the device attributes above can see that by creating udev rules that matches on SUBSYSTEM and ATTR{name} we can uniquely identity these two cameras.

Let’s add two match rules to our site configuration

name: restaurant
type: edge
device-labels:
  - label: kitchen-camera
    udev-patterns:
      - SUBSYSTEM=="video4linux",ATTR{name}=="kitchen camera"
  - label: entry-camera
    udev-patterns:
      - SUBSYSTEM=="video4linux",ATTR{name}=="entry camera"
...

The above rules will generate the labels kitchen-camera and entry-camera on hosts when these devices are discovered. Generated device labels are automatically propagated to the site object.

supctl show --site restaurant assigned-sites restaurant

name: restaurant
type: edge
labels:
  system/type: edge
  system/name: restaurant
host-labels: {}
volume-labels: {}
device-labels:
  kitchen-camera: /dev/video4
  entry-camera: /dev/video3
network-interface-labels: {}

And let’s inspect the hosts on the site:

$ supctl show --site restaurant system cluster hosts

- cluster-hostname: restaurant-001
 ...
  device-labels:
    kitchen-camera: /dev/video4
    entry-camera: /dev/video3
  devices:
    - name: /dev/video4
      labels:
        kitchen-camera: /dev/video4
    - name: /dev/video3
      labels:
        entry-camera: /dev/video3
 ...

Here we can see that the Edge Enforcer has successfully identified our kitchen and entry cameras.

Step 3: Requiring access to a camera in your application

Let’s create an application that asks for e.g. the entry camera

$ supctl show -c application entry-monitor

name: entry-monitor
version: "1.0"
services:
  - name: monitor-svc
    mode: replicated
    replicas: 1
    share-pid-namespace: false
    containers:
      - name: monitor
        image: registry-1.docker.io/alpine
        cmd:
          - sleep
          - infinite
        container-log-size: 100 MB
        devices:
          device-labels:
            - entry-camera
        on-mounted-file-change:
          restart: true

In this application specification, we request access to entry-camera . Note well the abstraction: as an application developer you just require a camera. When deploying this application the Edge Enforcer on the site will make sure that the application is scheduled to the host where the camera is connected. Furthermore, the deployment engineer does not need to have knowledge of which hosts provide which devices.

After deploying this application and “execing” into the container, we can see that the device is mounted into the device tree and hence available to the application.

supctl do --site restaurant applications entry-monitor service-instances monitor-svc-1 containers monitor exec-interactive sh
/ # ls /dev/
core     full     null     pts      shm      stdin    tty      video3
fd       mqueue   ptmx     random   stderr   stdout   urandom  zero
/ #

With label matching for devices, users can perform targeted deployments not only based on site- and/or host-specific circumstances (typical examples include geographical location, installed hardware or OS requirements) but also based on which devices are available on the edge hosts.