Skip to content
Back to Articles
Articles

Labels, groups and blast radius: organizing a fleet before it organizes you

Stefan Wallin 10 minute read
labels in avassa

As a fleet of edge sites begin to scale, it doesn’t stay simple for long. What starts as a handful of pilot locations turns into dozens, then hundreds, then thousands — spread across stores, factories, vehicles, or robots, each with its own mix of hardware. At some point, every team running that fleet hits the same question: how do you decide, reliably and repeatedly, where an application should run?

It sounds like one question. It’s actually two.

The problem: one question, two very different scopes

The first question is which sites should run a given application. A checkout-analytics service might belong on every store in the Nordics; a firmware-update agent might belong everywhere globally; a heavy inference workload might only belong on sites with sufficient compute. That’s a fleet-wide targeting decision, and it’s the one deployment teams think about most.

The second question only shows up once you’ve answered the first: within a site, which host should actually run the workload? Sites aren’t single machines anymore; they’re small clusters of hosts, some with a GPU, some attached to a camera or another leaf device, some just general-purpose compute. Somebody, or something, has to match the application to the right box.

The trouble starts when teams treat both questions as the same problem, solved the same way: with an inventory. A spreadsheet of site names. A list of hostnames and IP addresses. A separate note somewhere about which machines happen to have a GPU installed. That approach can work at ten sites. It quietly breaks down at a thousand, because now every hardware change — a swapped host, a GPU added to a kiosk, a new camera on a production line — has to be reflected back up into a deployment config that some engineer maintains by hand, often for equipment they’ve never seen and infrastructure they don’t own.

Why it matters

Edge fleets have three properties that make this harder than it looks, and less forgiving when it goes wrong.

Scale

Scale changes the math. A data center rollout might touch a few dozen nodes behind a load balancer; an edge rollout might touch a thousand independent sites, most of them unattended. A bad deployment doesn’t fail in one place where an on-call engineer can SSH in and look. Instead, it fails everywhere at once, in locations that may not have anyone on-site at all.

Heterogeneous hardware

Hardware is heterogeneous, and inventory drifts. Not every site has the same devices, and the devices a site has today aren’t guaranteed to be the ones it has in six months. Hardware gets added, replaced, and moved between machines. Any system that depends on a hand-maintained map of “which host has the GPU” is a system that will eventually be wrong, usually just when it matters.

Separation of concern

And the two teams that care about these questions are often different teams. The people deciding where an application should run — which markets, which store formats, which customer tiers — are thinking about the business logic of the fleet. The people who know what hardware sits in a given rack are thinking about physical infrastructure. Forcing the first team to maintain an inventory that’s really the second team’s job is where the friction comes from, and it’s exactly the kind of coupling that turns a five-minute deployment into a change-management exercise.

The result, when this isn’t handled well, is a rollout that’s either too conservative — nothing ships without a manual review of every site — or too reckless: one bad configuration change, pushed everywhere simultaneously, with no way to catch it before it’s already broken a thousand locations. The real goal is to make the intended blast radius of any given change a deliberate choice, not an accident of how the deployment happened to be scripted.

How Avassa solves it: two independent dimensions

Avassa treats “which sites” and “which host within a site” as genuinely separate problems, solved at different scopes, by different mechanisms so that no one has to maintain the inventory that would otherwise glue them together.

Dimension 1: Site labels decide the intended set of sites

Every site carries labels — a mix of built-in ones (system/name, system/type) and whatever custom labels the organization defines, like country = sweden or tier = flagship. A deployment targets sites with a boolean label expression: run this application everywhere system/type = edge and country = sweden, for instance. The deployment team writes label expressions — which kind of site should have this — not a list of hostnames. Nobody has to enumerate every store in Sweden by name, and nobody has to update the deployment when a new store opens; it just needs the right labels.

That label expression only defines the set of sites. It says nothing about which host inside any of those sites will actually run the container, and that’s deliberate — it’s the second dimension, below.

Because a label expression can match a thousand sites at once, Avassa gives the deployment team two independent levers to control how much of the fleet is exposed to a change at any given moment:

Canary sites let you deploy to a small, explicitly labeled subset first — a lab site, a handful of low-risk locations — and hold there. Promotion to the rest of the fleet can be automatic after the canary sites stay healthy for a defined window, or gated behind a manual go/no-go.

Rolling deployment then governs how the remaining sites are rolled out: a defined number of sites in parallel, with a wait time between batches to confirm health before moving on to the next.

If a canary or a rolling batch fails its health check, the deployment stops there — it does not continue rolling out to the rest of the fleet. That gives the team a natural checkpoint to roll back, investigate what went wrong, and retry, instead of discovering the problem only after it has already reached every site.

Dimension 2: within a site, placement is local and automatic

Once a site is in the target set, deciding which host inside that site’s cluster runs the workload is handled locally, by the site itself — not by the deployment team, and not by anything written into the deployment config.

For hardware-dependent workloads, Avassa uses discovery patterns: rules that scan hosts for specific hardware — a particular GPU model, a camera, a sensor — and apply a dynamic label to any host where a match is found. An application simply declares the device label it needs; the site-local scheduler finds a host carrying that label and places the workload there. If a GPU is swapped into a different machine next month, the label moves with the hardware automatically, and the application keeps landing in the right place without anyone touching the deployment.

For everything else, hosts can carry configured labels set by whoever manages the site’s infrastructure — a role, a zone, a rack position. An application can request a replica on every host matching a given label, which is how you express something like “run one instance of this agent on each edge node in the cluster” without naming a single host explicitly.

Either way, the deployment team never needs an inventory of hostnames, IP addresses, or which specific machine happens to have a GPU installed. That knowledge lives where it belongs — at the site, discovered and matched continuously — instead of being duplicated into a spreadsheet that immediately starts going stale.

(https://avassaio.sharepoint.com/:p:/s/AvassaSystemsAB/IQBgGoFYFzvuR6WK4lmUafvQAc5uXibIHGrT10e_xHGjsrQ?e=d1Cw1p)

Diagram showing two dimensions of fleet placement: fleet-wide targeting across sites using labels and rollouts, and site-local host placement via device discovery.

Best Practices for How to Configure Labels

Labels are only as useful as the thinking behind them. The practices below come from fleets that have grown from a handful of sites to thousands.

1. Generate site labels from a single source of truth.

Site labels should come from the system that already owns the facts about your sites, usually an inventory or CMDB, and be synced into Avassa from there. Don’t let them be edited by hand, one site at a time. If a store changes format or a site moves to a new region, update the source system and let the labels follow. Hand-maintained labels drift for the same reasons hand-maintained inventories do. Host and device labels are different: discovery patterns create them from the hardware that’s actually plugged in, so they don’t need a source system at all.

2. Let site labels describe what a site provides, not just what it is.

The main purpose of a site label is to be a deployment target, so design labels with that in mind. Alongside descriptive labels like country = sweden, define labels for the capabilities a site is expected to offer, such as vision-inspection, edge-ai or self-checkout. Each of those labels points straight to the applications that should run there. When a site gains a new capability, adding the label is all it takes to bring the right applications to it.

3. Treat site labels as a contract between site provider and application teams.

Application teams deploying from their own tenants don’t see the physical infrastructure underneath, and they shouldn’t need to. They deploy against labels. As the site provider, the labels you assign are a promise: every site labeled edge-ai really has the compute that label implies. Keep that promise and you can replace hardware, add sites or reorganize infrastructure without touching anyone’s deployments.

4. Prefer device labels over configured host labels for hardware.

If a workload needs a GPU or a camera, have it request the device label that discovery creates, not a host label someone configured by hand. Device labels follow the hardware when it moves; configured host labels keep pointing at the old machine. Save configured host labels for things discovery can’t detect, such as a host’s role, zone or rack position.

5. Plan for filtering and search, but as a secondary use.

Once the fleet reaches hundreds of sites, labels are also how operators find things: every flagship store in Sweden, every site in a given customer’s estate. That’s valuable, but deployment targeting should drive label design. A label that exists only to make a dashboard look nicer isn’t worth much.

6. Design labels as independent dimensions.

Before creating labels, decide which dimensions you’ll organize along: typically customer, region, site kind and capability. Keep each one as its own label (customer = acme, region = nordics, tier = flagship) instead of encoding several into one value like acme-nordics-flagship. Separate dimensions combine freely in label expressions, and that’s what makes blast radius something you choose: roll out to one region, one site kind or one customer first. The same goes for canary sites. A dedicated label such as canary = true makes the canary set explicit and easy to change.

Summary

Treating fleet placement as a single inventory problem is what makes it fragile: it forces one team to track the hardware details of another, and it gives every rollout an all-or-nothing blast radius by default. Splitting it into two dimensions removes both problems at once. Site labels, canary deployments, and rolling rollout let the deployment team express which sites should run an application and how cautiously to get there, with an automatic stop the moment something looks wrong. Discovery patterns and host labels let each site work out which host should run it, based on what’s actually plugged in right now, with no hand-maintained inventory required.

Organize the fleet around labels and groups instead of inventories, and it keeps organizing itself as it grows — rather than requiring someone to reorganize it by hand every time a site, a host, or a piece of hardware changes.

Stefan Wallin
Product Lead | Avassa
As a Product Lead at Avassa, Stefan forms the roadmap and evolution of the product together with the engineering team.
Keep exploring

See how the Avassa Edge Platform puts this into practice.