ADR-0008: Embed AIMP as the mesh control-plane bus

• Status: accepted; experimental since 2026-10-08, rotation / revocation / fleet claims frozen (see the status update at the end)
• Date: 2026-05-08 (v0.2.x mesh wire-up)
• Tags: mesh, aimp, gossip, identity, federated-state

Context

A single zion instance is autonomous: WAF gates, rate-limit windows, upstream-health probes — all run from local state. That works at one node and falls apart at fleet scale: an attacker probing instance A hits instance B unmarked, an upstream that flapped on C is still in the active pool on D, and a rate-limit window saturating on E doesn't trigger any caution on F. The fleet behaves like N independent proxies, not one edge.

Track B (v0.2.0) introduced aimp_node — a serverless gossip + Merkle-CRDT + Ed25519-identity primitive designed precisely for "edge nodes share signed evidence without a central coordinator". The decision in front of us was not "should we share state" but "what's the bus".

Decision

Embed aimp_node as an optional cargo feature (sovereign-aimp, implies sovereign-signals). AIMP carries the wire — Ed25519-signed envelopes over UDP, anti-entropy SyncReq rounds, Sybil-resistant peer enrollment. Zion sits above AIMP as a typed claim layer: local subsystems publish typed MeshClaims (WAF block, rate saturation, upstream unhealthy), the mesh aggregates and re-broadcasts, and consumers subscribe via tokio::sync::watch.

Local decisions remain authoritative. A mesh signal influences scoring — a known-malicious score forwarded to upstream as X-Zion-Mesh-Score, an XDP install only at high-confidence consensus — but never overrides the local WAF / auth / rate-limit gate. That gate is the trust boundary; the mesh is a force multiplier, not a delegated authority.

Consequences

• Positive — federated state without central infrastructure. A five-node fleet running on three continents can converge on shared WAF reputation, rate-saturation alerts, and upstream-health quorum in seconds, with zero coordinator process to operate, scale, or page on at 03:00.
• Positive — signed evidence. Every claim carries an Ed25519 signature against a per-node identity. The audit log records the publish + the receive; a forensic trail survives any single node's compromise.
• Positive — out-of-process viable. AIMP runs as a library today (in-process gossip task), but the protocol is identical to the sidecar mode aimp_node ships. Nothing about the integration forecloses operating it as a separate daemon for blast-radius reasons later.
• Positive — correlation-aware aggregation. AIMP's CRDT layer drops duplicates and merges concurrent edits; we don't have to hand-roll that on the zion side.
• Negative — new dep. aimp_node is research-grade (the protocol hasn't yet hit semver-stable); we pin to a specific commit hash in Cargo.toml and treat the integration as experimental.
• Negative — new attack surface. UDP gossip listener exposes envelope-parse + signature-verify code to peers. The threat model addendum (ADR-companion: docs/security/threat-model.md §10) walks the STRIDE rows — Spoofing is countered by Ed25519, Tampering by AEAD on Noise transport, etc.
• Negative — new operator surface. Operators have to manage a peer list and an Ed25519 identity file per instance. Defaults (auto-generate identity on first boot, chmod 600, persist under [sovereign_aimp].identity_path) keep the floor low.
• Neutral — versioned wire format. Envelope versioning is AIMP-side. A future major bump would require fleet-wide rollout coordination; the documented protocol stability is sufficient for v0.2.x but tracked for v0.4 mesh hardening.

Alternatives considered

• etcd / Consul. Both solve federated state correctly, but they require central infrastructure (a 3- or 5-node coordinator cluster). That breaks the "edge / autonomous" use case zion is designed for: an operator deploying one zion per region cannot also operate a coordinator quorum per region without doubling their on-call surface. Lost.
• Custom gossip protocol (NIH). We'd reinvent: signed envelopes, CRDT integrity, Sybil resistance, anti-entropy, peer enrollment. Each of those is a multi-week effort done correctly. AIMP already has them under test. Lost.
• OpenTelemetry as the bus. OTLP is built for telemetry export, not state replication: no quorum semantics, no convergence guarantees on partition heal, no cryptographic identity per emitter. Wrong tool. Lost.
• Out-of-process AIMP node from day one (sidecar). Possible — the protocol is wire-compatible — but trades one less in-process task for an extra network hop on every claim publish/receive. We start in-process to keep the latency budget tight; sidecar mode is reserved as a future enhancement when the blast-radius reasoning flips (e.g. running AIMP under a different uid for capability isolation).

References

• Source: src/aimp_cp.rs
• AIMP project: https://github.com/fabriziosalmi/aimp
• AIMP security model: https://github.com/fabriziosalmi/aimp/blob/master/SECURITY.md
• Operator-facing integration guide: docs/mesh/integration.md
• Threat-model addendum: docs/security/threat-model.md §10 (mesh)
• v0.2.2 wire-up commit: 2490fe8
• v0.2.1 mesh-score signal: PR #65

Status update — 2026-10-08

The decision stands (AIMP as an in-process, opt-in bus), but the mesh is declared experimental and part of what this ADR anticipated is frozen. What exists: signed gossip of WAF reputation to a static peer set, an allowlist of trusted node ids (trusted_keys, fail closed), replay and timestamp rejection, source-bound revocation, an optional per-source rate cap, a persisted identity. Frozen, with their specs kept in the issues: signed rotation and revocation claims (#68), RateSaturation and the upstream-health quorum (#66, #67), and what needs them (quorum, peer quarantine, the XDP reconciler, #53). Where the text above says these "land with" a later mesh slice, read "if a real multi-node deployment needs them".

Why: the code went the other way from the TOFU model sketched here. Trust is an operator-provisioned allowlist, which is simpler and safer than a self-extending one: a signed IdentityRetired claim would let the holder of a compromised key promote a new key that stays trusted after the operator removes the old one. With an allowlist, rotation and revocation are "add, switch, remove" if trusted_keys can be reloaded without a restart, and that is the first step to take when the mesh unfreezes; the signed claims matter only at a fleet size nobody has asked for. The rate and health claims duplicate, across nodes, what a load balancer with affinity and the in-band circuit breaker already do for one, and give a trusted peer a lever over other nodes' decisions. Their issues stay open as the parked specs.