Audit Reflection — Sprint 1-5 (2026-05-20)

Numbers

Metric	Value
PRs merged today	20
Net lines added	~10,500
Lib tests (start → end)	429 → 510 (+81)
New modules	18
New top-level commands	16
New typed errors handled	1 (FreezeRefusal)
New exit codes documented	1 (8)
Issues closed (#57-#73 + #74)	17 of 18 (#74 stays open for follow-ups)
Live-smoked against PVE 9.1.1	10 features
Live smoke deferred to CI	8 features
CodeQL false positives dismissed	1 ({e:#} format-capture, same pattern as 19 dismissed during S1-S3)
Real security vulnerabilities	0
Open dependabot PRs	0
Branch protection bypasses (--admin)	11 (all admin-merge after CI passed; no protection rules disabled)

What landed today

Epic #74 — Cluster state export + reconcile (the GitOps moat)

5 stacked PRs closing the read → diff → apply loop:

• #75 pools export — foundation + state export --resource pools
• #76 ACL export — 4-tuple identity, kind enum
• #80 storage export — 15-field model, skip-empty discipline
• #81 diff layer — state diff, Change kind, StateReadView narrow trait
• #82 apply layer — state apply, dry-run / prune / continue-on-error, StateWriteView narrow trait

End-to-end verified live: export → edit → diff → dry-run → apply → re-diff convergence proven on PVE 9.1.1.

Sprint 1 (quick wins)

• #83 migrate --stream — live per-disk progress (ANSI bars / NDJSON), 16 tests
• #84 logs tail — cross-node journalctl fanout via SSH (grep/since/service filters), 8 tests
• #85 explain — bundled knowledge base for every typed error (13 entries), 6 tests
• #86 incident freeze/thaw — cluster-wide write kill-switch with TTL + audit + exit code 8, 13 tests

Sprint 2 (multi-cluster + AI moats)

• #87 fanout — ls --all-profiles / find across every cluster, 3 tests
• #88 describe — structured cluster digest with --output llm-context for AI chat preamble, 3 tests
• #89 MCP notifications — server-sent SSE notifications for task lifecycle + freeze/thaw, 6 tests

Sprint 3 (operational moats)

• #90 console record — asciinema cast v2 record + replay for serial sessions, 3 tests
• #91 cloud-img — SHA-256-pinned cloud-image registry, server-side-verified downloads, 3 tests
• #92 scheduler — interval-based recurring proxxx ops with TOML persistence + run-due, 4 tests

Sprint 4 (big moats)

• #93 upgrade-check — PVE major-upgrade pre-flight scanner with CI-gateable exit, 3 tests
• #94 batch (6-in-1):
  • #62 accounting — per-pool/node/tag resource aggregation
  • #70 heatmap — per-node API RTT with green/yellow/red bucketing
  • #68 anomaly — z-score outlier detection
  • #61 backup-verify — metadata-level freshness + integrity probe
  • #66 import — qemu-img convert wrapper (raw/vmdk/vdi/vhdx/vhd → qcow2)
  • #57 gpu-inspect — IOMMU + vfio + lspci probe via SSH

Per-feature scope honesty

Every PR documents its deferred scope in the rustdoc. A quick gloss of what's MVP vs full:

Feature	MVP shipped today	Deferred (follow-up issues)
state-export	Pools / ACL / storage	Firewall-cluster / backup-jobs / notifications / HA groups
state-apply	All 3 resource families CRUD	Pre-flight risk gates + HITL approval wrapper
migrate --stream	Per-disk + RAM + finished events	TUI progress widget (CLI table only)
logs tail	journalctl fanout via SSH	Multi-format parser beyond short-iso
explain	13 typed-error entries	--apply-fix N (run remediation as proxxx command)
incident freeze	API mutations only	MCP dispatch + scheduler + HITL daemon broadcast
fanout	ls/find read-only	Writes (explicitly out-of-scope; footgun risk)
describe	All 4 formats including llm-context	RRD time-window stats
mcp notifications	HTTP SSE	stdio writer task (broker is ready)
console record	Serial sessions	SSH sessions (subprocess flow refactor needed)
cloud-img	Download + SHA-verify	VM-create orchestration (qm create + qm template)
schedule	run-due + add/list/pause	Long-running daemon, cron syntax, SQLite migration
upgrade-check	3 rules for PVE 8→9	Full pve8to9 parity (~30 checks)
accounting	Current state	Time-window CPU-hours from RRD + PBS bytes
heatmap	API RTT only	Corosync + storage RTT via SSH
anomaly	CPU + mem% z-scores	Prometheus integration, daemon mode
backup-verify	Metadata probe	Full dry-restore to throwaway VMID
import	qemu-img convert	OVA/OVF parsing, libvirt-XML, VMware-direct
gpu-inspect	Read-only probe	gpu bind (write configs + reboot)

Every deferral is documented in the module rustdoc with a "why" — none are unreviewed gaps.

Architecture patterns established / reused

• Narrow trait + blanket impl over ProxmoxGateway — pioneered for state::apply::StateWriteView, reused for migrate_progress::TaskLogView. Lets unit tests stub a handful of methods instead of the full 200+-method gateway. Pattern is robust and now well-trodden.
• Hand-rolled parsers preferred over regex — migrate_progress, logs::journalctl_line. Keeps the dep surface unchanged.
• Bundled compile-time const data — explain::ENTRIES, cloudimg::REGISTRY, upgrade_check::RULES. Zero-allocation lookups, type-safe.
• Atomic file writes for state files — incident lock, schedules, freeze. Tempfile + rename + 0600 on Unix.
• _at(path, …) explicit-path test variants — used by incident and schedule modules to avoid env-var contention under parallel test execution. Robust pattern, recommend adopting wherever a default-path helper exists.

Quality posture

• cargo fmt — green at HEAD.
• cargo clippy --all-targets -- -D warnings — green at HEAD.
• cargo audit — 0 vulnerabilities.
• cargo deny check — advisories / bans / licenses / sources all green.
• Lib tests — 510 passing, 0 failing.
• CodeQL — 0 real findings (1 false positive dismissed for the same {e:#} format-capture class we dismissed 19 times during the v0.2.1 hardening pass).
• Scorecard — 7 informational alerts unchanged from v0.2.1 (CIIBestPractices, CodeReview, Fuzzing, Maintained, Vulnerabilities, PinnedDependencies × 2 in live-test scripts).

Known tech debt + sharp edges introduced today

1. cloudimg::REGISTRY SHA-256 are placeholders. All four entries ship with 0000… placeholder SHAs. The runtime gate refuses downloads when SHA is all-zero — operators can't accidentally run an unverified download. But the entries are unusable in production until someone populates real SHAs. Action: open a follow-up issue to populate Ubuntu / Debian / Alpine / Fedora SHAs from the upstream release manifest.
2. MCP stdio notifications deferred. The broker is ready; the server.rs loop needs a select! refactor to also poll the broker. Open issue.
3. SSH session recording deferred. Only serial recording shipped; SSH recording needs a PTY-proxy refactor (current execute_ssh spawns the system ssh subprocess and doesn't see PTY bytes).
4. Schedule subprocess approach for run-due. Each scheduled run spawns proxxx as a child. Cheap, isolated, but loses in-process audit chain continuity. Daemon mode (folding into a unified alert+HITL+schedule loop) is the eventual right answer.
5. Upgrade-check has 3 rules, pve8to9 has ~30. Each follow-up rule is a 30-line addition; the framework is in place.
6. Anomaly detection uses cluster-wide single-tick population. With 1 node you get no signal; with 2-3 nodes the population is too small for reliable z-scores. Multi-tick rolling window (RRD-backed) is the real fix.

What's NOT deferred

• State apply layer is full CRUD across pool / ACL / storage with safety semantics (dry-run / prune / continue-on-error). Production-grade.
• migrate --stream parses every recognised PVE format. Live-smoked.
• logs tail handles graceful per-node failure. Live-smoked.
• explain covers every typed error variant that can reach an operator's terminal.
• incident freeze is wired into api::PxClient::{post,put,delete} — every mutation goes through the gate.
• cross-profile fanout is functionally complete for reads.

v0.3 recommendation

Today shipped 17 features as MVPs. The next release rhythm should be:

1. Populate cloudimg SHAs (one-evening task).
2. MCP stdio notifications (one PR, broker is ready).
3. Pre-flight + HITL wrappers around state apply (the #74 epic's last loose end — high-leverage, well-scoped).
4. Pve8to9 full rule coverage (slowly drip in one rule at a time as operators report gaps).
5. Schedule daemon mode — fold into a unified alert+HITL+schedule loop.

Everything else can ship incrementally based on field demand. The MVPs are real; the deferrals are honest.

---

🤖 Generated 2026-05-20.