New here? Start this series at Day 55: Why observability is not optional for AI systems
- Day 55: Why observability is not optional for AI systems
- Day 56: Why evaluating the model is not enough
- Day 57: Why retrieved content must stay untrusted
- Day 58: Why governance belongs in the architecture
- Day 59: Why production AI is coordinated infrastructure
- Day 60: From model demos to mission-ready AI systems
Run this test on any AI system you have in production: ask who owns it. A name, not a team: a person who answers when it misbehaves.
If the room goes quiet, that system has an architecture gap. It just does not appear on any diagram.
Governance decomposes into engineering artifacts
The word sounds like paperwork. In practice it is six concrete things, each with a specific failure when missing:
- Policy: what the system may and may not do, written before the incident that would have needed it. Missing: every edge case becomes an argument under time pressure.
- Approval: who signed off on deployment, on the record. Missing: "who decided this was ready?" has no answer.
- Audit: decisions reconstructable months later. This is a data-retention design choice, made up front or not at all.
- Privacy: whose data flows through, on what basis, and how it exits when someone requests deletion.
- Fairness: behaviour tested across the people actually affected, not just the average case.
- Incident response: who gets paged, what gets switched off, how fast.
Each one is architecture with a name attached rather than a document, and the pattern in teams that ship AI durably is that they design them in from the start, when each is a small decision rather than a retrofit.
Retrofitted accountability leaks
Adding audit trails to a system that was not built to produce them means reconstructing history you never stored. Adding a deletion path means finding data you never tracked. Adding approval gates means re-architecting flows already in production.
Each of these costs roughly an order of magnitude more after launch than before, and the retrofitted version is usually partial in ways nobody notices until it is tested by an actual request.
Connect policy to controls
The failure mode worth naming: a policy document that no technical control enforces.
"The system must not access customer PII without consent" is a sentence. What enforces it? If the answer is "the prompt asks it not to," you have documentation rather than governance, and the two are only distinguishable during an audit or an incident, which is precisely when the difference is expensive.
Governance is part of the architecture. It is how the system earns permission to operate, and autonomy that scales without ownership scaling alongside it is deferred liability.
More on these topics
Checklist · · 18 checks
When to bring in a compliance review
The changes that should pull legal, privacy or compliance into an AI project early, and what to have ready when you do. Not legal advice; a way to ask at the right time.
Explainer · · 1 min read
Why multi-agent systems need decision rules
Why agents need arbitration rules before disagreement reaches the final answer.
Discussion