Build log: shipping Unseen UI to npm
How the component library behind this site went from a private workspace to two published packages, including the three things the first consumer broke.
Everything written
Journal
Every entry, read by engineering discipline or in series. Each one also sits on the map, next to the component it is about.
121 entries: all 18 series complete, 100 parts, plus standalone pieces. Weekly notes live in Notes .
New here? Pick where to start by what you do
October 2026 5
LangGraph vs the OpenAI Agents SDK
Two ways to write the same supervisor. Compared on control flow, tracing, provider coupling, testing and what each makes hard.
Every question your AI readiness review asks was answered months ago
The last six days of a sixty-day series, and the pattern is that operability gets bought early or it does not get bought at all.
RAG production-readiness checklist
Twelve checks to pass before a retrieval system answers a real user. Tick them locally; progress stays in your browser.
What an AI gateway actually costs to run
The operational bill for one gateway in front of a dozen tool servers, including the costs nobody budgets for.
September 2026 24
Working with Claude, practices that hold up
Prompting, agents and tools, Claude Code, evaluation and safety. The habits that make Claude-based systems reliable, as a checklist you can run against your own setup.
Pattern: the outbox for agent actions
Agents that write to systems of record need the same transactional outbox that event-driven services use. This entry covers the shape and the trade-offs.
Adding a second agent does not add intelligence, it adds a contract
Six posts on multi-agent systems, and the failures were never inside an agent. They were between two of them.
Designing a Multi-Agent System for Real-World Use
How to design and run a multi-agent system in production, using the supervisor pattern, with its trade-offs and where human approval belongs.
When to bring in a compliance review
The changes that should pull legal, privacy or compliance into an AI project early, and what to have ready when you do. Not legal advice; a way to ask at the right time.
Security review checklist for an AI feature
What to check before an assistant, RAG app or agent goes in front of real users. Grouped by area, ticked off locally; progress stays in your browser.
The retrieval cache that served stale policies
A well-meaning cache in front of retrieval kept answering from last quarter's HR policy for eleven days.
Six ways to wire agents together, and the same three things break every time
The topology gets all the design attention. Ownership, termination and traceability are what decide whether it survives contact with production.
Nothing broke, and the system is still getting worse
Drift is quiet by construction. SLOs and a standing review are what make it audible.
You cannot roll back a prompt you never versioned
AI failures cross model, prompt, retrieval, tool and data boundaries. The runbook and the rollback controls have to as well.
Real users ask questions your test set never imagined
Offline evals cover the questions you thought of. Production tells you the ones you did not.
Your AI feature has unit economics whether you measured them or not
Cost and latency belong to the workflow, not the model call. Budget them before rollout, not after the invoice.
Logging the answer tells you almost nothing
A trace that links intent, prompt, retrieval, tools and output is the only thing that makes an AI failure debuggable.
Most agent controls do not actually control anything
Six days of notes on supervising autonomous systems, and the same failure shape kept turning up: the control exists, it is documented, and nothing in the running system is bound by it.
The architecture review that happens before launch, not after the incident
Quality, safety, cost, latency, resilience and operations have to be visible on one diagram, at the same time.
Show 101 older entries, back to May 2026
An approve button is not human oversight
Approvals, events and tenant boundaries are architecture. Bolt them on as UI and they become theatre.
You are already building a control plane, badly
Policy, memory, evals and rollout get rebuilt inside every feature until someone names the layer they belong to.
Most agents are workflows wearing a costume
Chatbot, workflow, agent, RAG. Pick the wrong one and you spend a quarter debugging autonomy nobody asked for.
Every model call should go through something you own
Routing, budgets, fallbacks and policy need one place to live. Scattered SDK calls give you none of them.
The hard part of building an agent is not making it act. It is making it stop.
Six days of notes on agentic systems, and not one of the fixes that actually worked lived inside the model.
A wrong answer tells you nothing about which agent was wrong
Multi-agent systems need trace timelines, trajectory evals and designed recovery, because the final output hides everything that produced it.
An agent that stops when it feels finished has no stop condition
Permissions belong to roles, budgets belong to runs, and termination has to be something a machine can check.
Every multi-agent pattern is an answer to one question: who controls the run
Supervisor, orchestrator-worker, planner-executor, pipeline and judge differ mainly in where control lives and when it returns.
Write role contracts, not agent personalities
"You are a meticulous senior researcher" is a costume. Inputs, outputs, permissions and done criteria are a contract.
August 2026 25
Most teams reaching for multiple agents do not need them
Separation of work has to buy quality, control or throughput. Usually it buys a harder debugging problem.
Production RAG does not fail loudly, and that is the whole problem
Six days of notes on operating retrieval systems after launch, where nearly every real failure arrives dressed as a good answer.
You cannot instruct a model into data protection
Minimise what enters context, watch the paths data can leave by, and keep the evidence an incident will demand.
Blast radius is a design parameter
Narrow tools, staged writes, sandboxes and a tested kill switch. Containment is built before it is needed.
Untrusted text does not get to give orders
Injection defence is layered validation and clear authority, not a better-worded system prompt.
Your agent should not be a superuser
Bind every AI action to a user, a service and a run, then scope the verb rather than only the data.
Safety is an architecture decision, not a moderation setting
Assets, actors, abuse cases, controls, owners, evidence. Six things a team can actually write down before launch.
Most RAG failures are faithful answers to bad evidence
Six posts on reranking, citations and decomposition, and the uncomfortable thing they turn out to have in common.
Launch day is when evaluation starts
Offline scores expire on contact with real users. Sampling, groundedness, drift and outcomes are the parts that keep paying.
If it runs after the deploy, it is a postmortem
Regression gates only work when they sit in the release path, with a threshold, a slice and an owner.
A judge that cannot name the failure is not a judge
Deterministic checks first, one rubric per criterion, and a calibration loop against human review.
Your golden set is a product, not a spreadsheet
Labels, edge cases, versions, an owner. Skip those and the number moves without anyone knowing why.
An eval that cannot block a release is just a report
Decide which decision the number is allowed to stop, then build the harness around that.
No model can reason about evidence your retriever never returned
Six days of retrieval notes, and not one of the failures announced itself.
You cannot run RAG on user complaints
Groundedness scores, per-claim citations and an operations dashboard are what tell you quality is drifting before your users do.
Chunk for the question, not for the token limit
Chunk size is a decision about what a complete answer looks like. Reranking, context budgets and refusal thresholds finish the job.
Test retrieval on its own or you will tune the wrong thing
Most "the model is wrong" problems are recall problems. Hybrid search, query rewriting and versioned indexes are where they get fixed.
Ingestion is a data product, not a loader script
Parsing sets your quality ceiling, failures need a destination, and the update and delete paths are the half nobody tests.
Your vector index does not know who is allowed to read it
Embeddings carry no access control lists. Ownership, permissions, freshness and lineage have to be designed before anything is indexed.
Everything that makes an AI demo impressive is a constraint you have not added yet
Six posts on the distance between a system that answers well once and a system people are willing to depend on.
One MCP server is an integration. Thirty is why you need a gateway
Notes from building an MCP gateway. The protocol standardised how agents call tools, then stayed silent about credentials, context budgets and who may call what. That silence gets expensive as the servers multiply.
The 3am agent run that nobody is watching
Scheduled agent work fails quietly by default. Traces, release gates and a runbook are what make it fail loudly.
A retried agent job is a second chance to send the same email
Retries are the first thing teams switch on and the last thing they design for.
Four kinds of agent state, and only one of them is memory
Conversation, task, artifacts and product data end up in one store. Then a deletion request arrives and applies to all of it.
Your AI system's biggest problem is probably not the model
Six days of production AI notes, and the model choice never once turned out to be the thing that broke.
July 2026 28
Your tool registry is an access control list wearing a different name
A list of tools is documentation. A registry decides who may act, as whom, and leaves proof behind.
Autonomy is a runtime decision, not a model decision
Agents get more capable by default. They only get bounded on purpose.
When MCP earns its overhead, and when a direct API is better
One question settles most of these arguments: will more than one AI application need this capability?
A clean connector does not make a reliable agent
Connector correctness and agent behaviour are separate problems. Most teams only test the first one.
Tools, resources and prompts are not interchangeable
Choosing the wrong primitive hands the model authority a person was supposed to hold.
Where your MCP server runs is a production decision
Transport looks like a technical detail during the demo. It is really a choice about ownership, reachability and trust.
What the MCP protocol actually standardises
It settles how capabilities are described and discovered. Every decision about whether to trust them is still yours.
From model demos to mission-ready AI systems
A practical map for moving from impressive AI demos to systems people can trust.
Governance is architecture, not a document after launch
Controls that live only in a policy document are not enforced. Build them into the system the policy describes.
Production AI is coordination: data, models, tools, controls, people
No single component makes production AI work. It is the coordination of data, models, tools, controls, evaluation and people.
If you cannot trace it, you cannot run it
Production AI needs traces across prompts, context, retrieval, tools, costs and decisions to debug anything at all.
Evaluate the workflow users see, not just the model
Model benchmarks say little about the product. Measure the whole workflow the user actually goes through.
Retrieved content is data, never instructions
Text pulled from documents and tools must never gain authority over what the system does next.
Agents need arbitration rules before they disagree
Decide how disagreements get settled before they happen, so they never leak into the final answer.
A handoff is a contract: state, evidence, open questions, an owner
A handoff that passes only a summary loses the evidence and the open questions. Make each one a structured contract.
Shared agent memory is an access-control problem
Once agents share memory, the question is who may read and write what, and that is access control.
Agents cannot collaborate without a shared message contract
Agents that pass free text to each other drift. A shared message contract is what makes collaboration reliable.
Role design matters more than agent count
Each role needs a clear purpose, authority, tools, memory and evaluation. Adding agents without that adds confusion.
Decentralized agents need convergence rules before they earn trust
Without a central coordinator, agents need rules that make them converge before production can trust the result.
AI debate is only as good as its judge
Agents arguing produce better answers only when the judge has reliable criteria for deciding between them.
Shared state needs structure, attribution and versions
When several agents write to the same state, it stays debuggable only if every change is structured, attributed and versioned.
Peer agents need protocols, not vibes
Agents talking as equals need communication rules, a limit on rounds and one owner of the final decision.
Dynamic dispatch needs visible routing and one final owner
When agents pick who handles what at run time, every routing decision must be visible and one agent must own the result.
Pipelines make AI handoffs inspectable, so start there
Predictable workflows are easier to inspect and fix as a pipeline. Add agent autonomy only where the pipeline runs out.
Supervisor agents help until they become the bottleneck
A supervisor works when routing, review and authority are clear, and becomes the bottleneck when they are not.
Hierarchy helps only where authority really has layers
Layers of agents make sense when the work has real layers of authority and responsibility, and add cost when it does not.
One giant agent is rarely the cleanest design
Split work across agents to reduce complexity through specialization, never to add agents for their own sake.
Every autonomous run needs a budget for cost, time, actions and risk
Limits on cost, time, actions, retries and risk are what make autonomy safe to switch on.
June 2026 29
Design recovery into an agent on day one
Retries, fallbacks, checkpoints, rollback and escalation belong in the first design, not the first incident review.
Good guardrails make safe progress possible, not just block
A guardrail that only blocks teaches users to work around it. Good ones steer uncertain work toward a safe path.
Human review belongs where it has timing, authority and context
A reviewer who sees too little, too late, or cannot say no is a rubber stamp. Placement decides whether review works.
Agent memory is several stores, each with its own rules
Useful memory is separated, scoped and governed, not one database the agent writes everything into.
Reflection is worth it only when it changes the next action
Reflection earns its cost when it decides the next move: revise, retry, escalate or stop.
Tool access is where AI becomes operational risk
Giving a model tools means designing permissions, observability, rollback and approval paths with them.
Plan before acting when actions cost money or carry risk
When actions have cost, risk or dependencies, a plan up front saves the expensive mistakes later.
Agent behavior is a loop, not a prompt
Reliable agents come from how the loop is designed, not from one clever prompt.
Give an agent an outcome, a boundary and a stop condition first
Before any autonomy: what done looks like, what is out of bounds, what it may spend, and when it must stop.
Evaluate RAG in parts: retrieval, grounding, generation, citations
One end-to-end score hides which stage failed. Measure each stage on its own so you know what to fix.
Weak evidence should trigger recovery, not a confident answer
When retrieval comes back thin, the system should search again, ask, or say it does not know, not write a fluent guess.
Reflection helps only when it checks against evidence
A model reviewing its own answer helps only when it checks against evidence and is allowed to change course.
A stale answer is a wrong answer
When the world changes faster than your index, freshness stops being a nice-to-have and becomes part of correctness.
When the evidence is a table, a chart or a recording
Evidence quality changes when the source is visual, tabular or audio, and text-only pipelines miss what changed.
Production knowledge lives in systems of record, not only in PDFs
The current price, the open ticket, the approved limit: much of what users ask lives in systems of record, not documents.
Some answers live in relationships, not documents
Who owns what, what depends on what: some answers sit in the links between records, where document search cannot see.
One question can need many searches
A single query often returns half the evidence. Some questions need several searches before the answer is safe.
Answer complex questions in parts, and check each part
A compound question answered in one pass hides its weakest part. Split it, answer each piece, then combine.
Good RAG needs the precise snippet and the context around it
A precise snippet without its surroundings misleads; surroundings without the snippet waste the context window.
Every important AI claim needs a receipt
Provenance lets users and teams inspect where a claim came from before they act on it.
Reranking is where retrieval becomes useful
Broad retrieval finds candidates; reranking decides which of them the answer can depend on.
Relevant data can still be data the user may not see
The most relevant passage is still the wrong one if the person asking was never allowed to read it.
A vector store is infrastructure, not magic memory
A vector database needs the same operating discipline as any other store: ownership, refresh, access and monitoring.
Production retrieval needs exact words and meaning
Product codes, names and error strings need exact matching; paraphrased questions need meaning. Real queries bring both.
Hybrid search is the practical default
Combining keyword and semantic signals beats betting the whole system on one search method.
Similar is not the same as correct: the limits of embeddings
Embeddings find text that sounds related. Whether it is correct, current or allowed is a separate question.
Chunk boundaries decide what your RAG system can know
Retrieval can only return what a chunk holds. Split in the wrong place and the rule arrives without its exception.
Retrieval quality is decided before anyone searches
Answer quality is set when knowledge enters the system, long before a user types a question.
RAG is an evidence design problem
Letting the model search is the easy part. The work is deciding what counts as trusted evidence.
May 2026 10
Chatbot, workflow or agent: choose by how much autonomy the task can bear
A practical way to choose between conversation, repeatable workflows, and bounded autonomy.
Structured output turns model text into something software can trust
Free text is for people. Software needs a schema it can validate before acting on what the model said.
A capable model is not a product yet
Capability is the starting point. Workflow, product design and ownership are what turn it into something people rely on.
Confidence is not evidence
A confident tone tells you nothing about correctness. Answers need evidence, boundaries and a way to say "not sure".
Temperature is a product decision, not a writing style
Sampling settings decide how consistent the product feels to users, not just how the text reads.
More context can make an AI system worse
A bigger context window is not a reason to stop choosing what goes into it.
Vague prompts become vague systems
Once real users depend on the system, a prompt is an operating instruction and deserves the same care.
LLMs generate one token at a time, so design for it
Fluent output is generated step by step, which is why it needs constraints, validation and repeatable formats.
Tokenization quietly sets your cost, limits and reliability
A low-level text detail that ends up as a product constraint on cost, latency and reliability.
Model choice is rarely your first production problem
Production AI succeeds or fails in the system around the model, not in which model you picked.
Coverage by disciplineThinnest so far: Networking (5), Identity & Access (7)
Coverage
Disciplines down, series across. An entry counts once for each discipline it covers. Select a cell to read those entries.
Thinnest so far: Networking (5), Identity & Access (7)
| Discipline | Basics | Demo → product | Retrieval | Evidence RAG | RAG ops | Agents | Control | Multi-agent | Coordination | Readiness | MCP | Agent infra | Prod RAG | Evaluation | Safety | MAS design | Arch patterns | AI ops | Standalone | All |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AI & Models | 6 | 5 | 1 | 3 | 2 | 5 | 4 | 6 | 4 | 2 | 1 | · | 2 | 1 | 1 | 5 | 1 | 1 | 10 | 60 |
| Data & Databases | 1 | 3 | 6 | 6 | 5 | 1 | · | 1 | 1 | 1 | · | 1 | 5 | 1 | 1 | · | · | 1 | 9 | 43 |
| Networking | · | · | · | · | · | · | · | · | · | · | 2 | · | · | · | · | · | 2 | · | 1 | 5 |
| Security | · | · | 1 | · | · | 2 | 2 | · | 1 | 2 | 1 | 2 | 1 | · | 5 | 1 | 1 | · | 9 | 28 |
| Identity & Access | · | · | 1 | · | · | · | · | · | 1 | · | · | 1 | · | · | 1 | · | 1 | · | 2 | 7 |
| Integration | · | 1 | · | · | 2 | 1 | · | 2 | 2 | 1 | 4 | 2 | · | · | · | 1 | 2 | · | 5 | 23 |
| Compute & Infra | · | 1 | 1 | · | · | 1 | 4 | · | · | 1 | 1 | 4 | · | 1 | 1 | 1 | 2 | 1 | 2 | 21 |
| Observability & Ops | · | · | · | · | 1 | · | 1 | 1 | · | 2 | · | 1 | 1 | 5 | · | 1 | 1 | 4 | 4 | 22 |
| Cost & Governance | 2 | · | · | 1 | · | 1 | 1 | · | 1 | 1 | · | · | · | · | 1 | 1 | 1 | 1 | 3 | 14 |
Series progressAll 18 series complete, 100 parts
Series
All series- AI Systems Basics6/6
- From LLM Demo to AI Product6/6
- Retrieval Foundations6/6
- Evidence-Driven RAG Patterns6/6
- Production RAG Operations6/6
- Agentic AI Foundations6/6
- Agent Control and Supervision6/6
- Multi-Agent Design Patterns6/6
- Agent Coordination Contracts6/6
- Production AI Readiness6/6
- 5 Days of MCP for Production AI5/5
- 5 Days of Agent Infrastructure5/5
- 5 Days of Production RAG5/5
- 5 Days of AI Evaluation5/5
- 5 Days of AI Safety Engineering5/5
- 5 Days of Multi-Agent System Design5/5
- 5 Days of AI Architecture Patterns5/5
- 5 Days of AI Operations and Observability5/5