Why model choice is rarely the first production AI problem

Most AI projects open with the same question: which model should we use?

It is a reasonable question. It is almost never the one that decides whether the thing works.

The question underneath

Model choice is the part a team can evaluate in an afternoon and demo by Friday. That is exactly why it absorbs the attention. It is legible, comparable, and there are leaderboards.

The system around the model is none of those things. What feeds it context. What constrains its output. What checks the result. What happens when it is wrong. Who gets paged. Those decisions are slower, less exciting, and they determine whether the model's capability ever reaches a user intact.

Swap a good model for a great one and a broken system produces slightly more articulate broken output.

Where the failures actually surface

This is hard to see because system failures do not look like system failures. They look like model problems.

The model scores well in isolation. Then in production it answers from a document that was superseded in March, or produces text a downstream parser cannot read, or takes eleven seconds when the user needed two, or confidently handles a case nobody defined a behaviour for.

Every one of those gets reported as "the AI is unreliable." None of them is fixed by a better model, and teams can spend a full quarter on model evaluation before noticing that the failing component was never the model.

What to decide before you pick

Three things, and none of them require knowing which model you will use:

• The user outcome. Not "answer questions about our docs", the specific job, done specifically well enough to be worth shipping.
• The boundaries. What the system may assume, retrieve, generate, and refuse. Especially refuse; that one gets skipped and then gets discovered in production.
• An evaluation loop. Even fifty labelled examples. Build it before the demo becomes the standard, because once a demo has impressed someone, "it works" is already established as the baseline and nobody wants to measure it.

A test you can run today

Take one real request and trace it end to end. What feeds the model, what constrains it, what checks its output, and what happens when the answer is wrong or uncertain.

The gaps in that trace are your actual roadmap. In sixty days of these notes, almost everything that follows is a way of filling one of them.

Closing thought

A good AI product is a controlled system that happens to use a model, rather than a model call with a UI around it.

If model choice were frozen today, which system behaviour would still need designing before real users could trust it?