What deployment actually changes

A prototype and a deployed system can share every line of model-calling code and still be entirely different projects. The prototype is trying to answer whether the approach works at all. Once it is in front of people, the interesting questions all move to the other side of that line.

The questions change

In a notebook, you look at the output. In production, nobody looks at most of the output, so you need to know when something went wrong without reading it. That means logging inputs and outputs somewhere you can query them, and it means deciding in advance what “wrong” looks like: empty responses, responses that failed to parse, responses that took eleven seconds, responses where the model declined.

Latency stops being a footnote. A four-second response is fine while you are testing and unacceptable in a form someone fills out forty times a day. Cost behaves the same way — the per-call figure that looked negligible becomes a real line item once the call count is set by usage rather than by you.

Failure becomes a design surface

The prototype has one path: it works. A deployed system needs an answer for the model being unavailable, for the response arriving malformed, for the request timing out, and for the model being confidently wrong. Some of those are retries. Some are fallbacks to a simpler mechanism. At least one of them should usually be showing the person a clear, unembarrassed “I couldn’t do this” and letting them proceed another way.

What we keep relearning

The work that makes an AI feature dependable looks almost nothing like the work that made it possible. It is logging, timeouts, schema validation, a way to replay yesterday’s failures, and a decision about what the system does when it has no good answer.

None of it is exciting, and it is most of the job.