From AI Prototype to Production Software: The Engineering Work That Matters

AI prototypes are quick to build and often impressive. A few prompts, a sample of data and a simple interface can show what is possible in days. Many of these prototypes then stall. The model was rarely the problem. The gap is the engineering around the model, which a prototype does not need and a production system cannot do without.

Why a good demo can fail in production

• It was tested on a small, clean sample rather than the messy data real users bring.
• It ran with one person's full access, so permissions were never considered.
• Nobody measured how often it was wrong, only that it was impressive when it was right.
• It had no plan for a slow or unavailable model provider.
• Its cost was invisible at demo volume and significant at production volume.

Evaluate the model's behaviour, not just its best answers

Before launch, build a set of test cases from real examples, including awkward and adversarial ones, with the answers you expect. Run it whenever the prompt, the retrieval logic or the model changes. Model behaviour shifts between versions, and without a repeatable evaluation you will not notice until users do. Decide in advance what level of error is acceptable for the task, and what happens when the system is unsure.

Treat the model as one component

In production, the model call should sit behind your own service with a clear interface. That service owns prompt templates, retrieval, input checks, output validation and logging. Keeping the provider behind that interface means you can change models, or run an open model on your own infrastructure, without rewriting the application.

Data integration and permissions

Most useful AI features need company data: records from a CRM, documents from a shared drive, rows from a database. The assistant must only see what the current user is allowed to see. Enforce this in the data access layer, using the same identity and roles as the rest of your systems. Do not rely on the prompt to keep data private. Instructions in a prompt are not an access control.

Authentication, authorization and actions

Reading data and taking actions carry very different risks. When a model can call tools, such as creating a ticket, sending an email or updating a record, each tool should be narrowly scoped, checked against the user's permissions, and logged. Consequential actions should need a human to confirm them. The OWASP Top 10 for Large Language Model Applications is a useful checklist here, especially its guidance on prompt injection and excessive agency.

Errors, timeouts and outages

Model providers have outages, rate limits and slow periods like any other dependency. Set timeouts, retry carefully, and decide what the user sees when the AI feature is unavailable. Often the right fallback is the workflow that existed before: the form still works, the ticket still gets routed, and the AI suggestion simply does not appear.

Logging, monitoring and cost control

• Log inputs, outputs, model version and latency for each request, with personal data handled according to your retention rules.
• Track token usage and cost per feature and per customer, with alerts on unusual spikes.
• Set per-user and per-account limits so one heavy user cannot run up the bill.
• Monitor quality signals, such as user corrections or thumbs-down feedback, not just uptime.

User experience and human oversight

Users should know when they are seeing AI output, be able to correct it easily, and have a clear path to a person. Show sources when answers come from documents. For high-stakes tasks, present AI output as a draft for review rather than a final result. Frameworks such as the NIST AI Risk Management Framework are helpful when you need to agree with stakeholders how much oversight a use case needs.

Deployment and maintenance

An AI feature needs the same release discipline as the rest of your software: version control for prompts and configuration, staged rollouts, the ability to roll back, and a feature flag to switch it off quickly. Plan for ongoing work too. Models are updated and retired, provider pricing changes, and your data changes. Somebody has to own re-running evaluations and keeping the integration current.

Ownership

Decide early who owns the system after launch. If an external partner builds it, the code, prompts, evaluation sets and infrastructure should live in your accounts, with documentation your team can follow. A production AI system is a piece of software you will maintain for years, not a one-off project.

A short checklist before launch

• Is there a repeatable evaluation set built from real examples?
• Does the assistant only see data the current user is allowed to see?
• Are tool calls scoped, permission-checked, logged and confirmed where it matters?
• Is there a working fallback when the model is slow or unavailable?
• Are cost, latency and quality monitored, with alerts?
• Can the feature be switched off quickly without breaking anything else?
• Is it clear who owns and maintains it after launch?