Launching an AI agent feels like the finish line. In practice it is the point where the real work starts. The agent now lives among systems, policies and customers that keep changing, and nobody is watching it with the attention it had during the pilot.
This article is for the manager whose agent is already in regular use. If you are still deciding what to do after a pilot, read the pilot decision guide first. Here we cover what to do in month three, month nine and year two.
Why a working agent stops working
Agents rarely fail with a bang. They degrade, and the causes are mostly outside the agent itself.
- The inputs change. A new product line, a renamed field, a revised policy document or a different customer mix means the agent sees cases it was never tested on.
- The model changes. Providers update and retire model versions. Behaviour on your tasks can shift even when your prompt does not.
- The tools change. An API gains a parameter, a permission is widened for another team, a connected system is migrated. The agent inherits all of it.
- The people change. Reviewers get used to approving, then approve without reading. The person who set the agent up moves to another job.
The NIST AI Risk Management Framework treats this as a normal part of the lifecycle: risks are managed and measured continuously, not once at deployment. A regular review is how that principle becomes a calendar entry.
Name an owner and a deputy
Every agent in production needs one named person who answers for it. Not a team and not a committee. That person does not have to be technical, but they must be able to decide: tighten, pause or switch it off.
Give them a deputy, and write both names where the agent is documented. Check the names whenever someone changes role. Orphaned agents, running with permissions nobody remembers granting, are among the most common findings when companies first inventory what they have. Our guide to agent identity and least privilege explains why the permissions deserve the same attention as the owner.
A monthly check that takes an hour
The monthly check should be small enough that it actually happens. Use the same one-page template each time and look at:
- Volume and outcomes. How many cases did the agent handle, and how did that compare with last month and with the baseline from the pilot?
- Corrections. How often did a person change or reject the agent's output? A rising rate is the earliest warning you will get. A rate that falls to zero is also suspicious, because it may mean reviewers stopped checking.
- Incidents and near misses. Anything that triggered your incident process, plus cases that nearly did.
- Cost. Usage, platform fees and review time against the budget. See cost controls and loop limits for the limits worth setting.
- Changes since last time. Any new model version, prompt edit, tool or permission change, and whether each was approved.
Sample a handful of real cases from the logs and read them end to end. Numbers show that something moved; the cases show why. If you have set up observation without leaking data, this sampling is quick and safe.
A quarterly review that asks harder questions
Every three months, widen the lens. The owner, a business sponsor and someone from security or compliance should spend a couple of hours on questions the monthly check cannot answer:
- Is the process the agent supports still the same? If the underlying work changed, the agent may be optimising a task nobody does any more.
- Are the permissions still the minimum needed? Remove anything unused in the last quarter.
- Does the original value case still hold, using the same counting method as before? Include review and rework effort, not only the time saved.
- Has the test set kept pace? Add recent real failures to the cases you run before any change, as described in evaluating reliability before production.
- Do the vendor terms, data retention and exit plan still match what you are doing? Revisit contracts and exit terms when volume or data sensitivity has grown.
Finish with a decision, not a discussion: continue as is, adjust, expand carefully, or retire. Record it in two lines with a date.
Control changes, even small ones
Most production surprises come from a "small" edit. A reworded instruction, a swapped model version or one extra tool can change behaviour broadly. Apply a lightweight change process:
- Describe the change and why it is wanted.
- Run the test set and compare with the previous result. Look at the failures, not only the average.
- Release to a small share of cases first where you can, with a clear way back to the previous version.
- Keep the previous version of prompt, configuration and model reference so rollback takes minutes.
- Record who approved it.
Treat a permission increase as a bigger change than a prompt edit. It usually deserves approval from the owner and from whoever owns the system being opened up.
Decide the triggers in advance
When something looks wrong, people argue about whether it is serious enough to act. Avoid that by writing triggers down while everyone is calm. Examples:
- Correction rate above an agreed threshold for two consecutive months: return to adjustment and re-test.
- Any serious incident, or the same near miss twice: pause the agent and run the incident process.
- Cost above budget by an agreed margin: reduce limits before investigating.
- No reviewed sample for a month: the agent works in a restricted mode until the review is done.
- Owner and deputy both unavailable: the agent is paused, not left to run.
The exact numbers depend on your process. What matters is that they exist, that the owner can act on them without asking permission, and that nobody is blamed for pulling the plug.
Retire without drama
Some agents should end. The process was redesigned, a better tool arrived, or the value no longer justifies the effort. Retiring is a normal outcome: revoke credentials and permissions, archive the logs under your retention rules, tell the people who relied on it, and make sure the manual fallback still works. Then note the reasons so the next project learns from them.
A one-page summary
- One named owner and a deputy, reviewed whenever roles change.
- A monthly one-hour check: volume, corrections, incidents, cost, changes.
- A quarterly review that ends with a recorded decision.
- Changes tested and reversible, permission changes treated as the biggest.
- Written triggers for pausing, adjusting and retiring.
An agent that is reviewed regularly earns trust with evidence. One that is not reviewed is simply running on the memory of how well it once worked.
