An AI agent has sent the wrong refund emails, overwritten a batch of records, or promised customers something your company cannot deliver. Your team has switched it off. That was the urgent part, and it is covered in AI agent incident response: kill switches, containment, and recovery.
This article is about what the manager does next. The agent is quiet, but customers are confused, records are wrong, and colleagues want to know whose fault it was. Three jobs remain: communicate, repair, and learn.
Name an incident lead and a single source of truth
Before anything goes out, decide who speaks for the company. Incident frameworks such as NIST's incident response guidance (SP 800-61 Rev. 3) assume a defined coordinator and defined roles, and the same applies here. You need:
- One incident lead who owns decisions and the timeline.
- One communications owner who writes to customers and staff. This can be the same person in a small company, but the role should be named.
- One shared incident document, with times in one time zone, what is known, what is not yet known, and what has been done.
The document matters more than it sounds. Without it, support, engineering, and management each tell a slightly different story, and customers notice.
Work out what the agent actually did
Do not size the damage from the complaints that have arrived. Customers report only what they notice. Instead, reconstruct the full list of actions from the agent's tool-call logs for the affected window: every message sent, record changed, payment triggered, or ticket closed. How to keep such logs without storing more customer data than you should is covered in observing AI agents in production without leaking data.
From that list, sort the actions into three groups:
- Clearly wrong and needing repair.
- Clearly correct, which need nothing.
- Uncertain, which need a person to review.
The third group is usually the largest. Assign it to named reviewers with a deadline, and say in the incident document when the review is expected to finish. If you have no action log at all, say so honestly to yourself first; it tells you the most important change to make later.
Tell affected people early and plainly
The pull is to wait until you know the full cause. Resist it. People affected by a mistake mostly want to know four things: what happened to them, what you are doing about it, what they need to do (often nothing), and who they can contact.
A useful first message is short:
- What happened, in one or two sentences and without technical language.
- Which of their data or transactions were affected, if you know. If you do not yet, say when you will.
- What you have done so far, for example "we stopped the automated process the same day".
- What happens next, and when you will write again.
- A named contact or a direct address, not a no-reply mailbox.
Avoid two things. Do not blame "the AI" as if it were a separate party; your company chose to deploy it, and customers will read that as dodging. And do not guess at causes or promise "this will never happen again". Say what you know and what you are changing.
Check your obligations before writing. If personal data was exposed, data protection law may require notification of a regulator or of the individuals within a fixed period, and contracts with business customers may have their own notice clauses. This is a question for your legal or compliance contact on day one, not after the first draft of the email. Regulated sectors have further duties; ask someone who knows yours.
Also tell your own staff. Support teams who learn about the problem from an angry customer cannot help anyone. Give them a short briefing and the approved wording, and tell them where to send questions they cannot answer.
Make it right
Repair has two parts: fix the data and fix the relationship.
Fix the data from the reviewed action list. Prefer reversing a change through the system's normal audit-friendly path over editing records by hand, and record each correction against the incident. After repair, check a sample of fixed records and, where it is cheap, all of them.
Fix the relationship by deciding in advance what a fair remedy looks like: a refund of a wrong charge, a corrected invoice, a service credit, or a personal call to a major account. Agree the range of remedies with finance before support starts improvising. A modest, quick, consistent remedy usually does more than a generous one that arrives weeks later.
Keep the agent off, or running with reduced authority, until the repair is under control. Restoring it in stages is described in the incident-response article linked above, and the stage design is the same one used for a first launch; see a staged rollout plan for AI agents.
Run a blameless post-mortem
Within a week or two, while memory is fresh, hold a post-mortem. Google's SRE material on postmortem culture describes the principle: assume that people acted reasonably with the information they had, and look for the weaknesses in the system that let a mistake do harm. Atlassian's guidance on blameless post-mortems makes the same case for ordinary operations teams. Blame teaches people to hide problems; a blameless review teaches the organisation.
For an AI agent, ask questions at four layers rather than stopping at "the model got it wrong":
- Task design. Was the agent given a job that needed judgement a person should have kept? Was the instruction ambiguous?
- Permissions. Could it do more than the task needed? Would a lower limit, a smaller batch size, or an approval step have contained the damage? See identity and least privilege for AI agents.
- Detection. How long did the problem run before anyone noticed, and who noticed first: your monitoring, your staff, or a customer?
- Testing and release. Which case would have caught this, and was it in the regression set? Was the change released with the shadow period and thresholds you agreed?
Write the timeline from the incident document, add a short "what we learned" section, and finish with actions. Each action needs an owner, a date, and a way to tell it is done. "Be more careful" and "retrain the team" are not actions. "Add an approval step above 50 changed records by the end of next month, owned by the operations lead" is.
Close the loop
Share a summary of the post-mortem with the people who were affected or who worked on it, with confidential details removed. Add the failing case to your tests. Review the actions at the next scheduled meeting until they are done, and note the incident in your risk register so that future decisions about agent authority can refer to it.
NIST's AI Risk Management Framework and the public AI Incident Database both reflect the same idea: failures of AI systems are a normal subject for organised learning, not a secret to manage. Companies that handle the first mistake well tend to earn more trust than those who never had one to explain.
A short checklist for the first week
- Who is the incident lead, and where is the shared document?
- Do we have the full list of what the agent did in the affected window?
- Has someone checked notification duties with legal or compliance?
- Have affected people heard from us, with a contact and a date for the next update?
- What remedy is approved, and who delivers it?
- When is the post-mortem, and who will own each resulting action?
