Most companies that deploy an AI agent say the same sentence: "a human stays in the loop." Few can say which human, at which step, looking at what, and where it is written down afterwards. This article gives a manager a way to answer those four questions.
It assumes you already limit what the agent is allowed to do; see identity and least privilege for AI agents. Approval is the layer for actions the agent is allowed to attempt, but where you want a person to decide.
Decide by consequence, not by confidence
Asking a person to approve everything defeats the purpose of automation. Asking them to approve nothing leaves one mistake free to reach a customer. A workable middle is to sort actions by what happens if they are wrong:
- Money. Payments, refunds, discounts, purchases, contract changes. Set a threshold; above it, a person approves.
- Irreversibility. Deleting records, sending external email, publishing, closing accounts. If it cannot be undone cheaply, a person approves.
- Outside reach. Anything sent to a customer, partner, or the public in the company's name.
- Personal or confidential data. Exports, sharing, or changes to sensitive records. See data handling for AI agents.
- Unusual volume. A batch of fifty changes is a different risk from one change, even if each is routine.
Reading, drafting, summarising, and changing internal working copies usually need no approval. Keep that list explicit and short, and write it down as a table: action, who may approve, threshold. OWASP lists "excessive agency" among the main risks of LLM applications for this reason: the agent has more freedom than the task needs, and nobody is asked before it uses it.
Do not use the agent's own confidence as the trigger. A model can be wrong while sounding certain, and the rule is easier to audit when it depends on facts such as amount, recipient, and reversibility.
Design approvals people do not rubber-stamp
An approval that is clicked in two seconds, every time, is theatre. Three design choices help.
Show the action, not the explanation. The approver should see the exact thing that will happen: the email as it will be sent, the refund amount and account, the records to be deleted. A summary written by the agent ("I will resolve the customer's request") hides the detail that matters.
Show the evidence next to it. Link the source the agent relied on, such as the ticket, the document, or the policy. The approver should be able to check in under a minute.
Make declining easy and useful. A reject button that asks for a short reason gives you both control and a source of improvement. If rejecting is awkward, people will approve.
Also choose approvers deliberately. The person needs the knowledge to judge the action, the authority to own the result, and enough time. An approver who is also measured on speed will approve quickly. For high-value actions, consider two people, and make sure the person who requested the work is not the only one who approves it; the same separation of duties you apply to payments applies here.
Be aware of injected instructions: text inside an email or web page can try to steer the agent into requesting something harmful, and a tired approver may let it through. See prompt injection controls for AI agents.
What the audit trail should contain
An audit trail answers a later question, perhaps from a customer, an auditor, or you: who allowed this, and on what basis? For each approval-controlled action, record:
- The request: what the agent proposed, with its exact parameters, and when.
- The context shown: the content or a stable reference to what the approver saw, including the version of any policy or prompt in force.
- The decision: approved, rejected, or changed, with the approver's identity, the time, and the reason if given.
- The execution: what was actually done afterwards, and the result, linked to the same identifier.
- Edits: if the approver changed the proposal, both versions.
Link all of it with one identifier, so a single search follows an action from proposal to outcome. Use one time zone for all entries.
Two properties matter more than the details. First, the log must be append-only and out of the agent's reach: the agent should not be able to write, edit, or delete its own approval records. NIST's guidance on log management (SP 800-92) covers protecting log integrity and retention in general terms. Second, approvals must be bound to the action: if the proposal changes after approval, the approval no longer counts and a new one is needed. Otherwise a person approves one thing and another is executed.
Keep personal data in the trail to the minimum needed, and decide a retention period with your legal or compliance contact. A log you keep forever and fill with customer details becomes its own risk; observing AI agents in production without leaking data shows how to log usefully with less exposure.
Review whether the control works
Approval is a control that decays quietly. Once a month, look at a few figures:
- Approval rate and time to decide. Nearly 100% approved in a few seconds suggests rubber-stamping, or that you ask for approval on things that do not need it.
- Rejections and edits, by reason. These show where the agent is weak. Feed them into your test set; see evaluating AI agent reliability before production.
- Queue age. If requests wait for hours, people will start to look for ways around the approval, or the agent's usefulness drops.
- Actions that should have asked but did not. Sample executed actions above your thresholds and confirm each has an approval record.
If a type of action is approved without edits for several months and the stakes are modest, you can raise the threshold or let it run alone, with sampling instead of approval. Move the other way after an incident. This is the same staged approach used for a first launch, described in a staged rollout plan for AI agents.
Regulation may set a minimum
If your agent falls under rules for high-risk AI systems, human oversight can be a legal requirement, not a choice. Article 14 of the EU AI Act, for example, sets out requirements for human oversight of high-risk systems. Whether your use case falls into that category is a legal question; ask early, because it affects what you must log and who must be able to intervene.
A one-page starting point
- A table of actions with approval rules: threshold, approver role, backup approver.
- An approval screen that shows the exact action and its sources.
- A log with request, context, decision, execution, one identifier, and no write access for the agent.
- A named owner who reviews approval figures monthly.
- A date when you will decide which approvals can be relaxed, and which tightened.
