Sooner or later a customer, an employee or the owner asks the same question: what did the agent actually do? Without a record the honest answer is "we think it did the right thing". With a simple activity log the answer takes a few minutes and can be shown to the person who asked.
This article is for the owner or manager of a small or mid-sized company that already runs, or is about to launch, an AI agent. It does not need a monitoring team. For the technical side of tracing agents in production, see observing AI agents in production without leaking data. Here we stay with the management questions: what to record, who reads it, how long to keep it and what to do when something looks wrong.
Why a log matters more for an agent than for a tool
A normal program does the same thing every time. An agent chooses its steps, so two similar requests can end differently. When something goes wrong you need to see the sequence: what it was asked, what it looked at, what it did and what came back. Without that, every complaint becomes an argument about memory.
A log also protects you in the other direction. When the agent did the right thing, the record shows it. That helps in a customer dispute, in an internal disagreement and in any conversation with a client or regulator about how the agent is controlled. General security guidance treats logging as a basic control for exactly this reason: it lets you detect problems and reconstruct what happened (NCSC: Logging and monitoring).
What to record
Think in actions. For every task the agent performs, one entry should answer six short questions:
- When did it happen?
- Who or what asked? A named employee, a customer message, a scheduled job.
- What did the agent do? For example "looked up order status", "drafted a reply", "changed delivery date".
- In which system, on which record? An order number or ticket ID is enough; you do not need the full content.
- What was the result? Done, refused, failed, handed to a person.
- Did a person approve or change it? Who, and when.
Add the version of the agent's instructions or settings in use, so that you can tell later whether a strange result came before or after a change (see changing an AI agent after launch). Where the agent hands a case to a person, record the handover too; it is one of the most useful entries when a customer says "nobody got back to me".
The OWASP logging guidance gives a similar list of what makes an entry useful: the time, the actor, the event and its outcome (OWASP: Logging Cheat Sheet).
What to keep out of the log
More is not better. A log that copies every customer message, every attachment and every key becomes a second place where sensitive data lives, and often a less protected one.
- Secret keys and passwords: never. If one appears, treat it as a leak and replace it.
- Full personal data: refer to the record by ID rather than copying names, addresses or health and payment details.
- Whole documents: store a reference, not the content.
If you need the full text for a short time to investigate a problem, keep it in a separate place with stricter access and a short life.
Make it readable by the owner
A log made only for engineers will not be read. Ask whoever builds or configures the agent for a plain view: one line per action, in words, sortable by date, agent, system and result. A spreadsheet export is acceptable at the start. The test is simple: can the process owner, without help, pick a customer case from last Tuesday and see what the agent did?
Who reads it, and how often
A log without a named reader is an archive. Agree two things.
A daily glance, five minutes. The process owner looks only at failures, refusals and handovers to a person. Are there more than usual? Are the same errors repeating?
A weekly sample, twenty minutes. Pick five to ten ordinary entries at random and follow them to the real result: was the order really changed, was the reply really sent, did it say what it should? Sampling finds quiet problems that a list of failures never shows.
Write down who reads when the owner is away. A reader who is on holiday for two weeks is the same as no reader. The wider routine for keeping an agent in good shape is described in maintaining an agent after launch.
How long to keep it
Decide a period rather than keeping everything indefinitely, because old logs are a cost and sometimes a risk, especially when they contain personal data. A common starting point for a small company is a few months of full action records, for example three to six, and a shorter period for anything more detailed. Adjust it to your own situation: contracts, industry rules and the time in which customers usually raise complaints all matter. If you are unsure whether a rule applies to you, ask your lawyer or data protection contact once and write the answer next to the retention setting.
Also decide who can read the log, who can change it and who can delete it. The agent itself should not be able to edit or delete its own record.
What counts as suspicious
Agree the list before you need it. Typical triggers for a small company:
- The agent acted outside its normal hours or on a system it normally does not touch.
- A burst of actions, such as many messages or many changed records in a short time.
- A write, delete or send action where the agent is supposed to only prepare drafts.
- Repeated refusals or errors on the same task, which may mean someone is trying to push it off course.
- An action that no request in the log explains.
- Any entry that contains a secret key or unexpected personal data.
What to do when an entry looks wrong
Keep the response short and rehearsed, in the spirit of incident response and kill switches:
- Pause the agent, or the one integration involved, if more harm could follow. The log reader should be allowed to do this without asking for permission.
- Preserve the relevant entries; do not tidy them up.
- Check scope: which records, customers or systems were touched, and whether the effect can be reversed.
- Fix the cause, which may be the instructions, an access right that was too wide (see giving an agent only the access it needs) or a missing approval step.
- Tell the people affected, if their data or orders were involved, and record what you told them.
- Add a line to the log review notes so the same pattern is caught sooner next time.
A short checklist
- Every action has an entry: time, requester, action, system and record, result, approval.
- No keys, no full personal data and no whole documents in the log.
- The process owner can read the log without technical help.
- One named reader and a deputy; five minutes daily, twenty minutes weekly.
- A retention period is written down; the agent cannot edit its own log.
- The list of suspicious signs is agreed, and the reader may pause the agent.
If you cannot answer "what did it do last Tuesday?" today, start with the simplest version: one line per action, one reader, one weekly sample. It will be improved by use.
