Customer support looks like an ideal job for an AI agent. Questions repeat, answers mostly exist in a help centre, and customers value a quick reply. The risk is just as clear: a confident wrong answer about price, delivery or a contract becomes a statement your company made.
This article covers three decisions that matter most once an agent talks to real customers: what it may answer, when it must pass the case on, and how you find out whether the answers are any good. It complements our guides on when an agent should hand a case to a person and on what to do after an agent makes a mistake with a customer. It is a practical outline, not legal advice.
Write down the scope of answers
"Answer customer questions" is too vague to be a control. Start from your last few hundred real tickets and sort them into three groups:
- Answer on its own: questions with a stable, documented answer, such as opening hours, how to reset a password, where an order is, what a plan includes.
- Answer and verify: questions that depend on the customer's own data, such as order status or invoice details. The agent may answer only after it has looked the data up in your system, never from memory.
- Never answer alone: refunds and exceptions, complaints, legal or contractual interpretation, health or safety, anything about another customer, and any request to change an account's ownership or payment details.
Keep this as a short document with an owner and a date, not as a sentence buried in a prompt. When a new topic appears, someone decides which group it belongs to. Until then it belongs to the third.
Give the agent approved sources, and let it say "I don't know"
Most wrong answers come from the agent filling a gap with something plausible. Reduce that in two ways.
First, restrict the sources. The agent should answer from a defined set: the help centre, price and policy pages, shipping and returns terms, and a small set of internal answers your team has approved. Each source needs an owner and a review date, because an outdated page is an outdated answer. Our guide on preparing company documents for an AI agent describes how to do this.
Second, make the right behaviour when the source has no answer explicit: say it does not know, offer a person, and do not guess. Test it. Ask questions your documents do not cover and check that the agent declines instead of inventing a policy. A good answer also points back to the source, for example with a link to the relevant help article, so the customer and your team can check it.
Separate informing from committing
An agent explaining your returns policy is informing. An agent saying "we will refund you in full" is committing the company. Courts and regulators in several countries have treated what a company's chatbot says as the company's statement, so assume it counts.
Practical rules:
- Give the agent no authority to promise money, discounts, delivery dates or exceptions. It can collect the facts and prepare the request for a colleague.
- If it can take an action, such as cancelling an order, limit it to a few reversible actions with clear conditions, and confirm with the customer before doing it.
- Never let it quote prices or terms from memory; it reads them from the current source each time.
- Let it state what happens next and when, such as "a colleague will reply by tomorrow noon", only if that is a promise your team can keep.
If the agent talks to customers, tell them it is an AI and how to reach a person. Rules differ by country, so check what applies to you, and see our note on telling customers about an AI agent and offering a human alternative.
Define the triggers for escalation
An agent left to decide alone will often keep trying for too long. Write triggers the system checks, and do not rely on the agent's own sense that it is in trouble:
- The customer asks for a person. Always honour it, without a second attempt to persuade them.
- Signs of frustration: strong language, repeated complaints, the same question asked a third time.
- The topic is in the "never alone" group, or is sensitive: a complaint, a legal threat, a safety concern, a vulnerable customer.
- Low confidence or no source: the agent finds nothing, or finds conflicting answers.
- A loop: more than a set number of turns without resolution.
- High value: a large order, a key account or a contract renewal.
A handover is only as good as what travels with it. Pass on the summary, what the customer wants, what the agent already told them, the data it looked up and the reason for escalation, so the customer does not repeat everything. Set a response time for the human queue and a fallback when nobody is available, such as an honest message with a time of reply. A customer who was promised a person and then waits a day is worse off than if there had been no agent.
Measure the quality of answers
Resolution rate alone is misleading, because an agent that closes tickets with wrong answers looks successful. Look at a small set of measures together:
- Correctness: is the answer right according to the source? Review by a person on a sample.
- Appropriate escalation: did it escalate when it should, and avoid escalating simple cases?
- Tone and clarity: would you be comfortable if this reply appeared under your name?
- Repeat contact: did the customer return within a few days with the same problem?
- Customer feedback: a simple thumbs up or down, with the comments read, not only counted.
Every week, a support lead reads a sample, for example 30 conversations, with some chosen at random and some from escalations, low ratings and long threads. Score each on a short scorecard and note the cause of any miss: the source was wrong, the source was missing, the agent misread the question, or a trigger did not fire. Fix the cause in that order. In most cases the fix is a better article or a new rule, not a new model.
Before launch, build a fixed set of 50 to 100 realistic questions, including tricky ones, and rerun it after every change to the sources, the prompt or the tools. The OWASP list of risks for LLM applications is a useful source for adversarial cases, such as a customer pasting text that tries to change the agent's instructions or extract data about other customers. For a broader approach to managing these risks, see the NIST AI Risk Management Framework.
Keep the records and the data under control
Conversations contain personal data. Decide what the agent may see, how long the transcripts are kept and who can read them; our article on data handling and retention for customer data covers this. Keep a log of each conversation with the sources used and the actions taken, as described in what to record in an agent activity log. When a customer disputes what they were told, the log is your evidence.
Also decide who can pause the agent. If a source turns out to be wrong or a pricing page changes, someone must be able to switch off a topic, or the whole agent, within minutes.
A short checklist
- A written list of topics the agent answers, answers after lookup, and never answers alone.
- Approved sources with owners and review dates, and a tested "I don't know" behaviour.
- No promises of money, exceptions or dates without a person.
- Escalation triggers enforced by the system, with the full context passed on and a human response time.
- A weekly review of a conversation sample and a fixed test set rerun after each change.
- Logs, a data retention rule, and a way to pause a topic quickly.
Conclusion
A support agent earns its place by being predictable. Start with a narrow scope of common, well-documented questions, make escalation easy and quick, and review real conversations every week. Widen the scope only when the review shows that the answers are right and that customers who need a person reach one.
