Most teams that put an AI assistant in front of customers start with the tool and discover its limits from complaints. A calmer way is to decide first what it may do, in a form that the support lead, the assistant's vendor and a new colleague can all read in two minutes.
This article is for support leads at small and mid-sized companies. It covers one sheet and one weekly habit. It is not about choosing a tool; for that, see AI for customer service chatbots. For a wider treatment of scope, escalation and quality, see AI agent in customer support.
Start from your real questions
Export or skim the last two or three hundred support requests and group them by type. Do not invent categories; use the ones your team already uses. Most teams find that a handful of types cover most of the volume: opening hours and delivery information, how to use a feature, order or account status, complaints, billing disputes, and requests that need a decision.
The sheet: three columns
For each question type, decide which of three columns it belongs to.
Answer. The assistant replies on its own, using approved sources only. Typical examples are published information, step-by-step help from your documentation and the status of something it can look up.
Route. The assistant does not answer, but it collects the details and sends the case to the right queue. A billing question is a good example: the assistant can ask for the invoice number and pass it on without commenting on the amount.
Hand over. The assistant stops and a person takes over at once. This is for complaints, legal or safety topics, a customer who is clearly upset, and anything that asks for an exception, refund or promise.
A short extract might look like this:
| Question type | Column | Source or owner |
|---|---|---|
| Opening hours, delivery areas | Answer | Published help pages |
| How to reset a password | Answer | Help centre article |
| Where is my order | Answer, if the status can be looked up; otherwise route | Order system or support queue |
| Invoice looks wrong | Route | Billing contact |
| Refund or exception | Hand over | Support lead |
| Complaint, legal or safety topic | Hand over | Named person on duty |
The example is illustrative. Your table will differ, and it should be short enough to fit on one page.
Write limits as rules, not as feelings
"Be careful with sensitive topics" cannot be checked. Rules that can be checked look like this:
- The assistant answers only from the approved sources. If the source does not contain the answer, it says so and offers a person.
- It does not promise prices, dates, refunds or exceptions.
- It does not give legal, medical or financial advice.
- It never asks for a full card number or a password.
- It tells the customer that they are talking to an automated assistant.
Each rule should be something a colleague could apply to a sample answer and say yes or no.
The hand-over rule
A hand-over fails quietly when the customer has to repeat everything. Write down three things:
- Who receives it. A named person or a shared queue with an owner, with working hours stated. If nobody is on duty, the assistant should say when a reply will come, using the real time.
- What travels with it. The whole conversation, the customer's contact details and the reason for the hand-over.
- When it happens. After the customer asks for a person once, after two answers in a row that the customer rejects, and always for the "hand over" column.
Test this yourself before launch: start a conversation, ask for a refund and see who gets it and how long it takes.
The weekly sample check
Once a week, one person reads twenty answers, chosen at random from the past seven days, plus every conversation that ended in a hand-over or in a bad rating. It takes about thirty minutes. For each answer, record only four things:
- Was it correct according to the approved source?
- Did it stay inside the sheet?
- Was the tone acceptable?
- If it was handed over, did the right person receive it in time?
Keep the results in a simple table with the date, the number of answers checked and the number of problems. What matters is what you change afterwards: a corrected source page, a new rule on the sheet or a question type moved to another column. Write that change down next to the date.
When to widen the scope
Move a question type from route to answer only when several weeks in a row have passed without a problem in that type, and when the source it relies on has an owner who keeps it current. Move a type the other way as soon as a problem appears. Narrowing is quick and needs no meeting; widening is deliberate.
Common mistakes
- Launching with a sheet nobody outside the project has read.
- Letting the assistant answer from old pages that nobody maintains.
- Counting only how many conversations it closed, not how many answers were right.
- Reviewing only the answers customers rated badly; unhappy customers who say nothing are missed.
- Forgetting to update the sheet when a product, price list or policy changes.
A short checklist
- Every common question type sits in answer, route or hand over.
- Limits are written as rules that can be checked.
- A named person receives hand-overs, with the full conversation.
- A weekly sample of twenty answers is in someone's calendar.
- Every change to the sheet is dated.
If you also want to know who takes over when the assistant itself is replaced or retired, see retiring or replacing an AI agent safely.
Sources
- Our related articles linked above.
- The approach described is a general working practice, not a standard; adapt it to your organisation. The table and rules are illustrative and do not describe a real client. No figures or client results are claimed in this article.