Sooner or later a trial of an AI tool does not deliver what was hoped. The assistant answered too many questions wrongly, the team stopped using it after two weeks, or the time saved was smaller than the time spent checking the output. The project lead then has an uncomfortable task: explain it to the people who approved the trial.
This article is for project leads and team managers who have to report such a result. It does not discuss why AI projects fail in general; for that, see why most AI projects fail and how to change that. It describes one practical document: a one-page lessons record that leadership can read in three minutes and act on.
Why a short record works better than a long explanation
Leaders rarely need the full story of the trial. They need to know four things: what you wanted, what happened, what the company now knows that it did not know before, and what you suggest doing. A long narrative invites two bad reactions. Either it sounds like a defence, or it hides the main point on page six.
A short, structured record also changes the tone. It treats the trial as a cheap way to learn, which is what a trial is for. A trial that ends with a clear "not this way, and here is why" has done its job.
The four parts of the record
Keep everything on one page. Use plain sentences, not slides.
1. Goal
State what the trial was meant to achieve and how success was to be judged, using the words agreed before the start. For example: "Reduce the time to answer routine customer questions, judged by average handling time over four weeks." If no criterion was written down at the time, say so plainly and state what you used instead. That is itself a lesson.
2. What happened
Give the result next to the goal, in the same measure. Two or three facts are enough: how many people took part, for how long, what you measured and what the numbers showed. Include what went well. Mention anything that changed during the trial, such as a different group of users or a tool update, because it affects how far the result can be trusted.
Be careful to separate what you measured from what you think. "Handling time did not fall" is a fact. "People did not like the tool" is an opinion unless you asked them.
3. What we learned
This is the most valuable part. Write three to five findings, each one sentence and each tied to something observed. Good findings are specific enough to change a later decision:
- "The tool worked for questions with a written answer and failed for questions that needed a look at the customer's order."
- "Staff were never shown when to trust the output and when to check it."
- "The two weeks of setup took longer than planned, so only two weeks remained for real use."
Describe conditions and decisions, not individuals. "The scope was too wide for a four-week trial" is useful. "The team did not try hard enough" is neither fair nor helpful, and it makes the next volunteer reluctant.
4. What next
Finish with a recommendation leadership can approve or decline. There are usually only four honest options:
- Stop: the idea does not fit; record why, so nobody repeats it blindly.
- Narrow: repeat on a smaller, better-defined task.
- Change a condition: for example, add training, better source material or a clear checking step, then try again.
- Continue as is: only if the result was partly positive and the remaining question is small.
Say what it would cost in time and who would own it. A recommendation without an owner is only a wish.
How to hand it over without a search for someone to blame
The way you deliver the record matters as much as the page itself.
- Send it before the meeting. Leaders who read it first ask better questions and are less likely to react to a surprise.
- Open with the result. Do not build up to it. "The trial did not reach its goal; here is what we learned and what we suggest" sets a calm frame.
- Share credit and limits. Name what the team did well and what was outside its control, such as the time available or the data it was given.
- Keep decisions separate from people. If the real cause was a decision, such as a rushed start or a vague goal, write it as a decision that can be improved, and include your own part in it.
- Ask for one decision. End the meeting with a yes or no on the recommendation, not with a general discussion.
A short example
Imagine a small distribution company that tried an AI assistant to draft replies to supplier emails, with the goal of cutting drafting time. After four weeks the average time per reply was about the same, because staff re-read and rewrote every draft. The one-page record says this in the first lines. Under "what we learned" it notes that drafts were good for routine confirmations and weak for price disputes, and that nobody had defined which emails the assistant should handle. The recommendation is to narrow the next trial to routine confirmations, with a named owner and a two-week check. Leadership approves it in the meeting. The example is illustrative and does not describe a real client.
What to do this week
If a trial has recently ended, open a blank page and write the four headings. Fill in the goal and the result first, since they are the easiest to state. Then write down the three most useful things you now know, and one recommendation with an owner. If you are planning a new trial, write the success criterion before it starts, so the record will be easy to write at the end. A good place to continue is how to review a pilot and decide whether to expand, adjust or retire it.
Sources
- Our related articles linked above.
- The approach described is a general working practice, not a standard; adapt it to your organisation and to any internal reporting rules. The example is illustrative and does not describe a real client. No figures or client results are claimed in this article.