Twelve months after a new system, process or tool goes live, someone in leadership asks whether it was worth it. If the team answers with a long slide deck of activity figures, the question stays open. If it answers with five numbers, a baseline and an honest list of caveats, the discussion moves to what to do next.
This article gives a template for that review. It is deliberately short. For the wider framework of metrics across a whole transformation programme, see measuring digital transformation; here we look at one change and one annual conversation.
Start with the baseline
The review is only as good as the "before" picture. Ideally you measured the same things in the month before the change. If you did not, reconstruct what you can from old reports, tickets or a sample of past cases, and mark those figures as estimates. Our article on measuring the cost of a manual process in a week shows a quick way to do it.
Compare like with like: the same process, the same definition of "done", the same way of counting.
Five measures for the annual review
1. Time
How long does the work take now, from start to finish? Use elapsed time and, where you can, hands-on time. Report the median and a slow case, not only the average.
2. Errors and rework
How often does the work come back for correction? Count returned cases, corrections and complaints about the output. A change that is faster but produces more rework has moved the cost, not removed it.
3. Real use
How much of the work actually goes through the new way, and how much still runs in spreadsheets, side channels or by phone? High login counts are not enough. Compare cases handled in the new system with all cases of that type.
4. Effect on the customer or the next team
Choose one outcome that the receiving side can feel: response time, delivery time, number of follow-up questions. If the change was internal, the "customer" is the next team in the chain. Our guide on why digital transformations stall explains how often this link is missing.
5. Total running cost
Add up licences, hosting, support, internal time to maintain the system and any extra work it created. Compare it with the cost of the old way for the same volume. This is the number a CFO will look for first; see also what AI return is realistic after 3, 6 and 12 months for the same discipline applied to AI projects.
| Measure | Before | After 12 months | Note for leadership |
|---|---|---|---|
| Time per case | baseline | current median | state the sample size |
| Rework rate | baseline | current rate | state how it is counted |
| Share of work in the new way | n/a | current share | what stays outside, and why |
| Customer or next-team effect | baseline | current value | one outcome only |
| Total running cost per case | baseline | current cost | include internal time |
Separate the change from everything else
In a year many things move at once. Before you credit the change with an improvement, write down the other factors and check each one:
- Volume. Did the number of cases rise or fall? Compare per case, not in total.
- Season. Compare the same months, not the busiest with the quietest.
- People. Were there new hires, departures or training that affect speed?
- Other projects. Did another change touch the same process?
- Definitions. Did anyone change what counts as an error or a finished case?
If part of the organisation did not get the change, or got it later, compare the two groups. That is the strongest evidence you will usually have. If there is no such group, say that the result is "consistent with" the change rather than "caused by" it.
Worked example (invented numbers, for illustration only). A team introduces an online request form to replace e-mail. In the month before, a sample of 50 requests took a median of 4 working days, and 9 of the 50 had to be sent back for missing information, 18%. After twelve months, a sample of 50 shows a median of 3 days and 4 returned, 8%. Request volume grew by a quarter in the same period and one experienced colleague left. The team reports: median time down by one day, returns down from 18% to 8%, despite higher volume. It adds that 70% of requests now come through the form and the rest still arrive by e-mail. It does not claim the one-day gain is due to the form alone, since staffing changed too.
What not to promise
- A precise return figure when the baseline was an estimate. Give a range and show how it was built.
- Future savings based on one good quarter. Report the year you have.
- Credit for effects you did not measure. Morale, culture and agility are real, but without a measure they belong in the narrative, not in the table.
- A perfect picture. Leadership trusts a review that names its weak spots.
Common mistakes
- Reporting activity (logins, tickets closed) instead of outcomes.
- Changing the definition of a measure halfway through the year.
- Presenting averages only, so one slow month hides a real improvement.
- Leaving out internal time spent maintaining the system.
- Ending the review without a decision: keep, adjust, extend or stop.
Where to start this week
Pick one change that is about to reach its first anniversary. Find the baseline, or sample 30 to 50 recent cases and 30 to 50 old ones. Fill in the five-row table, write the list of other factors and add one paragraph on what you cannot prove. End the review with a single recommendation for leadership. If you are still planning the change, take the baseline now; it is the cheapest part of the review.
Sources
- Your own process records, samples and cost data, collected as described; all figures in the worked example are invented for illustration.
- Our related articles linked above.