A new tool is announced, people attend a short training, and in the first weeks the numbers look fine. Then attention moves elsewhere. Three months later, nobody can say with confidence whether the tool is part of daily work or only a place where people go when someone reminds them.
This article is for a head of a department who owns the result of a recent rollout: a new internal system, a shared platform or a replacement for a spreadsheet. It does not repeat the general advice on managing change, which we cover in our guide to technology adoption. Here the question is narrower: how do you find out, with little effort, whether the tool is really used?
Why three months
In the first two weeks people use a tool because they were told to and because someone is watching. After about three months that pressure is gone. What remains is habit or avoidance. This is the earliest point where the picture is honest, and still early enough to change something cheaply. If you already ran the plan from the first 30 days after launch, this check is the natural next step.
Pick a date when the tool goes live and put the review in the calendar right away. A review that has no date tends not to happen.
Signal 1: People who use it every week
Start with one number: how many people used the tool in a normal week, compared with how many are expected to. Not how many have an account, and not how many logged in once.
Most tools can show this in an admin screen or a simple export. If yours cannot, ask each team lead to count the people who worked in it last week. The count will not be perfect, but it is good enough to see whether it is closer to one in ten or nine in ten.
Signal 2: Repeat use
Compare the people who used the tool in week one with those who still use it now. A tool that many people tried and few kept has a different problem from a tool that few people tried. The first usually points to something annoying in daily use. The second points to awareness or access.
Look at the pattern over time, week by week. A gentle decline is normal. A sharp drop after the first fortnight is a sign that something in the daily routine does not work.
Signal 3: Real work happens inside it
Logins are a weak signal. What matters is whether the work the tool was bought for is actually done there.
Choose two or three typical pieces of work: an order, a request, an approval, a report. Ask whether the latest ones were created and finished in the tool. Open five recent items at random and check them. If the records are empty, half filled or created days after the work was done, people treat the tool as paperwork rather than as the place where work happens.
Signal 4: Workarounds that survived
Look for what is still running next to the tool: the old spreadsheet that is still updated, the email thread where decisions are made, the paper list on a desk. Ask directly, in a friendly way, "What do you still do outside the tool?"
A workaround is information. It often shows a step the tool handles badly, a missing field or a report people need and cannot get. Write each one down with the person who uses it and the reason. Do not treat it as disobedience at this stage.
Signal 5: What people say
Numbers show what happens, not why. Talk to four or five people from different roles for ten minutes each. Three questions are enough:
- What do you use the tool for in a normal week?
- What do you avoid using it for, and why?
- What would make your day easier if it changed?
Include at least one person who uses it rarely. They will tell you things the enthusiastic users have stopped noticing. Also listen for support questions: if the same question comes up again and again, the cause is usually a missing instruction or an unclear screen, not a lazy user.
A short example
Imagine a department of 20 people that moved its internal requests from email to a shared tool. At three months the owner checks the five signals. Twelve people used it last week, which is more than half. Of those who used it in week one, about four out of five still do. Of five recent requests she opens, three are complete and two were entered a week late. Two colleagues still keep a spreadsheet, because the tool cannot show requests by customer. In the short conversations, people say that the approval step takes too many clicks.
The numbers are invented for illustration, but the conclusion is realistic: the tool is used, though not fully. The owner has two concrete causes, a missing view and a clumsy step, not a vague feeling that "adoption is weak".
If the tool is not used
When the signals are poor, resist the urge to send a reminder to everyone. Work through causes in this order:
- Access. Can everyone who needs it log in, from where they work?
- Fit. Does the tool cover the real task, or only part of it? Fix the biggest gap first.
- Effort. Is it slower than the old way? Remove one step, one field or one screen.
- Habit. Is the old way still allowed? If requests still arrive by email and get answered, nobody has a reason to change.
- Ownership. Does someone answer questions and collect requests for changes? If not, see who owns an internal tool.
Change one cause at a time and check again after a month. If the fit is wrong and cannot be fixed, it is also a legitimate result to decide that the tool should be replaced or dropped. That is a cheaper decision at three months than at three years.
What to avoid
- Measuring only logins and reporting success.
- Making the review an inspection of individuals. The goal is to find what is wrong with the tool or the process.
- Asking people for feedback and then changing nothing. Tell them what you fixed.
- Setting a target that is not tied to work, such as "everyone must log in daily".
What to do this week
Put a review date in the calendar, three months after go-live. Write down the five signals as a one-page checklist with the figure you expect for each. On the day, collect the numbers, hold four short conversations and choose one change to make. For a later view across a whole programme, see five measures for an annual review.
Sources
- Our related articles linked above.
- The five signals, the order of causes and the example are a general working approach, not a standard; adapt the timing and thresholds to your own tool and team. The figures in the example are invented for illustration.
