Measure adoption, not just implementation
Deploying an AI tool and reporting success are not the same thing; track active use rather than availability, because the gap between the two is where unvoiced resistance hides and where the investment fails to earn its return.
The natural metrics for an AI initiative are implementation metrics. Licences purchased. Staff trained. Systems integrated. Rollouts completed. They are easy to count, easy to report and easy to chart upwards. They are also almost entirely disconnected from whether the initiative is working.
The working rule is to track active use rather than availability. Daily active users, number of queries, types of tasks attempted, the proportion of licensed staff who return to the tool after the first week. Low adoption on any of these is information; it signals where Unvoiced staff resistance is the primary failure mode of AI initiatives is visible in the data, where infrastructure or knowledge gaps are blocking use, or where the tool was oversold relative to what staff actually need.
Why implementation metrics mislead
Implementation metrics are built for the programme management layer. They answer “did we do the thing?” rather than “is the thing working?”. For a conventional software rollout those two questions usually converge — if the system is installed and staff are trained, it gets used. For AI rollouts they diverge, because AI is fragile enough, counterintuitive enough and politically sensitive enough that installation does not produce use.
Reports built on implementation metrics tell leadership that the programme is succeeding at precisely the moments it is not. The gap between rollout completion and actual adoption can run for months, and the longer leadership operates on the implementation numbers, the less time there is to correct before budgets and patience run out.
What adoption metrics reveal
Three categories of signal emerge from adoption tracking.
Use that is actually happening, which identifies where AI is earning its keep. These are the cases to protect, resource and build on.
Use that started but stopped, which identifies where a first impression was good but sustained engagement did not hold. Usually the tool is fine and the supporting infrastructure — context, training, workflow fit — is not. See Useful AI is a context problem for why this is the usual diagnosis.
Use that never started, which identifies where leadership assumed a use case that staff did not share. These are the cases to de-scope or re-explain rather than push.
Also worth tracking: where is AI not being used that it could be? That question often highlights more important issues than the adoption data itself. It connects to Channel shadow AI use as signal, not risk to suppress — gaps between official and unofficial use are usually pointing at infrastructure that needs work, and the map of that gap is one of the most useful artefacts an adoption-measurement programme produces. The specific signal of staff routing around the AI to access the underlying source directly is informative enough to warrant its own decoding — see Treat AI-pilot bypass behaviour as evaluation data.
When to launch
Adoption measurement starts the moment the tool is in front of staff, not when leadership formally declares the rollout begun. The implication is a launch-readiness rule: do not put the tool in front of pilot users until it is genuinely useful for their work, because first-use frustration in change-fatigued cultures sends people back to old sources and is nearly impossible to recover from. The tool that disappointed a user on first encounter rarely gets a second chance, regardless of how much it improves later. Premature launch puts the adoption-measurement system in the wrong starting position; the metrics will read low and the diagnosis will be ambiguous, when the underlying problem was simply that the tool wasn’t ready yet.
The same rule applies to the artefacts surrounding the tool — onboarding flows, induction skills, the firm-context project — and not only to the tool itself. An onboarding artefact run end-to-end across the roles it claims to serve, with guard-rails deliberately probed before staff exposure, is the same discipline at one layer up; skip it and the artefact teaches the user on first encounter that the AI is unreliable, which is the failure the launch-readiness rule exists to prevent.