Sit in on any month-two agent rollout review and listen to what actually gets reported. The dashboard is green, p95 latency sits under four seconds, tool-call errors are running at 0.3% — and then the operations lead mentions, almost as an aside, that her team is still working the queue by hand.
That gap between a healthy deployment and an empty invocation log is the most common way enterprise agent programs stall out. The agent works exactly as built; the humans upstream of it never routed anything into it.
Agent observability instruments the agent outward — traces, spans, tokens, tool calls. Adoption instrumentation measures whether the workflow routed work to the agent in the first place.
What follows is the instrumentation layer that closes the gap: four metrics, the events they require, where those events come from, and what the numbers should read at week two versus week eight.
Why Your Observability Stack Cannot See Adoption
Every signal in a standard agent observability stack begins at the moment of invocation. Traces open when a request arrives, spans close when a tool returns, and the entire picture is assembled from inside a run that already exists.
The denominator is what goes missing. If your support agent resolved 412 Zendesk tickets in July, that number reads as a win when the eligible queue held 500 and as a rounding error when it held 9,200.
What's more, the missing denominator hides in the direction that flatters the program. Success rate, p95 latency, and cost per run all improve as volume falls, so a rollout nobody uses produces the cleanest dashboard you will ever ship.
A falling invocation count improves every observability metric at once — success rate, p95 latency, and cost per run. Low adoption and excellent engineering look identical on an agent dashboard.
Keep in mind that this is a measurement gap rather than a tooling failure. Your traces are answering a narrower question than the one your CFO asked when she approved the agent business case.
The Four Metrics That Define Adoption
Adoption is measured against the process, not against the agent. All four metrics below depend on one event your pipeline almost certainly does not emit today — a signal that fires when work becomes eligible for the agent, whether or not the agent ever sees it.
Invocation Coverage Per Process Step
Invocation coverage is the share of eligible instances of one specific process step that were actually routed to the agent. Measure it per step, never per workflow.
A six-step process with one agent-enabled step will report cheerful workflow-level “usage” while five-sixths of the labor stays manual. In fact, workflow-level adoption reporting is the single most common way a stalled rollout survives its first quarterly review.
Invocation coverage is agent invocations divided by eligible instances of a single process step, segmented by team and role. Measured at the workflow level, it hides exactly which step is being bypassed.
Segment the number three ways from day one: by team, by role, and by shift. Aggregate coverage of 48% routinely decomposes into one team at 91% and three teams at 4%, and those two readings call for entirely different interventions.
Bypass Rate
Bypass rate counts eligible work a human completed manually while the agent was available and healthy. It is the arithmetic inverse of coverage and a different thing in diagnosis, because bypass only means something when you can prove the agent was up.
Accordingly, join bypass events against the agent's uptime record and agent SLO history before reporting the number. Work that routed around a degraded agent is a reliability finding; work that routed around a healthy one is an adoption finding.
Bypass also needs a reason code, and the cheapest place to collect one is the interface the human is already sitting in. A single required picklist on the manual path — “faster by hand,” “it was wrong last time,” “did not know it applied,” “needed to see the source record” — turns a flat curve into a work list inside of a week.
Escalation-To-Human Ratio
Escalation-to-human ratio is the share of successful agent runs whose output a person then reviewed, edited, or redid. A run can return cleanly, pass its validation suite, and still consume more human minutes than the manual path it was meant to replace.
Escalation-to-human ratio measures successful runs a person still had to touch. It separates technical success from workflow value, which no error rate or eval score captures on its own.
Split escalations into two buckets, because they have opposite fixes. Threshold escalations — the agent declining low-confidence work by design, usually through your agent approval gates — are a tuning exercise, while silent rework points at a hole in your agent evaluation suite.
Silent rework is harder to instrument and worth more when you catch it. Diffing the agent's proposed output against the value ultimately committed in Salesforce or ServiceNow, then counting edits above a similarity threshold, gets you most of the way there without asking anyone to fill in a form.
Time-To-First-Use Per Role
Time-to-first-use is the number of days between a named user receiving access and that user's first successful invocation. Measured per role — claims adjuster, tier-2 support, AP clerk, field supervisor — it is the earliest reliable predictor available to you.
The distribution is reliably bimodal. People either try the agent inside the first week or they never open it at all, and the middle of the curve is thin enough to plan around.
Time-to-first-use per role predicts adoption earlier than any volume metric. Roles with no first invocation by day 14 rarely convert on their own and need a targeted intervention rather than another announcement.
Observability Signal Versus Adoption Signal
The table below maps the questions an infrastructure leader gets asked in a month-two review against what each telemetry layer can actually answer. Of course the left column is not wrong — it is simply answering a narrower question.
| Review question | Observability answers | Adoption instrumentation answers |
|---|---|---|
| Is it working? | 99.4% success across 1,240 invocations | Whether 1,240 was 8% or 80% of eligible steps |
| Is it fast enough? | p95 of 3.1 seconds, time-to-first-token under 900ms | Whether reps waited or opened Zendesk and typed the reply themselves |
| What did it cost? | $4,180 in Bedrock spend last month | Cost per eligible process step, including the steps that bypassed |
| Is quality holding? | 0.3% tool failures, evals passing | Escalation-to-human ratio on the runs that succeeded |
| Who is using it? | Invocation counts by service account | Coverage and time-to-first-use by team, role, and shift |
Read the two columns side by side and the month-two conversation changes shape. The left column tells you whether to page the on-call engineer; the right column tells you whether the investment is being realized.
How To Instrument Eligibility Without Rebuilding The Pipeline
The entire layer rests on one new event type. An eligibility event fires from the system of record the moment a unit of work reaches the step your agent was built for, and it fires whether or not the agent is ever called.
Emit it from where the work lives rather than from the agent runtime. A Salesforce platform event, a Zendesk trigger, or a ServiceNow business rule published onto EventBridge costs an afternoon and survives every future change to your orchestration layer.
From there the join is mechanical. Land eligibility and invocation records in the same store — DynamoDB for hot lookups, Snowflake for the weekly rollup — and correlate them on a workflow-instance ID that both paths carry.
Emit an eligibility event from the system of record when work reaches the agent's process step, then join it to the invocation log on a shared workflow-instance ID. That join is the adoption layer.
Correlation-ID discipline is where these builds usually break. If your agent handoff patterns mint a fresh ID at every hop, the eligibility record and the invocation record never meet, and month three gets spent writing fuzzy matchers against timestamps.
Instrument the human path at the same fidelity as the agent path. The manual completion event needs the same workflow-instance ID, the same actor field, and the same timestamp precision, or your bypass number is an estimate rather than a count.
Retention is a smaller ask here than for the agent's own logs. Adoption analysis needs identifiers, timestamps, actors, and outcome codes rather than payloads, so it fits inside whatever your agent data retention policy already permits.
What The Numbers Look Like In A Healthy Rollout
The shape of the curve matters more than any absolute target, but operators need something to hold that first review against. The ranges below describe a single well-scoped process step with a defined eligible population — not a department, and not a platform.
- Week two. Coverage of 15% to 30%, bypass at 40% to 70%, and roughly half of named users past first invocation. Early adopters carry the whole number at this stage, which is expected.
- Week four. Coverage of 35% to 55%, with every named role represented at least once. A role still sitting at zero here is the most actionable signal on the board.
- Week eight. Coverage of 60% to 80%, bypass under 25%, and escalation-to-human trending down week over week. This is the first point at which an ROI number is defensible.
- Week twelve. Coverage stable inside a 10-point band, with bypass concentrated in one or two legitimate reason codes. Stability matters more than the ceiling.
Note that a step which plateaus at 65% coverage with clean reason codes is a finished rollout rather than a failed one. Some share of eligible work genuinely belongs to a human, and the reason codes are what let you prove which share.
The month-two review needs four numbers on one slide: coverage per step, bypass rate with reason codes, escalation-to-human ratio, and the count of roles still at zero invocations.
Any review that opens with latency and cost is measuring the agent instead of the rollout.
What A Flat Curve Is Usually Telling You
Four causes account for most flat month-two curves, and they present almost identically on an agent dashboard. Telling them apart takes the segmented numbers above rather than another all-hands demo.
- Entry-point friction. The agent lives in a separate console while the work lives in ServiceNow, so using it costs a context switch nobody budgeted. Coverage runs high for the pilot team that helped build it and near zero everywhere else.
- Trust debt from an early bad run. One wrong answer in week one propagates through a team faster than any enablement session, and the bypass reason codes will say so in plain language. The remedy is a visible correction plus a published change, not a reminder email.
- Incentive mismatch. Teams measured on average handle time route around anything that adds a review step, even when it saves twenty minutes downstream. This shows up as solid coverage on complex items and near-zero coverage on the fast ones.
- Access gaps. A role that never appears in the invocation log frequently never received the entitlement, which your agent identity and access records will confirm in about ten minutes. Check this before concluding anything about willingness.
All four are org-readiness problems wearing a technical costume. That said, only the instrumented version of the question tells you which one you are holding.
Wiring Adoption Into The Operating Cadence
Adoption metrics decay into a vanity dashboard unless someone owns the number every week. Put coverage per step on the same review where you already read release health and incident counts, and give the business owner — rather than the platform team — the target.
The platform side owns the pipeline, the correlation-ID contract, and the freshness of the rollup. That division belongs in your agent operating model alongside the on-call rotation and the release calendar.
Finally, treat a flat curve as a change-management finding with a technical remedy attached. The playbooks in AI change management only work once you can name which team, which role, and which step is bypassing — and that naming is precisely what this layer produces.
Frequently Asked Questions
When should you start tracking agent adoption metrics?
Before go-live. Capture 30 days of baseline volume per process step so month two has a denominator; teams that instrument after launch cannot separate a flat curve from a small addressable population.
What is a healthy bypass rate in month two?
Expect 40% to 70% in week two and under 25% by week eight for a well-scoped step. A rate holding above 50% past week six is an entry-point or trust problem rather than a model problem.
How do you emit eligibility events without changing the agent?
Fire them from the system of record — a Salesforce platform event, a Zendesk trigger, or a ServiceNow business rule — onto EventBridge, then join to the invocation log on a shared workflow-instance ID.
Does a high escalation-to-human ratio mean the agent is broken?
No. Split escalations into tool failures and confidence-threshold handoffs; a rising ratio against a flat error rate usually means thresholds are tuned conservatively or reviewers do not trust the output yet.
Who owns agent adoption metrics, platform or the business unit?
Platform owns the pipeline and the correlation-ID contract; the business owner owns the coverage target and reports the weekly number. Splitting it any other way leaves the flat curve unowned for a quarter.
Working Session
If you are eight weeks into a rollout with a green dashboard and a flat usage curve, the team at iSimplifyMe instruments and operates production agent systems across CRM, ticketing, and data warehouse environments every week. Reach out for a working session — we will map your eligible-work events, stand up the coverage and bypass rollup against your existing warehouse, and leave you with the four numbers your month-two review actually needs.