Why does an AI code reviewer start in shadow mode?

Shadow is the only rung where every property of the gate can be measured without the gate influencing anything. The reviewer runs its full pipeline on real pull requests — risk-class router, find stage, verify stage, verdict record — while posting nothing and blocking nothing. Four weeks of shadow operation establishes availability and latency and accumulates the verdict sample a precision audit needs. A reviewer that cannot run reliably for a month with no authority has no business holding any.

Why is an unmeasured AI reviewer worse than none?

Because it manufactures false confidence: with no reviewer, people read every diff, but with an unaudited one they start trusting findings on tone and reading a missing verdict as a quiet approval. The ladder closes both gaps — precision is audited before any authority is granted, and a missing verdict counts as an availability miss.

What does it mean that the gate fails neutral?

Once a gate holds a required check, an infrastructure failure — an expired credential, a model throttle, a timeout — produces a neutral check conclusion, a decrement to the gate's availability metric, and an alarm. It never produces a red X, so a dead credential cannot block an urgent fix, and it never produces silence, so a dead gate cannot go unnoticed. Neutral-plus-alarm is the property that keeps the availability number honest.

How are findings verified before they count?

A second, adversarial stage attempts to refute every finding against the actual code before it is recorded; only survivors count toward precision. A replay against forty-six merged pull requests with known outcomes showed why: raw find-stage precision measured 66 percent, and the findings labeled highest-severity were the least reliable band at roughly 38 percent. High claimed severity therefore earns the closest scrutiny.

When does the hard gate turn on?

It has no scheduled date. A hard gate is earned by holding the soft-gate bars for weeks first — 95 percent or better audited precision, zero silent-failure days, and an override rate under 10 percent. A blocking gate promoted on a calendar rather than on measurements would hold exactly the unearned authority the ladder exists to prevent.

The Trust Ladder — How AI Code Review Earns Autonomy

Shadow → advisory → soft-gate → hard-gate: AI code review earns merge authority on measured precision.

Abstract

The Trust Ladder is iSimplifyMe's promotion system for AI code review: a four-rung ladder — shadow, advisory, soft-gate, hard-gate — that a non-deterministic reviewer climbs on published metrics before it is allowed to block a merge. It exists because the failure mode of AI code review is not bad findings; it is unearned authority. This dossier documents the ladder, the harness that climbs it, and the bars that decide promotion.

Why a ladder?

Boris Cherny, the creator of Claude Code, published Steps of AI Adoption in July 2026 — five maturity levels running from Gated, with zero agents, to AI-native, with a thousand or more agents. Step 3, supervised autonomy, is the level where agents write nearly all of the code and a human manages by exception.

His stated bottleneck for that step is the reason this system exists: trust in the loop. The trap, in his framing, is scaling agent count before the loop has earned widespread trust.

iSimplifyMe already runs the furniture of step 3 — scheduled agents, overnight batches, and deterministic CI gates across a fleet of more than forty repositories behind roughly a hundred live properties — with agents doing most of the writing. The missing piece was a measured answer to a single question: when the reviewer is a model, how do you know when to stop reading every diff?

The two rules

An unmeasured reviewer manufactures false confidence. Its findings read as authoritative whether or not they hold up, and a dead reviewer posts exactly what a satisfied one does: nothing.

The first rule is that a non-deterministic gate must be measured before it is trusted, because precision is a number, not a feeling. The second is that silence must be impossible, since a gate that can die quietly is worse than no gate at all.

The four rungs

Every AI gate starts at the bottom. Promotion is earned per gate, per risk class, against published thresholds on a trailing window:

Rung	What it may do	Bar to enter this rung
L0 Shadow	Runs on every pull request, logs a structured verdict, posts nothing, blocks nothing	None — every gate starts here; the first promotion audit needs ~30 logged verdicts and availability ≥ 99% over four weeks
L1 Advisory	Posts one comment per pull request — no emails	Audited precision ≥ 85%, p95 latency under three minutes
L2 Soft-gate	Required check, dismissable via a logged override label	Precision ≥ 95%, zero silent-failure days; an override rate under 10% must then be held at L2
L3 Hard-gate	Required, no label bypass	Deliberately unscheduled — earned only

Demotion is automatic. A silent-failure week, precision below the rung's bar on a weekly audit, or more than two wrong blocks in a week drops the gate one rung and raises an alert.

What gets measured

Trust in an AI code-review gate is the minimum of its four signals. Precision is audited-correct findings over total findings; availability is days the gate reported over days with eligible pull requests; latency is the p95 from trigger to verdict; the fourth is the false-block rate.

A fifth, lagging signal — escape rate, production incidents traced back to a merge the gate passed — is tracked as the recall proxy.

The ritual that turns those numbers into decisions is a weekly audit kept to ten minutes on purpose, sampling five verdicts and labeling every finding in them right or wrong. Promotion decisions cite the dashboard those labels feed.

The harness

A risk-class router runs first. It classifies every change by blast radius, and the verdict carries that class, because trust earned on documentation-only diffs says nothing about secrets-adjacent ones.

After classification, the reviewer runs as a two-stage pipeline: a coverage-first find stage reports every candidate issue with severity and confidence, then an adversarial verify stage attempts to refute each finding against the actual code before anything is recorded. Only findings that survive refutation count.

The verify stage exists because replaying forty-six already-merged pull requests with known outcomes put the model's raw find-stage precision at 66 percent and surfaced an inversion worth publishing: the thirteen findings it labeled highest-severity were its least reliable band, at roughly 38 percent. Claimed severity is a prompt to verify, never a reason to trust.

The harness fails neutral: a find-stage call that stays throttled, times out, or returns unparseable output records a neutral verdict for the pull request, and the same throttle or parse failure on a verify call leaves that finding unverified and uncounted. Once a gate holds a required check, the ladder requires any infrastructure fault to resolve to a neutral conclusion plus an alarm, so a broken reviewer can never block a healthy merge.

Verdicts bind to the exact commit they reviewed, so a review that finishes after a fast merge still counts, and a stale verdict reads as stale. Findings are never auto-applied: the harness proposes, a human disposes, and below the soft-gate rung only deterministic checks block a merge on their own.

Status

Everything below is live as of 2026-09-27:

Across the full fleet, the availability watch alarms on any required check that has stopped reporting on an active repository.
The review harness runs at L0 shadow on two production repositories and writes a verdict row for every eligible pull request. As of 2026-09-15 the record held ninety rows on eighty-six pull requests, since a re-review writes its own row.
Harness v0.3.0 has run on both shadow repositories since 2026-09-15. Its verify stage reads the repository source behind the symbols a finding names, within a per-finding budget, before it rules.
Since v0.3.0, a pull request over 1,000 changed lines receives a neutral verdict that the record counts as an availability miss.
A daily check watches the weekly audit itself and alarms once the newest audit label is more than ten days old; an unlabeled week, past its three days of grace, counts as a silent-failure week.
A trust dashboard backs the weekly audit: ladder position, per-gate metrics against the promotion bars, the full verdict log, and the escape-rate record.
The white-paper edition is published, with the full reference architecture, the rung rationale, and the measurement doctrine.

The ladder is also the promotion discipline underneath the validator-gated architecture iSimplifyMe deploys for clients — model judgment proposes, deterministic checks block, and nothing non-deterministic gains authority it has not measured its way into. For teams that want their own AI surfaces held to that bar, engagements start with the Validator Gap Audit.

Trust Ladder

What is the Trust Ladder?