When agents write most of your code, the pull request stops being a defect-catching gate and becomes something more important: the steering wheel of the entire system. The compiler, the linters, and the automated reviewer already caught the null-pointer bug. What’s left is the question no machine reliably answers — is this the design we want the codebase to keep growing into?
This is the governance piece of our series on running agentic development loops at scale. Effective human PR review of AI agents isn’t about reading diffs faster. It’s about reviewing at a higher altitude and — the key idea — turning every review decision into something the loop never makes you repeat.
🎚️ Raise Your Review Altitude
Traditional review mixed three jobs that agents now split apart. The first two are no longer yours:
- Mechanical correctness — typos, style, obvious bugs — is owned by linters, type-checkers, and the automated first-pass reviewer. (GitHub Copilot’s code review has crossed 60 million reviews, growing 10× in under a year; more than 1 in 5 reviews on GitHub now involve an agent.) Doing this by hand is waste.
- Local intent — does this function do what the PR claims? — is shared: the machine flags candidates, you confirm on the critical path.
- Global design fit — is this the right abstraction? Does it belong here? Does it quietly re-implement something three modules over? Does it honor our architecture? — is human-only, and now the main event.
That last one is the whole game, because a reviewer who sees only the diff cannot tell that a change is architecturally wrong — wrongness is a property of the change relative to the whole codebase, not of the diff itself. GitHub’s own guidance nails the new posture: “Let the agents scan first. You trace the critical path.” You’re not asked to read every line; you’re asked to spend scarce judgment on the highest-consequence path.
🔧 The Judgment Ratchet
Here’s the concept that separates a scaling loop from a treadmill. In a naive setup, a senior reviews 40 agent PRs and leaves the same five comments over and over — “we already have a PaymentGateway,” “put validation at the boundary,” “don’t add another HTTP client” — and burns out. The loop learned nothing, because the agent doesn’t remember across sessions.
In a ratcheted loop, every recurring review comment is treated as a bug report against the agent’s instructions, and you fix it by encoding the lesson so the agent gets it right next time without being told:
- A recurring structure comment → a lint rule or a line in
REVIEW.md. - A recurring “you reinvented X” → a skill pointing at the real pattern, plus a fitness function that fails the build on the anti-pattern.
- A recurring architectural violation → an automated boundary/dependency check so the agent and the automated reviewer catch it before it ever reaches you.
The strongest rungs are deterministic (lint, CI, fitness functions), because CLAUDE.md-style prose is only guidance the model follows ~70% of the time — good enough for taste-level lessons, not for invariants. (Anthropic’s Claude Code even splits this out: REVIEW.md holds review-only rules so CLAUDE.md stays small.) The reframe for your team: a review comment is no longer feedback to a junior who’ll remember it — it’s input to the system’s configuration. The highest-leverage engineer isn’t the one who reviews the most PRs; it’s the one who converts the most review decisions into reusable rules. Review becomes a teaching loop, not a gate you stand at forever.
😴 Approval Bias: Why the Gate Weakens Over Time
Now the uncomfortable part. The human gate doesn’t just strain under volume — it degrades quietly, and it gets worse with exposure, not better.
A 2026 longitudinal study, Habituation at the Gate, documented exactly this: as reviewers see more agent submissions, their approval rate rises while their scrutiny falls. Familiarity breeds complacency. Worse, GitHub found reviewers actually feel better about approving agent-generated code — even though that code carries more redundancy, roughly 1.7× the defects, and often an OWASP-class vulnerability. Fluent, well-formatted, confidently-commented output triggers less suspicion precisely when it deserves more.
The governance takeaway is blunt: review discipline must be structural, not motivational. You cannot willpower your way past habituation. Checklists, rubrics, mandatory critical-path tracing, and PR-size limits work; “try to be more careful” does not, because a tired human will reliably feel less suspicious of code that looks great and passes CI.
🚦 Backpressure: Review Is the Bottleneck Now
The physics are simple: agents are a high-throughput producer feeding a fixed-capacity consumer — human judgment. DORA’s 2025 report measured the collision directly — high-AI teams merged 98% more pull requests, but PR review time rose 91%. Generation got nearly free; review didn’t. Code review is now the dominant downstream bottleneck.
The elite response is not “review faster” — that’s how rubber-stamping creeps in. It’s two moves: reduce the number of PRs that need deep human judgment (via the ratchet and the automated first pass) and regulate the inflow — PR-size limits, WIP caps, and, as of mid-2026, GitHub’s literal throttle that lets maintainers cap concurrent PRs from untrusted contributors. Flag any PR that touches more than five unrelated files, can’t state its purpose in one sentence, or arrives without an implementation plan — and review the plan before the code.
⌨️ When to Take the Keyboard
Steering is mostly directing — comment, request changes, update a rule. But a small set of situations call for a human to edit directly instead of round-tripping through the agent:
- Taste-dense changes where writing the instruction precisely takes longer than making the edit (a naming choice, the right abstraction boundary).
- Novel architecture with no precedent — the loop can’t pattern-match what doesn’t exist yet, so a human seeds the pattern, then encodes it.
- Agent thrash — when it’s failed the same change two or three times, stop prompting and fix it.
- Hot paths — auth, money, migrations, public APIs: keep a human-owned gate here.
The elite framing: taking the keyboard isn’t a failure of the loop — it’s a signal. Every direct edit is a data point about where the loop’s instructions are too thin, and it should usually be paid back into the ratchet as a new rule or skill. (Take the keyboard too often, though, and you’re the bottleneck again, and the ratchet never gets built.)
📋 The Architecture-Intent Review Rubric
When a PR survives the machine gate and reaches you, review it at design altitude with a rubric like this:
- Belonging — right module/layer? Does it cross a boundary it shouldn’t?
- Reuse — does it re-implement something that already exists? (The #1 agent failure mode — search before accepting.)
- Abstraction fit — right level, or an over-eager generalization the agent invented to look thorough?
- Pattern consistency — does it follow the established way we do error handling, validation, DI, logging?
- Change-friendliness — easy to modify in six months, or does it hard-code assumptions?
- Critical path — trace input → output on the highest-consequence path; check boundaries, external-input validation, and permission checks on every branch.
- Test intent — do the tests assert real behavior, or did the agent write tests that just game the coverage gate?
- Provenance — what was the AI’s role, what context did it get, what did the human contribute?
❓ Frequently Asked Questions
How does PR review change when AI agents write the code? It moves up in altitude. Machines own mechanical correctness; humans review global design fit — belonging, reuse, abstraction, and architectural drift that no diff-level tool reliably catches.
What is the “judgment ratchet”? The practice of turning every recurring review comment into a reusable rule, skill, or fitness function, so the loop never needs a human to make the same call twice. Review becomes a teaching loop instead of a treadmill.
Why is “approval bias” dangerous? Because reviewers approve more and scrutinize less the more agent code they see — and they feel better about approving polished code that actually carries more debt. The fix is structural discipline (rubrics, size limits), not willpower.
How do I stop review from becoming the bottleneck? Reduce the PRs that need human judgment (ratchet + automated first pass) and throttle inflow (PR-size limits, WIP caps, PR caps) — don’t just try to review faster.
When should a human edit the code directly? For taste-dense changes, novel patterns with no precedent, agent thrash, and security/correctness hot paths — then encode the lesson back into the loop.
📚 Further Reading
- The Agentic Development Loop, Explained — the loop this review steers.
- DORA Metrics for AI Coding Agents — the backpressure data behind the review bottleneck.
- How to Stop AI Agents Turning Your Codebase Into Slop and Securing a Fleet of Code Agents — the fitness-function and human-gate mechanics referenced here.
Part of the “Agentic Loops for Enterprise Codebases” series.