Three separate features shipped this month that all touch the same unanswered question. Effort levels let you dial how deeply Copilot reviews a change. Code quality trends let you track findings over time. Multiplayer agent sessions in Slack and Teams let a group watch an agent work together.
None of them tells you what a human is actually still responsible for checking. That allocation has quietly become a decision every team is making by default rather than on purpose.
What agent review is reliably good at
Pattern-matching against known problems: obvious security anti-patterns, style violations, missing null checks, common bugs with a well-established signature. This is the review work that was always somewhat mechanical, and automating it is a straightforward win rather than a trade-off.
It is also good, when set to Balanced effort, at flagging that something looks unusual even without fully understanding why — which is a genuinely useful signal to a human reviewer, distinct from a genuinely useful verdict on its own.
What it systematically misses
Three categories, and they share a root cause: none of them is visible from the diff alone.
- Intent mismatch — code that works exactly as written but does not do what the business actually needed, because the requirement was never fully captured in the ticket or the prompt
- Organisational context — a change that violates an unwritten convention, conflicts with a decision made in a meeting six months ago, or steps on a team boundary that exists only in people's heads
- Consequences outside the diff — a change that is locally correct and breaks something two services away, where the breakage depends on a system the agent was never shown
A concrete allocation by change type
Not a universal rule, but a starting point worth adapting to your own codebase.
- Routine, low-risk changes — agent review at Lite effort is probably sufficient. A human skim rather than a line-by-line read is proportionate here
- Anything touching auth, payments, data migration, or infrastructure — agent review at Balanced effort, plus a human review that specifically checks the three categories above, because these are exactly the changes where intent mismatch and cross-system consequences are expensive
- Anything from a multiplayer chat session — treat it identically to a change from any other entry point, with one specific person named as the reviewer of record, regardless of how many people watched it happen
- Anything the agent itself flags as uncertain — always a human look. An agent that says "I am not confident about this" and gets waved through anyway has told you exactly where to check and been ignored
The trend to watch, not the snapshot
The code quality trends dashboard GitHub shipped on 19 August is useful for exactly one thing here: noticing when review is quietly eroding rather than staying calibrated. A rising trend paired with lighter effort levels being selected more often tells a different story than a rising trend on its own — the first is a symptom of trust outrunning verification, and the report only shows that if you look at the two together rather than either alone.
Check that pairing monthly, not the raw finding count. A stable finding count with effort levels quietly drifting toward Lite across sensitive repositories is the early warning, and it does not show up unless you specifically go looking for it.
The habit that keeps the allocation honest
Agent review quality genuinely improves month over month, and it is tempting to let human attention shrink at the same pace. That is backwards for one reason: the categories agent review misses — intent, organisational context, cross-system consequences — are not solved by a better model. They are solved by someone with context the agent does not have, and no model release changes that.
The habit worth keeping: whatever your team decides a human should specifically check, write it down as a short checklist rather than trusting memory, and revisit that checklist quarterly rather than letting it erode the same way review effort quietly can.