AI Code Review False Positives: Cutting the Noise Without Losing the Signal

The complaint that kills AI review rollouts

Ask a team that turned off its AI reviewer why they did it, and you will almost never hear “it missed bugs.” You will hear “it was noisy.” Fifteen comments on a forty-line diff. Style opinions restated as risks. The same nitpick on every file of a rename. A week of that and engineers scroll past everything the bot says, which is worse than having no reviewer at all — because now the one comment that matters is buried in the fourteen that do not.

The frustrating part: most of that noise is not a model quality problem. It is a configuration and product problem, and it is fixable. This post breaks down where AI code review false positives actually come from, which knobs reduce them, and how to measure whether your reviewer is earning its place on the PR.

What counts as a false positive in code review

Static analyzers have a crisp definition: the tool claimed a defect, the defect is not there. Review comments are fuzzier, and it helps to separate three failure modes that get lumped together as noise:

• Wrong. The comment misreads the code — it claims a null path that cannot occur, or flags an injection risk on a value that is already parameterized. These are true false positives.
• True but useless. The observation is technically correct and changes nothing: hypothetical edge cases in test fixtures, defensive-coding suggestions on internal tooling, performance notes on code that runs once a day.
• True but misplaced. A real issue raised in the wrong place or at the wrong severity — a style preference presented as a bug, or a pre-existing problem in code the PR merely touched, assigned to an author who cannot fix it here.

The distinction matters because each failure mode has a different fix. Wrong comments improve with better context. Useless comments improve with severity thresholds and scoping. Misplaced comments improve with policy — deciding, explicitly, what your reviewer is for.

Where the noise actually comes from

Missing context. A reviewer that sees only the diff must guess at everything outside it — and guesses generate comments. It cannot see that the “unvalidated” input was validated in the caller, or that the “unusual” pattern is the codebase convention. The more of the repository the reviewer can consult before speaking, the fewer wrong guesses survive to become comments.

No severity floor. A model asked “review this code” will produce observations until it runs out, because commenting is what it was asked to do. A reviewer needs an explicit bar: report what a strong human reviewer would block or question, stay silent on what they would wave through. Silence is a feature. A review with zero comments on a clean PR is the system working.

Overlap with the linter. If a formatter or linter already enforces a rule, the AI reviewer repeating it is pure noise. The reviewer’s job starts where deterministic tooling ends: semantics, logic, intent — not import order.

One-size-fits-all depth. A dependency bump, a generated-file churn, and a payment-flow change do not deserve the same scrutiny. Reviewing everything at maximum depth manufactures comments on exactly the PRs where nobody wants them.

The knobs that cut noise without cutting signal

• Set a severity threshold and hold it. Separate blocking findings from suggestions, and let teams choose how much of the suggestion tier they see. Most teams want potential bugs and security issues only; let style stay with the linter.
• Scope what gets reviewed. Exclude generated code, lockfiles, vendored directories and fixtures. Route risky paths — auth, payments, migrations — to deeper review than the rest. In a monorepo, per-path configuration is the difference between a reviewer teams trust and one they mute.
• Feed the reviewer your conventions. Most tools accept project instructions. Two hours spent writing down what your team considers idiomatic deletes entire categories of “true but useless” comments.
• Cap the comment count. A reviewer forced to rank its findings and keep the top handful per PR is forced to prioritize — which is what senior reviewers do.
• Close the feedback loop. Track which comments get resolved versus dismissed, and tune. A finding class that is dismissed nine times out of ten should stop being reported, whether by prompt, rule, or filter.

Measure it like a product, not a vibe

“Noisy” is a feeling; measure it. Two numbers tell you most of the story. The first is actionability: what fraction of the reviewer’s comments led to a change or an explicit human agreement. Mature setups live above one in two; below one in four, engineers are already ignoring the bot. The second is comments per PR: if the median PR gets more than a handful, your thresholds are wrong, whatever your accuracy is. We covered the wider measurement picture — including why time-to-merge matters more than finding counts — in AI code review metrics.

Run the tuning as a loop, not a launch: ship conservative, review a week of dismissed comments with the team, adjust scopes and thresholds, repeat. Two cycles is usually enough to move a reviewer from “mute it” to “actually caught that.”

How PURA approaches noise

PURA was built by people who muted their share of review bots, and it shows in the design. Reviews run with repository-wide context, not just the diff, so the obvious wrong-guess comments never get written. Findings carry severities, and what blocks versus what merely comments is your policy, not ours. Per-repository and per-path configuration — the same machinery behind per-repository budgets — lets a platform team give each codebase the reviewer it needs. And because PURA is self-hosted with your own keys, you can tune prompts and thresholds against your real PR history instead of guessing.

The goal is not a reviewer that never errs — humans err too. The goal is a reviewer whose comments your engineers read because reading them has kept paying off. Noise is not the price of AI review. It is the sign that nobody tuned it yet.