Large pull requests are where bugs, security issues, and breaking changes go to hide.
If you maintain an open source project, you've been there: a 400-line PR lands in your review queue. You scroll through it, skim the obvious parts, approve it, and move on. Two weeks later, something breaks — and the culprit was buried on line 312 of that diff.
You're not lazy. You're human. And large PRs are designed (unintentionally) to exploit exactly how humans review code.
The Science of Skimming
Studies on code review consistently show that review effectiveness drops sharply after ~200 lines of diff. Beyond that threshold, reviewers start skimming. They focus on the parts they understand and gloss over the rest.
For open source maintainers, this is especially painful. You're reviewing code from contributors you may not know, in your spare time, often on a phone between meetings. The incentive structure practically guarantees things slip through.
What Actually Gets Missed
After looking at hundreds of open source PRs and the issues that followed them, patterns emerge. Here's what reviewers consistently miss in large diffs:
1. New Dependencies
A contributor adds a helper library — seems reasonable. But buried in package.json or go.mod, there's now a new transitive dependency tree you didn't audit. That tree might include packages with known vulnerabilities, packages with minimal maintenance, or packages with licenses incompatible with yours.
Real example pattern: A PR that refactors a utility module also adds lodash as a dependency — when the project previously had zero runtime dependencies. The refactor gets reviewed; the dependency change gets a rubber stamp.
2. Permission and Scope Changes
CI config changes are the ultimate "eyes glaze over" zone. A .github/workflows/ modification that adds write permissions to a workflow, or a Dockerfile change that switches from a non-root user to root — these are high-impact, low-visibility changes.
Real example pattern: A PR updates a GitHub Action workflow to fix a build issue. Buried in the YAML diff: permissions: contents: write was added. The contributor needed it for their fix, but now that workflow can push to your repo.
3. Configuration Modifications
Changes to .env.example, config.yaml, nginx.conf, or infrastructure-as-code files often ride along with feature PRs. Reviewers focus on the application logic and skip the config changes entirely.
Real example pattern: A PR adding a new API endpoint also modifies the CORS configuration to allow * origins. The endpoint code gets thorough review. The config change? Nobody notices.
4. Secrets and Credentials
Hardcoded API keys, tokens, or internal URLs sometimes appear in test fixtures, example configs, or debug code. In a large diff, they blend into the noise.
5. License and Legal Changes
A new file with a different license header gets added. A LICENSE file is modified. A vendored dependency brings its own licensing terms. These changes are invisible to most reviewers unless they're specifically looking for them.
Practical Tips for Reviewing Large PRs
If you maintain an open source project, here are concrete things you can do today:
Break the scroll habit. Don't review a large PR top-to-bottom in one pass. Start with the files that matter most: config files, CI/CD, dependency manifests, permission-related code.
Use GitHub's file filter. Filter the diff to show only .yml, .json, .toml, .lock files first. Review those separately from application code.
Check the dependency diff explicitly. If package-lock.json or go.sum changed, don't just collapse it. At minimum, check what was added.
Ask contributors to split large PRs. This is the single most effective policy. A 400-line PR should usually be 3-4 smaller ones. Make this a documented expectation in your CONTRIBUTING.md.
Set up automated guardrails. Manual review doesn't scale, especially for the boring-but-critical stuff like dependency changes and permission modifications.
Automating What Humans Miss
This is where tooling matters. Not as a replacement for human review, but as a safety net for the things humans predictably miss.
Axiomo was built specifically for this problem. It's a GitHub App that scans every PR and surfaces structured risk signals — new dependencies, permission changes, config modifications, and more — so you don't have to read 500 lines of diff hoping you'll catch the important parts.
It doesn't replace your judgment. It tells you where to focus it.
What Axiomo flags automatically:
- New dependencies added to your project (and their risk profile)
- CI/CD and workflow changes that modify permissions or secrets access
- Configuration changes that affect security posture
- License modifications across your dependency tree
- Sensitive file patterns — env files, credentials, infrastructure config
For open source maintainers, Axiomo is free and unlimited for public repositories. You install the GitHub App, and it starts working on every PR — no configuration required.
The Maintainer's Dilemma
Open source maintainers are asked to be security experts, dependency auditors, license lawyers, and CI/CD specialists — on top of actually reviewing code logic. That's not sustainable.
The answer isn't "review harder." It's having structured signals that point you to what matters, so you can spend your limited review time on the things that actually need human judgment.
Large PRs will always exist. But missing what's hidden inside them doesn't have to be inevitable.
Axiomo is a free GitHub App that surfaces risk signals in every PR. Unlimited for public repos. Install it in 30 seconds → axiomo.app
Top comments (0)