Every pull request in our world used to end the same way: a developer drops the link in #pull-requests and types some version of "can someone merge this?" Across ~150 repositories and a dozen client projects, "someone" was whoever was free, and "reviewed" meant whatever that person happened to check.
We wanted one reviewer on every frontend and backend repo — something that reads each PR, checks it against our quality checklist, and says what it found before a human spends a minute on it. Not a gatekeeper: a second pair of eyes that never gets tired and never skips the boring checks, with the merge decision left to a person.
What shipped is a single GitHub Actions workflow that lives in one repo and is called by all the others. It works out whether a repo is frontend or backend, pulls the right checklist from Notion, asks Claude to review the diff, comments on the PR, and posts a card to Slack the moment the PR opens. Here's how we built it, rolled it to 83 repos in an afternoon, and the three walls we hit on the way.
1. The problem was the paste, not the review
The symptom was the Slack paste. The real problem was that nothing was consistent. Our repos split cleanly into two stacks — Next.js/React on the front, FastAPI on the back — and each team reviewed however its lead preferred. There was no shared list of the things we always check, and the things we always check are exactly the ones a tired reviewer skips at 7pm: the leftover console.log, the hardcoded URL, the form with no loading state, the endpoint with no auth.
2. One workflow, a 15-line caller per repo
The logic lives in exactly one place: a repo called shared-workflows. Every other repo carries a 15-line caller file that points at it. This is GitHub's reusable-workflow pattern, and the reason to use it is drift. Copy a review script into 83 repos and the day you want to change the prompt, you're editing 83 files and hoping they stay in sync. Instead the caller is a stub that names the hub workflow and passes through three secrets. Change the hub once, and every repo picks it up on its next PR. Onboarding a new repo is a copy, a paste, and two secrets.
3. The repo name decides the rules
The hub needs to know whether it's reviewing frontend or backend code, because the checklist differs. We made the repo name decide: backend in the name runs backend rules, frontend runs frontend rules. For the few that don't follow the convention, it falls back to the primary language — Python is backend, TypeScript/JavaScript is frontend — and skips anything it still can't place rather than guess.
The checklist itself doesn't live in the code. It lives in a Notion page a non-engineer can edit, and the workflow fetches it live through the Notion API. There's a full copy hardcoded as a fallback, so a Notion outage never blocks a PR.
The first test returned a Notion 404 and the bot quietly used its fallback. That 404 meant the page hadn't been shared with the integration — not that the token was wrong. Notion returns "not found", not "forbidden", when an integration lacks access. Worth knowing before you debug the wrong thing.
4. Claude Haiku reads the diff, and stays advisory
The review is one call to Claude Haiku. It gets the checklist and the diff and returns a comment marking each relevant item handled, worth verifying, or violated, then ends with an overall risk — green, yellow, or red — and a sentence saying why. Two decisions mattered:
- It only flags what's relevant to the diff — a backend change doesn't get nagged about mobile touch targets.
- It's advisory. The workflow always exits successfully. It never fails the check because the code has gaps; it comments, and a human decides.
It's also cheap: Haiku runs this for roughly $1–5 a month across the whole team. A single Copilot Enterprise seat is $39. And the comment updates in place — each push deletes the old bot comment and posts a fresh one, so the PR never grows a pile of stale reviews.
5. We wrote the backend checklist by reading a real backend
The frontend checklist already existed. The backend one didn't, so we wrote it by profiling the best FastAPI service we run and reading what "good" actually looked like in our own code: routes that delegate to controllers, explicit status codes and response models, Pydantic validation, a JWT dependency on protected routes, every query scoped to its tenant.
Then the useful part: the reference repo broke several of its own ideals. Passwords hashed with bare sha256. Logging through ic() calls, including the user's password at login. A pytest config and not a single test. That made the gaps in our best repo the highest-value rules to write down — bcrypt or argon2, never sha256; standard-library logging, never ic() in a request path; never log a credential; tests for new endpoints. The checklist encodes the standard we want, not the one the reference happened to meet.
6. We proved it with a good PR and a bad one
We opened two demo PRs per stack — one written to pass, one to fail. The failing backend PR earned a red: hardcoded JWT secret and database URL, a SQL query built by string concatenation, sha256 hashing, the password printed to the log, and a bare except. The verdict read "Do not merge." Correct.
Our first "compliant" PR also came back red — the bot flagged inline logic with no controller layer and no tests. We rewrote it into a proper route-controller-schema split with a pytest suite, and it dropped to yellow. The bot held a higher bar than the example we wrote to pass it.
7. Eighty-three repos, by PR, into a wall of 404s
Rolling out meant touching every active repo: 83 of them. We did it by opening a pull request in each, not committing straight to the default branch. Then GitHub started lying to us — calls began returning 404s for repos that obviously existed, while the core rate limit read 5000 of 5000. It was the secondary anti-abuse throttle for rapid writes, and it doesn't announce itself politely. The fix was to slow down: sleeps between repos, smaller batches, retries on reads. Moving fast through the API is the thing that makes the API fight back.
8. The gates we found, and chose not to break
Merging the rollout PRs revealed the fleet's real security posture. 47 merged without resistance. 14 had classic branch protection an owner can override, so we did. The last 21 had repository rulesets requiring a human review with an empty bypass list — no one, not even the owner, can merge those without an approval.
We left all 21 open. We could have added an owner bypass and forced them through, but the entire point of the bot is to support human review. Bulldozing 21 deliberate review gates to install a review tool would have been the joke writing itself.
9. Where it shows up, and what broke
The last piece is Slack, where the "can someone merge this" habit lived. The first version posted the entire review into the channel — every PR became a wall of text. We rebuilt it as a compact card: frontend/backend header, linked title and author, one line with the risk verdict, the reminder not to merge until checks finish. The full review stays on the PR.
And the parts that broke, plainly — because a tidy case study is a dishonest one. Under the rate limit, branch detection got flaky and misread which repos had a staging branch. Two repos were renamed or deleted mid-rollout and fell out from under the script. One ended up orphaned, its PR closed with no replacement. None of that is a flaw in the design; it's the texture of changing 83 live repositories at once through an API built to slow you down. Next time: detect state with retries, never close the old PR before the new one is confirmed open, and roll out in batches of ten.
Questions about the build
Does it block merges?
No, by design. The workflow always succeeds; it comments and posts to Slack, and the merge decision stays with a person.
How much does it cost to run?
The only ongoing cost is the Anthropic API on Claude Haiku — around $1–5 a month for a team our size. A single Copilot Enterprise seat is $39.
Why keep the checklist in Notion?
So the people who own the quality bar can change it without opening a pull request. The workflow reads the page live at runtime, with a hardcoded fallback so an outage never blocks anyone.
Internal, or something you'd build for a client?
We built it for our own 83 repos first. Because it's one reusable workflow plus a 15-line caller, dropping it into a client's org is a copy, a paste, and two secrets — and the checklist is just a Notion page they edit themselves.