A reviewer is only as good as the brief it receives, and my first one was a friendly paragraph of five lines. The report that came back was a polite list of small things, so over six rounds the instructions changed a great deal.
The reviewer is a separate AI session from the one that built the code, and it has read-only access. It can read files, run the tests and the type checker, and send harmless test requests to the live site. It cannot edit code, deploy, touch the database or contact any external service the app depends on, and I wrote those limits into the brief itself instead of trusting the reviewer's judgment to hold them.
Before anything else it reads the previous reports, which tell it what has been claimed as fixed. Then it believes none of it. For each claim it opens the code, finds the line and, where possible, tries the live site. That rule has paid for itself more than once: one of my own reports said a webhook now verified signatures, and the verification function was sitting in the file with nothing calling it. The reviewer sent an unsigned request and got a success answer back.
Only plain tests are allowed: a GET or a POST with made-up data, a headless browser that opens pages and reads the console, the test suite, the type check and the dependency audit. It must not try to guess passwords, and it must not use real customer data.
Each report has the same shape, with the flaws in the code, the legal and privacy risks and the things done well in three separate sections. Findings are sorted by severity, and each carries the file and line, a concrete scenario for how it goes wrong, a proposed fix and a label saying whether the reviewer verified it or only suspects it. A finding with no scenario goes back to the reviewer.
The report closes with a verdict in plain words: ready for fake data, ready for a small pilot with real data, or not ready, followed by the reasons. By the sixth round the verdict was a yes for a manual pilot with one condition, a signed data processing contract.
I have also learned to keep the builder out of the verdict. The agent that writes a fix is the wrong one to say that it works.
