PR Reviewers

A PR Reviewer watches a GitHub repository and, on every pull request, uses an AI model to author hardware tests for the change, flash the relevant firmware onto a real box, run those tests, and post the result back to GitHub as a check. It closes the loop between “someone opened a PR” and “we know whether it works on hardware” — without a human hand-writing a test for each change. This is different from a fixed, hand-maintained test suite (that’s Factory). A PR Reviewer reasons about the diff and generates the tests it thinks the change needs.

​

Prerequisites

• The GitHub integration connected for your org, with access to the repo. See GitHub.
• At least one box online with the instruments the tests will need.

​

Creating a reviewer

From PR Reviewers → New, configure:

• Repository — owner/repo. Add path prefixes to scope the reviewer to part of a monorepo, so a PR only triggers it when it touches those paths (one reviewer per product).
• Model — the Claude model that authors and drives the review (Claude Opus 4.7, Sonnet 4.6, or Haiku 4.5; Haiku 4.5 is the default).
• Firmware source — either a prebuilt binary at a path in the repo, or a CI artifact pulled from a named GitHub Actions build workflow (by artifact name or regex pattern). Optional flash load address and erase.
• Boxes — run on any online box, or restrict to an allowed list. You can also require specific net types so a run only lands on a box that actually has the hardware. An optional box-selection mode ranks candidate boxes automatically.
• Tests — where generated tests are written and run (single file or directory module), extra files to include, and a per-run timeout.

​

Guardrails and automation

Each reviewer has controls for how far the AI is allowed to go on its own:

• Human approval (default on) — pause each run for a designated person to approve before anything flashes or runs on hardware.
• Commit generated tests — commit the tests the reviewer wrote back to the PR branch.
• Agentic self-heal / code & firmware remediation — let the reviewer iterate (up to a configurable number of rounds) to fix a failing test or propose a fix, rather than just reporting red.
• Pass triage — have the model sanity-check passing runs.
• Rate limit — cap runs per hour (1–1000) so a storm of PR updates can’t monopolize the fleet.
• Slack — post run notifications to a channel.

​

Runs and approvals

Every matching pull request creates a run. Runs that are waiting on human approval collect in an awaiting-approval queue — the count shows as a badge on the PR Reviewers sidebar item. Open a run to see the model’s reasoning, the generated tests, the flash and test logs, and the final verdict. The same verdict appears as a Stout check on the PR in GitHub.

​

Related