Data flow
Last updated 2026-10-10日本語All documents
This page explains where the data of a ReviewFixLoop run goes. The handling of personal data is described in the Privacy Policy.
Terms used on this page
- Harness: a coding agent program, such as a vendor's command-line tool (a "vendor CLI").
- Reviewer: a combination of a harness, a model and an effort setting that reviews a pull request.
- Resolver: the coding agent that conducts a review of a pull request: it decides how to treat each finding and changes the code.
- Run: one execution of the review-and-fix loop on one pull request.
- Finding: an issue that a reviewer reports in a run.
- Judgment: the resolver's decision on a finding.
- Judge: the automated classifier with which the service assigns a tag, a severity and duplicate relations to each finding.
- Workspace: the unit of the service to which runs belong. Members of a workspace can see its runs.
The three layers
The CLI divides the data of a run into three layers.
| Layer | Content | Sent to ReviewFixLoop |
|---|---|---|
| L0 (metadata) | The configuration of the reviewers and the resolver (harness, version, model, effort, review prompt variant, manifest identification), token counts, the severity, tag, judgment and duplicate relations of findings, which judge was used, and timestamps | Always |
| L1 (free text) | The title, body and evidence of findings, the reasons for judgments and decisions, excerpts of the pull request description, and excluded findings | Always |
| L2 (full sessions) | All reviewer responses, tool output and logs | Not by default. Only when you upload them with a bug report |
- L0 is needed to recompute the leaderboard.
- L1 is needed to judge findings and their duplicates. All free text is treated as L1.
- L2 stays in the run's directory on your machine unless you upload it with a bug report.
Where the data goes
| Destination | Data |
|---|---|
| Stays on your machine | The checkout, the diff, L2, and your vendor logins and keys |
| From your machine directly to each vendor | The code, diffs and prompts of the reviewers and the resolver, under your own account with that vendor |
| To ReviewFixLoop | L0, L1, the identification of the repository and the pull request, and L2 only when you upload it with a bug report |
| From ReviewFixLoop to external AI models | As described under "The judge" and "Re-judging to measure validity" below |
| Public | As described under "What is public" below |
GitHub
- Sign-in to the web site uses GitHub OAuth for identity only. It requests no scopes and obtains no access to your repositories. The service stores your GitHub numeric user id, your login, your GitHub account's creation time, the country code of the sign-up request, and the version of the notice you agreed to at sign-up with the time of that consent.
- ReviewFixLoop does not ask you to install a GitHub App on your repositories. Reads from GitHub happen on your machine with your own
gh. - The service accepts the repository and pull request information that the CLI reports. The one thing it verifies itself is whether the repository is public: it queries the GitHub API with a token that has no repository permissions. Only a repository visible to that token is treated as public. Every other repository is treated as private.
The judge
- The service sends the file, line, title, body and evidence of each finding, and of its duplicate candidates, to external AI models that judge its tag, severity and duplicate relations.
- L0 records which judge judged each finding.
Re-judging to measure validity
To measure the validity of resolver judgments, the service samples findings judged in the last seven days from both public and private repositories. It sends the finding's file, line, title, body, evidence and the resolver's stated reason to an external AI model of a different family from the resolver's. Findings and reasons in which a secret-looking string is detected are left out. The agreement rate is published on the leaderboard. Re-judging does not change the original judgment, reward or rating.
Private runs in the personal workspace of a deleted account are left out of new re-judging. Data that was already sent cannot be recalled.
Sending to external AI models
- The judge and re-judging send data with the service's own credentials. This is separate from the vendor CLIs that run on your machine with your credentials.
- What is sent includes findings from private repositories.
- The service requests a route on which the data it sends is not used to train models. It does not require that the providers retain no data at all.
- The providers are companies in the United States of America. You consent to this provision when you sign up; the Privacy Policy states what you are told then.
- The providers the service uses can change. Their names are available on request to the contact in the Privacy Policy.
Vendor credentials
- The official manifests only launch the unmodified vendor CLI that you installed and configured, on your machine.
- ReviewFixLoop does not read, store or relay your vendor credentials, and does not record which authentication method a vendor CLI uses.
See Policy toward each vendor's terms.
Secrets
- Before sending, the CLI masks strings in evidence that look like secrets.
- The service checks again before publication. A finding in which a secret-looking string is detected is not published, and its owner is notified on the run page.
What is public
| Data | Who can see it |
|---|---|
| Aggregates (ratings, intervals, cost multiples, account counts) | Anyone, on the public leaderboard |
| Runs on public repositories | Anyone, on the public leaderboard, down to the findings, judgments and cost of each pull request. The page links to the repository and the pull request. It never shows the handle of the pull request's author |
| Runs on private repositories | They contribute to the aggregates of the public leaderboard only |
| Your personal leaderboard | Only you |
| Contributor names | Shown only for users who opted in |
- Everything else belongs to a workspace and is visible only to that workspace's members.
- Judgments shown publicly are labelled as AI judgments.
- Free use requires contributing to the public leaderboard.
Retention
See "Retention" in the Privacy Policy for the periods for L0, L1 and L2, including the seven-day period before deletion after a repository stops being public. Its "Account deletion" section describes which account data is removed, the temporary digest used to prevent resetting free limits, and what run data remains.