Adversarial Loop Evidence¶
Use existing Code Review routing and specialist-manifest validation for independent execution evidence. The loop adds bindings between that evidence, source contents, returned findings, current source; it does not create or promote a new execution route. Read each helper's --help before invocation.
Capture source¶
Use shared/collect_diff.py snapshot mode with an explicit repository and bounded literal repository-relative scope paths. Retain one source JSON file per review round, a fresh current-source JSON before acceptance. Keep artifacts outside captured source scope, or in an ignored report directory, so writing evidence does not change the source being checked.
Snapshots record repository, normalized scope, actual HEAD, staged-index digest, each selected tracked or nonignored untracked file's kind, mode, bytes digest, encoding, content. Deleted files remain visible. UTF-8 source is readable in the snapshot; for base64 binary content, name each affected path and its binary review or format-verification limitation in the assessment limits. The validator checks that the paths and binary are mentioned, not that the explanation is truthful or sufficient. Scope omissions and excluded paths stay explicit in the review context/report. Never send credentials or sensitive source to a route that doesn't permit it.
Collect the source snapshot and supporting diff from the same state. Include the snapshot's complete retained JSON bytes and the complete diff in every participating reviewer's frozen context, not merely their hashes or paths. Complete source is authoritative for inspection; the diff is supplementary and can be empty for an existing implementation or problem with no local changes. The context starts with the exact shipped challenger role card followed by one newline, or directly with the first evidence label; no other preface is allowed. Keep explicit Review request:, Frozen source:, Frozen diff:, and, when applicable, Supporting source: labels. Each section contains exactly its retained bytes and ends at the next label or context end; an empty diff follows the same rule. The no-model preflight compares the retained tracked patch with the current scoped Git diff; clean final validation repeats that comparison for both current reviewer routes while historical schema-1 evidence remains readable. Post-review validation also binds the exact retained bytes to the authenticated review. Request fresh evidence if context or source changes. Current acceptance recaptures the same repository/scope, rejects drift in current contents, index, file inventory, revision; clean acceptance also requires equality with the final independently reviewed source.
For every new round, freeze the caller's goal, specification, and done_when strings in the schema-2 evidence request object; schema-3 uses the same shape. Send Review request: followed by its json.dumps(request, sort_keys=True, ensure_ascii=True) form and a newline in every authenticated reviewer context. This proves delivery of the parent's declared review criteria, while the parent remains responsible for interpreting the user's natural-language request and scope. Never send resolver handoffs or implementation explanations in reviewer context. Select relevant unchanged callers and consumers by source inspection; record their repository-relative supporting_paths, capture their complete source JSON at each round and before acceptance, and send its exact bytes in every reviewer context. The gate requires a captured file, symlink, or tracked-deletion record for every declared supporting path and checks its delivery and current bytes; one present supporting file cannot conceal another absent path. The parent owns the semantic judgment that this list covers the relevant dependencies. If none are relevant, use an empty list and null supporting snapshot paths. A changed request or supporting source needs a fresh independent review.
Before an isolated or scoped native reviewer dispatch, run validate_evidence.py --preflight-plan <frozen-plan.json> --request-evidence <run>/loop-evidence.json --supporting-source <frozen-supporting.json> when supporting source is declared, omitting --supporting-source otherwise. For schema-3 native preparation, pass --challenge-only --paged --challenge-evidence <run>/loop-evidence.json to review_prepare.py prepare; create the referenced primary and supporting snapshots before preparation. The native prepare step also invokes this preflight. It uses the same exact context-material check as post-review validation, recaptures source, checks evidence-declared source copies and the current scoped tracked patch, and rejects mismatched canonical bytes, digests, request text, response contract, or Supporting source: label before a paid turn. It is a local no-model check; the separate host check and completed reviewer receipt remain required.
For the isolated Codex review route, apply Code Review's transport and capacity checks before paid dispatch. Larger contexts retain this exact complete-source contract whether delivered directly or loaded byte-exact into the same thread's history before its short trigger turn. History delivery requires the adapter's validated acknowledgement/context-digest binding; short trigger text alone is not source evidence. Successful byte admission, history loading or a requested token window does not establish model capacity or review completeness. Missing, changed, partial, or substituted source remains rejected at the existing evidence gate.
For a diff or complete file scope too large for one admitted review context, use the packaged chunk_diff.py planner with --scope-mode changed or --scope-mode all and required --goal, --specification, --done-when fields. Its plan binds the declared coordinator request and requested file inventory to disjoint whole-file chunks and exact source/diff bytes. Each chunk is a separate ordinary loop with unchanged reviewer provenance and source validation; its request must equal the coordinator request. The coordinator records a decision for every chunk pair, including a source-backed no-interaction rationale, and declares known material relationships spanning three or more chunks as groups. Every decision and group needs an ordinary independent run over the exact union of complete member chunks; one run may cover several entries when its frozen source and diff contain their complete union. An interaction run retains the coordinator goal and done_when and appends only the canonical Interaction assessment: block to its specification, so the reviewer can challenge omissions and classifications. The chunk_diff.py check --results gate verifies the declared decision map, internal request consistency, authenticated request delivery, and clean current-source results; the parent still judges whether any undeclared multi-chunk relationship is plausible. New coordinator candidates declare metadata.challenge_scope as chunked with literal chunks/chunks.json and chunks/results.json paths; final validation reruns the chunk checker. A single-run candidate declares {"mode":"single"} and cannot contain a visible chunks/ directory. Replan after edits, retain old evidence, and rerun affected independent reviews. The local reviewer wave context ceiling applies only to that optional route; check native inspection separately before classifying independence as unavailable. A validated partition proves file coverage and declared assessments, not the absence of unknown interactions. Record and review relevant callers and consumers before an overall clean claim.
For remediation continuing earlier validated rounds, use schema-1 chunk plans with --caller-run, --prior-ledger, and --prior-coverage. The coverage JSON is a list with one { "signature": ..., "source_paths": [...], "review_kind": "chunk|interaction|group", "chunk_ids": [...] } entry per final-round finding. Keep the earlier authenticated result and report; the planner structurally validates and copies the ledger, pins its digest, and appends a canonical Prior findings to reassess: block containing each prior signature, original tier, structural flag, evidence, and declared source paths to the coordinator specification. Exact child and interaction request checks then bind this block to reviewer delivery. Before each paid dispatch, verify the assigned run's frozen primary and supporting source includes its declared paths and run the ordinary no-model context preflight; do not spend a turn on known missing coverage. At clean result checking, the assigned run must independently authenticate the exact prior signature with its original tier and structural flag as verified-fixed or rejected; mismatches remain unresolved. A stopped summary lists unresolved assigned findings as pending_prior_signatures, including when all supplied runs are otherwise clean. New plans use schema 1 with explicit origin; a first run records null caller and prior ledger, while a continuation binds its declared caller run and copied ledger. Candidate and final coordinator results copy the manifest origin into metadata.challenge_origin, which the validator compares under both filenames. Exhaustive continuation completion requires schema 1; a falsely declared first run or undeclared caller history cannot be inferred from local artifacts. The checker rejects unsupported manifest versions for source, clean, and stopped checks so version changes cannot bypass prior obligations. Neither a digest nor a parent-declared path list proves prior reviewer authenticity or semantic completeness: missing old receipt, unknown dependency, or omitted finding remains unassessed and blocks an overall clean claim.
For a first-run scoped review of supported UTF-8 text files, freeze an explicit full comparison-base commit OID and a verified detached review mirror. Run chunk_diff.py plan with the mirror as --repository, the invoking checkout as --origin-repository, and the OID as --diff-base; verify that OID is the user's intended comparison base. The profile checks that the original checkout and mirror agree on revision, selected source, inventory and comparison diff. Use scoped-plan --manifest <chunks.json> --assignments <assignments.json> --out <scoped-chunks.json> to assign each material interaction to a chunk and each chunk to one independent audit. The assignment object has interactions entries { "chunk_id": "...", "requirement": { "id": "...", "source_paths": ["..."], "claim": "..." } } and audit_assignments entries { "id": "...", "chunk_ids": ["..."] }; sort IDs and paths. Run check on the scoped manifest before dispatch. Each child uses schema-3 loop evidence with the same diff_base_oid and its exact interaction requirements; its reviewer receives the assigned source and required supporting endpoints. The independent auditor receives its assigned units, the full declared interaction inventory, and the exact schema-3 audit requirements, avoiding full pairwise chunk unions while retaining independent audit. Retain unknown or disputed material interactions as findings; the parent and auditor must examine source and cannot prove all hidden relationships absent. Use a schema-2 results object with ordered chunks and audits entries containing id and result_path, then run check --results ... --summary and retain its exact summary in metadata.chunked_review under challenge_scope.mode=scoped. The checker revalidates current original and mirror source, each authenticated child result, audit independence, and the declared interaction and unit verdicts. Binary or symlink source or diffs, oversized chunks, hidden index flags, base-only deleted source, continuations, and scoped stopped summaries remain unsupported; keep them unreviewed and stop the scoped clean claim rather than silently falling back. This scoped profile does not replace the schema-1 exhaustive continuation route.
A prior path is visible only when the retained primary or supporting snapshot contains its actual file, symlink, or tracked-deletion record. A matching name in
scope_pathsorsupporting_pathsalone does not establish reviewer coverage.
For schema-1 chunk manifests and their coordinator results, chunk_diff.py check --results ... --summary emits the checked manifest and results digests plus every validated child or interaction result path and its own score series. Copy this exact JSON to metadata.chunked_review; the validator reruns the check for candidate and promoted results. When all declared child and interaction work is clean, the coordinator result and review gate pass with zero findings. It has no independent whole-scope round, own ledger, combined weighted score, or reviewer receipt. Its chunk-only Results row retains N/A counts; every child's score remains visible through the summary. A separate caller completion gate and semantic coverage judgment still apply.
When chunk results are pending or non-clean, keep a separate schema-1 partial results map and use chunk_diff.py check-stopped --manifest ... --results ... --summary. It validates every supplied child or interaction result through the ordinary artifact gate and reports completed score series plus pending chunk, interaction, declared-group, and prior-signature slots. Copy that exact status: stopped summary to metadata.chunked_review; candidate and promoted validation rerun this mode only for a failed coordinator with a failed review gate. It cannot authorize a clean result or replace the complete schema-1 manifest check. With no validated child result, report no child-backed score.
Current validation applies the same table, scope, action, source, and reviewer evidence checks to candidate and canonical results. Old JSON can be inspected as archive data; its filename does not grant current completion. A clean child or interaction result remains scoped evidence for the coordinator, which still needs its independent interaction and final coverage checks.
Bind existing reviewers¶
Keep loop-evidence.json alongside loop-ledger.json:
{
"schema_version": 2,
"repository": "absolute path of the reviewed repository",
"scope_paths": ["src/parser.py"],
"current_source_path": "current-source.json",
"request": {"goal": "Review parser behavior", "specification": "Accepted input contract", "done_when": "Independent review is clean"},
"supporting_paths": ["src/caller.py"],
"current_supporting_source_path": "current-supporting.json",
"origin": {"kind": "first-run", "caller_run": null, "prior_ledger_sha256": null},
"rounds": [
{
"index": 1,
"source_path": "round-1-source.json",
"supporting_source_path": "round-1-supporting.json",
"review_run": "path of the existing Code Review run",
"role": "challenger",
"triage": []
}
]
}
Each entry corresponds to exactly one ledger round; source and report paths stay inside the loop run. The referenced review run retains its original manifest, routing, source contexts, reports, runtime evidence. Revalidate it with Code Review's manifest-only validator; no substitute or unsupported legacy route qualifies as independent coverage. All participating required reviewers must have inspectable source and independent evidence; selecting one role does not hide another role's missing coverage.
Schema-1 evidence and earlier untriaged schema-2 JSON remain readable as historical data; neither can validate as a current challenge-resolve result. Current schema-2 and schema-3 evidence with origin require the request, supporting-source fields, and a complete parent triage map in every round.
The implementation author must match both the active host CODEX_THREAD_ID and the validated review parent thread; every participating reviewer must differ from that author. Read runtime identity from the executing host, never from candidate artifacts or a manually overridden environment. Missing identity or a borrowed review from another parent prevents accepting that round. Resume under the genuine owning session or obtain a fresh permitted review under the current owner; never rewrite old receipts. Parent-only source inspection may continue with its independence limit disclosed.
The selected role's observed thread identity must match the ledger reviewer. The ledger report is an exact copy of its validated returned output; schema 7/8 native rounds may retain either the normalized selected output_path bytes or the exact selected raw_output_path bytes, both authenticated against the child terminal response. The review manifest input digest, retained review diff, loop round diff must match. Preserve the runtime route's original limits: native lineage and parent-observed local reviewer wave evidence are not interchangeable claims. Native specialist manifests use schema 8 (historical schema 5 inline delivery and schemas 6/7 paged delivery remain readable) and isolated Codex reviewer manifests use schema 4; older schemas require a fresh supported review, not relabeling. No fabricated host declarations, output hashes, or manual accepted receipts.
Bind returned findings¶
Current schema-2 and schema-3 loop evidence records origin as { "kind": "first-run", "caller_run": null, "prior_ledger_sha256": null } or { "kind": "continuation", "caller_run": "<absolute canonical caller run>", "prior_ledger_sha256": "<ledger_digest of caller loop-ledger.json>" }, where shared/adversarial_loop.py ledger_digest hashes a historical schema-1 file's exact bytes and a schema-2 header together with its loop-rounds.jsonl round log. For a continuation, append Prior findings to reassess: followed by sorted JSON of the caller's final-round findings to the request specification. Every reviewed continuation retains each earlier signature with its original tier and structural flag. A clean final review requires a verified-fixed or rejected disposition; a stopped reviewed continuation may retain an open or fixed-pending-verification disposition and reports the prior finding as unresolved. The validator compares the declared caller ledger bytes and final dispositions; it cannot discover an undeclared caller run or prove historical reviewer authenticity. Historical schema-1 and earlier schema-2 evidence without origin stays readable as data; current candidate and canonical completion require schema 2 or 3 with origin.
For schema-2 or schema-3 evidence with origin, assessment.rationale is a JSON object encoded as a string with exactly reviewed_paths, unreviewed_paths, limits, and judgment. List every concrete frozen primary source record and all supporting paths in sorted, unique reviewed_paths; naming a directory alone does not cover its captured files. Every requested primary scope must have a captured source or tracked-deletion record, including at preflight. Additional inspected evidence may also be listed, but does not extend the frozen scope or replace any required path. Set unreviewed_paths to [] only when every required path was covered, and state a nonempty limit and judgment. The validator requires complete declared coverage for current completion. An empty finding set is clean only when the reviewer's source rating is 1. This declaration is a reviewer claim, not proof of line-level inspection.
Schema-3 interaction evidence retains those source, request, coverage and provenance checks and adds sorted interaction_requirements entries with id, source_paths, and claim. Append Interaction assessment: followed by the canonical JSON of those entries to the frozen specification. Every ordinary interaction must name captured endpoint source delivered to the reviewer. The authenticated reviewer output adds interactions entries for exactly the assigned IDs, each with verdict (reviewed or unresolved) and evidence as a nonempty list of nonempty strings. The reserved interaction-inventory-audit requirement also carries sorted unit_ids; its reviewer verdict carries matching typed unit_verdicts. Use an integer assessment rating from 1 to 5, with 1 for an empty finding set, and encode the coverage rationale as a JSON string whose limits and judgment are nonempty strings. Missing, duplicate, extra or unresolved verdicts cannot certify a clean round. A coordinator must independently derive the complete unit inventory and verify its assignment and actual source delivery; schema-3 response validation alone does not prove inventory completeness or semantic absence of hidden interactions.
For local reviewer waves, require one raw JSON object with the fields below, without Markdown fences or surrounding prose; the shared runner enforces this shape through turn/start.outputSchema and validates returned content. Native inspection accepts one raw JSON object or exactly one fenced adversarial-loop block. Schemas 7/8 require no copied provenance header; historical schemas 5/6 retain their provenance header. Historical fenced local reviewer wave responses remain readable. For this loop, the structured response replaces the role's free-form report sections; its review responsibilities and execution restrictions remain unchanged. No introduction, conclusion, additional fence, or finding prose may appear outside the response. Rejected responses remain intact; request a fresh conforming response only through the owning workflow's permitted retry decision, never edit the original output.
{
"source_sha256": "SHA-256 of exact retained source JSON bytes",
"diff_sha256": "SHA-256 of exact retained diff bytes",
"findings": [],
"assessment": {"rating": 1, "rationale": "{\"reviewed_paths\":[\"path/to/source.py\"],\"unreviewed_paths\":[],\"limits\":\"none\",\"judgment\":\"clean\"}"}
}
Native assessments use the two-field rating/rationale contract shown above. A source-rated local reviewer wave additionally supplies source_rating_version: 1 and verification_readiness: "complete"; incomplete or blocked assessments cannot certify a challenge round. Choose a rating consistent with judgment (1 for clean).
For current native preparation, pass --challenge-only to both review_prepare.py prepare and assemble. Declare exactly one challenger in review-routing.json and review-briefs.json; the challenger brief contains the exact Review request:, Frozen source:, Frozen diff:, and optional Supporting source: envelope defined above, ending immediately after its final newline. Preparation retains the role card followed by that exact brief, validates selected source coverage, and does not append the ordinary Code Review task wrapper or source excerpts. Ordinary preparation retains its multi-axis routing contract.
If the first native context read fails before any source evidence is available, retain a preassessment blocker with source_sha256: null, diff_sha256: null, findings: [], and assessment rating 5 plus a nonempty failure rationale. Only authenticated incomplete-dispatch recovery may read this unavailable-digest receipt, after proving no source was delivered and an observed malformed reader call failed. It certifies no source inspection or coverage and cannot become a loop round or completed review. Replacement inspection still requires the supplied exact digests and ordinary authenticated source/context/output evidence. Never invent unavailable digests.
Supply those computed digests and the exact response contract in the frozen context; never ask the reviewer to guess or calculate digests mentally. Each raw finding uses signature, tier, structural, disposition, and evidence as a reviewer proposal. Put all finding narrative, refutations, closure evidence inside those records. Any material missing-input or coverage gap preventing acceptance is a finding, not an omitted prose caveat. Retain coverage and route limits in loop-report.md, source snapshots and original execution evidence. Carry every previous signature with its original counterexample and evidence into the review request, include the source of the validator and next consumer that decide its behavior, and require the fresh reviewer to assess each signature explicitly. In a continuation's final round, each prior signature, tier, and structural flag must occur unchanged in the authenticated reviewer output; parent triage cannot map an unrelated finding onto it, map it away, or restore its tier. A solver's fix action, patch summary, and passing local check remain pending evidence, never a reviewer verdict. After the response, the parent records one schema-2 triage entry per raw signature: reported_signature, stable signature, selected tier, reason, and evidence. Unchanged proposals use empty reason and evidence; identity or severity corrections require a concrete reason and supporting evidence. The validator rejects missing, duplicate, or colliding mappings and binds the ledger to the mapped findings while preserving the raw structural flag, disposition, and evidence. The parent judges whether a correction's rationale is sound. An authenticated report that still finds a defect cannot become an empty clean ledger by parent editing.
When reviewers return the same signature with matching tier, structural flag and disposition, keep one ledger record, combine all exact evidence strings in manifest/reviewer order, removing exact duplicates only. Different evidence wording alone is corroboration, not a verdict conflict. Any disagreement in tier, structural classification or disposition still fails closed; never alter original outputs to resolve it.
After each fresh challenger response, preserve its original findings and create the ordinary ledger before any fix or further review. Validate the ledger, score, actions, reviewer provenance, and exact current-source binding with the existing helpers; missing or conflicting evidence stops the run. Historical response backfills may report scores with original hashes and conflicts, but may not be promoted as current independent challenge rounds.
Run this skill's validate_evidence.py before final promotion; shared validation repeats it. A rejected receipt remains rejected evidence. With no permitted independent review, retain the failed attempt separately, use an empty round list and independence-unavailable, fail the review gate, provide a concrete remediation action. Never fabricate an independently reviewed round to produce a complete-looking artifact.