Experiment retrospective¶
Read a completed managed numeric campaign. Do not execute measurements, modify its source, or change its original history. Historical or pass/fail runs without campaign.json have no managed reconstruction contract: explain that limitation rather than inventing state.
- Create a separate optimize result run. Copy the source run's
campaign.jsonand byte-exactexperiments.jsonlinto it; retain the source path and both original hashes inhypothesis.md. Never replace a file in the source run. If source pending state exists, report unfinished work and keep the retrospective preliminary; do not promote it as complete. - Run
PLUGIN_ROOT/bin/python PLUGIN_ROOT/skills/optimize/campaign.py retro --run-dir <new-run>(native Windows:.cmdlauncher). Save its stdout asretrospective.json. This validates configuration and the full ledger without requiring the original live checkout. - Write
comparison.mdwith baseline, after/best, delta, guard history, confidence and limits. Explain kept/reverted/inconclusive/failed counts, repeated hypotheses and rejected hypotheses worth revisiting. The helper's next-hypothesis list is a queue of prior non-kept approaches, not novel research or an instruction to repeat them unchanged. Add concrete next hypotheses only when evidence supports a changed mechanism. - Flag large jumps relative to the campaign's observed scale for investigation, not as proof of a defect; record any absent per-iteration logs. Adaptive, selected iteration results do not establish statistical significance. Repeated independent trials and an appropriate experimental design are separate work, never fabricated from the ledger.
- Bind the copied campaign/log hashes in
metadata.optimize_lifecyclewith moderetrospective. Keep the existing final columns for every retained iteration or exactly oneIteration=campaignsummary, using the exact numeric, guard and decision conventions in Managed campaigns. Explain in prose that guards and decisions describe historical attempts; they are not new measurements. Mark code lint/type/test execution inapplicable when no executable change or probe occurred; run review and artifact checks and complete the ordinary final handoff. Lifecycle admission validates retained history and reconciles the structured Results cells without requiring live source, but unresolved pending state cannot be promoted.