📦 WP-150 — refine(tests): rescope seed-driven assertions as tier-D determinism¶
Engineering log · Phase 13 — Contributor admission, augmentation guard and keypoint parity, v0.6.0
WP-150 — the tier that is allowed to move¶
Six determinism assertions were scattered across five test files, each sitting beside the behaviour assertions for its own transform: TestDeterminism in test_affine.py, test_fused_warp.py and test_mosaic.py, TestMixupDeterminism and TestCopyPasteDeterminism in test_mixup_copypaste.py, and two seeded cases inside test_photometric.py's jitter and flip classes. All six now live in tests/data/test_aug_determinism.py, labelled tier D.
The move is not tidying. It is about what a failure will mean. Every assertion in the file is keyed to a seed, and a seed pins whichever sampler happens to be installed. A replacement engine will not draw the same numbers from the same seed — different call order, different distributions, different consumption of the stream — so these values moving at an implementation swap is expected and says nothing at all about correctness. Tiers A, B and C moving at a swap is a defect. Those are opposite verdicts from the same red test run, and the only thing that tells them apart is which file the failure is in.
Mixed in, the two verdicts share a keystroke. A tier-D failure at swap time reads as a regression, and the natural response — re-freeze it, the values were always going to move — is exactly the response that would silently destroy a tier-B guard sitting three classes above it in the same file. Separated, the response follows from the filename. This is the same argument the plan makes for never re-freezing an expectation in the same commit as a call-site swap, applied one level up: keep the thing that is allowed to move away from the thing that is not.
Two counterparts were added rather than only moved. Tier D is a two-sided property — same seed gives the same output, different seeds give different output — and only the first half existed. The second half is what catches a transform that ignores its generator and reads the global RNG: such a transform passes the equal-seeds case whenever the global state happens to line up, which under a seeded test suite is most of the time. That is not hypothetical here; WP-079 was opened by an augmentation RNG collapse. The flip case gained a related assertion, that sixteen draws at p=0.5 contain both outcomes, so a trigger stuck on one branch fails rather than agreeing with itself.
test_letterbox.py's TestDeterminism stayed where it is, and that is the useful distinction. It asserts that two calls on the same input return the same output — purity, not reproducibility. Letterbox has no generator, nothing about it is keyed to a seed, and it will not be re-frozen at any swap. A rescope that swept every class named TestDeterminism into the tier-D file would have moved it, and the name is the only thing the two have in common.
No value in the tier-D file appears in goldens/, and none should: reproducibility is a property of a run rather than a number worth carrying across releases.