📦 WP-132 — feat(ptl): keypoint training path¶
Engineering log · Phase 12 — Keypoint detection, v0.5.0
🔗 Depends on WP-121, WP-122, WP-123, WP-124, WP-131 · Needed by WP-125, WP-134, WP-135, WP-137, WP-139
WP-132 — every part built, nothing connected¶
Phase 12 shipped five consecutive rows of keypoint work and never once ran a keypoint. WP-120 added the target channels, WP-121 the COCO reader, WP-122 the head stem, WP-123 the RLE loss, WP-124 the OKS scorer -- each landed green, each with its own passing unit suite, and the composition none of them owned did not exist. DetectionLitModule's task enumeration still read detect/segment/obb, so the keypoint stem was never constructed by anything; RLELoss was exported from losses/__init__.py and had zero call sites outside the module that defined it; keypoint appeared nowhere in ptl/module.py, ptl/datamodule.py, data/collate.py or models/build.py. The suite was green because every test was a component test, and a component test cannot fail for a wire that was never run.
The trigger for finding it was a question about running the overfit gate, not a failing test -- which is the part worth recording. Nothing would have surfaced this until someone tried to train, because "wire absent" and "wire correct" are indistinguishable to a test suite that only ever exercises the parts. WP-087 and WP-088 exist for exactly this reason and are the precedent this row follows: each is a training path row that carries its own overfit-100 golden as DoD, rather than a component row trusting that a later integration will notice. The gate is the thing that makes a wire testable at all.
Additive, not substitutive. The RLE term enters the detection total at keypoint_gain (A68) alongside the box and class terms, which makes this WP-087's structural shape rather than WP-088's -- the oriented path is the one task that replaces terms, zeroing box_gain/l1_gain inside the dual loss and respending them on rotated equivalents. Keypoints add a channel of supervision without contesting any existing one, so the arrangement is the simpler one and the keypoint_gain=0 bit-exactness test is what proves it: with the gain at zero the detection total is bit-identical to a detect run, so the term is genuinely additive and the gain genuinely consulted rather than folded in somewhere upstream.
The whole transform layer, after a scoping that was wrong twice. Every geometric transform rebuilds Targets(...) from scratch, and all of them dropped the keypoint fields on the floor -- letterbox.py, mosaic.py, affine.py, mixup.py. The first scoping fixed Letterbox alone and argued the boundary was principled: _warp_targets is a pure forward affine that resizes and pads, never crops, so no point can leave the canvas and there is no policy to decide, where mosaic/affine/mixup crop and would force a clamp-vs-demote decision an augmentation-off gate could not settle honestly. Every sentence of that is true and the conclusion was still wrong, because the premise it rested on was never checked: the claim "augmentation is off" was verified against the validation loader, which is letterbox-only, and generalized to the training loader, which is not. _TrainPipeline runs mosaic, FusedAffineLetterbox, mixup, copy-paste, HSV and flip unconditionally. The gate died on pad_keypoints' instance-count mismatch on its first batch, and the fix was the whole layer -- which is what made A70 this row's decision instead of WP-125's.
The loud bug was hiding a silent one. Re-reading the transform path after that failure -- rather than only the line that raised -- turned up HorizontalFlip constructed at datamodule.py:767 with no keypoint_flip_pairs argument at all. The flip is K-generic by design and cannot know which index mirrors which, so the omission did not mean "no swap needed", it meant every mirrored sample supervised flank_left toward the location of flank_right. That trains, converges to a left/right-confused head, and reports nothing: no shape check fails, no loss value looks wrong, and the overfit gate would very likely still have passed. Worse, the first draft of A64 asserted the pairing was being threaded through -- a documentation claim written from what the code should have done rather than from what it did, and corrected only because the affine's hard error forced a second reading of the same path. A64 now resolves by reading the pairing off the dataset's own COCO category names (left_eye, flank_left), which is where the anatomy is actually stated.
Symbols over animals, for reasons that outrank "simpler". R21's SymbolShape family carries K=7 against the animals' K=16, and its landmarks are placed analytically -- three-quarters of the way from each outline's area centroid toward a named vertex -- rather than traced anatomically. Two consequences beyond the cheaper stem. Only 5 of 7 slots are populated on most symbols, so a symbol run carries v=0 points natively and exercises A66's visibility exclusion inside a real training loop instead of only in test_rle_loss.py. And because the points are analytic, R12's OKS sigma table -- 17 values derived from measured annotator standard deviations on human anatomy -- describes a quantity that does not exist here, which is what A67 records: a uniform sigma is not a simplification of the real table but the only honest reading, since any per-point vector this project invented would fabricate a structure the data does not have.
The pin bump was not fixture-neutral, and the frozen tree could not absorb it. Reaching SymbolShape meant advancing R21 from 74af985 to 0a0cc64, a commit that also redefines GeomShape.TRIANGLE from equilateral to obtuse-scalene and re-centres every outline on its area centroid. Both moved the existing fixtures substantially -- obb_num_annotations 44 to 61 (+38.6%), data_checksums.bbox_area_sum -45.2%, image_std_sum -35.5% -- and eight of 27 goldens went red. Six of those eight were in goldens/frozen/0.2|0.3|0.4, the per-release snapshots whose whole purpose (check_goldens.py: "a release's frozen goldens staying green is the frozen-golden regression") is that current code still satisfies every value a past release pinned. Re-freezing those would have rewritten what v0.2.0 actually produced, so the question went to the principal rather than being resolved by the row that raised it. What decided it was that the failures partitioned exactly: every pure-code frozen golden (assignment_cases, optim_toy, params_flops_*) still passed, and the only two that failed were data_checksums and fixture_checksums -- the two whose producers render images through an external package. A generator-derived metric can never satisfy "current code must still satisfy every past value" once the generator changes, so those two were never really freezable; they were a category error in what got snapshotted, not a regression. They were removed from the frozen release tree (recoverable in history) and the two current copies re-frozen, leaving 21/21 green and the frozen-golden regression still meaning something for everything it can actually test.
One flow or two, and a gate that cannot tell. Wiring the RLE term forced a choice neither R14 nor R1 speaks to: the dual head has two branches, and a normalizing flow is the only trainable-weight loss in this project, so the term either shares one density across both branches or fits one per branch. One shared flow was taken (A69) on two arguments that are each weak alone -- it is what R14 actually describes, one density over one regressor's residuals, where two would assert the branches' errors differ in kind rather than merely in schedule; and it is the smaller parameter count. What matters more than the choice is that this row's own gate cannot check it: an overfit-100 run drives both branches to the same near-zero residual, which is precisely the regime where one flow and two are least distinguishable. So A69 ships open with WP-125 named as the first run whose branches disagree enough to tell, rather than being quietly ratified by a green gate that was never sensitive to it. The same caution applies to A68's keypoint_gain = 1.0: a memorization run will pass at almost any gain, so passing is not evidence the weight is right.
Two numerical faults that no component test could have caught. With the wiring done and the transforms carrying points, the gate still would not run -- it went non-finite inside the RLE flow, and the two causes are the clearest evidence this row could offer for why a training path is its own work package.
The first is a frame mismatch (A71). decode_keypoints returns absolute input pixels, deliberately and correctly -- it is the deployed decode. R14 forms the residual as (mu_g - mu_hat) / sigma_hat and bounds sigma_hat into (0, 1) with a sigmoid, which it can do because it regresses inside a top-down person crop: the crop is the normalization, so the paper never names a frame and this project had no reason to notice one was missing. Composed, the two make a residual whose smallest expressible value for a 40 px error is 40. Measured on the first batch, the keypoint term entered at 1300 of a 1342 total loss -- three orders above the box and class terms it is supposed to sit beside -- and the flow overflowed on the second step. Normalizing both point tensors into the assigned box's frame first brings the median residual to 1.28. Nothing about this is visible to test_rle_loss.py, which evaluates the loss at O(1) residuals because that is the regime the equations describe; the loss was never wrong, its input frame was unstated.
The second is a missing stabilization (A72). WP-123 built the coupling conditioner from R14's stated recipe -- 3 fully-connected layers, 64 units, Leaky-ReLU after each -- and emitted its raw linear output as the log-scale. R14 cites RealNVP for the coupling construction rather than restating it, and the parameterization of s is stated only in Dinh et al.: "To compute the scaling functions s, we use a hyperbolic tangent function multiplied by a learned scale", given there explicitly as a stability measure. A raw linear log-scale is not a simplification of that, it is a different layer, and the stack multiplies exp(log_scale) six times so the difference compounds. Measured after the frame was fixed: a residual of 27 arrived at the latent as 4e11 and at the loss as 7e22. The general lesson is narrower than "read the citations": WP-123 read R14 closely and implemented what R14 states, and what bit it was a detail R14 delegates -- the one place a paper's own citation is load-bearing rather than contextual.
The gate's number is small and that is the ruler, not the model. With both faults fixed the gate runs its 100 epochs clean and scores OKS AP 0.3357 over 548 instances -- next to the detection gate's 0.995, the oriented gate's 0.939 and the mask gate's 0.815, a number that reads like a failure. It was checked before it was believed, and the check says otherwise. Feeding the ground truth back through the scoring path as the prediction returns exactly 1.0, so the harness, the sigma vector and the label space are all wired correctly and the ceiling is real. Displacing every point of every instance by a uniform offset then measures the slope: +1 px scores 0.934, +3 px scores 0.269, +8 px scores 0.000. On symbols a few dozen pixels across, scored at A67's uniform sigma, OKS AP is a cliff rather than a slope, and 0.3357 is where a landmark error of roughly 3 px lands. The floor was set from that measurement -- 0.30, with the same 0.05 tolerance the oriented gate uses -- rather than from the 0.75 that had been guessed before any number existed. One honest limit on the reading: the probe displaces every point uniformly, so it bounds the interpretation without proving it; the same AP is also consistent with a mixture of exact and badly-missed instances, which only a per-instance breakdown would separate.
Both faults were found by running the gate and neither by any test in the suite, which is the same shape as the wiring gap this row opened with. What is deliberately not claimed here is that the numerics are now healthy: the residual tail still spikes into the thousands on this data, the A70 off-canvas points meeting a flow whose base density is quadratic in the residual. The tanh removes the compounding feedback path; it does not make the tail bounded, and A72 says so rather than letting a green gate imply otherwise.