📦 WP-163 — perf(ptl): validation ground truth crosses to the host once¶
Engineering log · Phase 15 — Audit remediation, v0.8.0
WP-163 — one transfer per batch, and the three candidates that did not survive measurement¶
This row is the surviving quarter of a survey of RF-DETR's 1.9.0..develop range (137 commits, ~40 perf-tagged, Apache-2.0, read under D13 and never copied). Most of what that survey found was either already implemented here or inapplicable; what follows is the part that measured well, and the part that measured badly, because the second is the more useful record.
What the change is. Every validation accumulator moved its ground truth to the host one image at a time: two tensors per image for detection, three for oriented detection and keypoints, and a mask stack for segmentation. Each of those transfers carries the accelerator's fixed per-transfer latency and a payload of a few kilobytes, so a batch of 16 paid that latency 32 to 48 times to move something that fits in a single transfer. _host_split concatenates along the batch axis, moves once, and slices the result back into the same per-image pieces. Values, dtypes and ordering are unchanged, so nothing the metrics read changes and no golden moves.
Measured, not assumed. On MPS, 16 images x 7 boxes: 2.823 ms to 0.210 ms, 92.6% less, median of 40 timed runs after 15 warm-ups with the device synchronised inside the timed region. On CPU the same batch goes from 0.004 ms to 0.013 ms — a 9-microsecond regression, because on CPU there is no transfer to amortise and the concatenation is pure overhead. That trade is taken deliberately: the accelerator path is where validation actually runs, and 9 microseconds against 2.6 milliseconds is not a close call.
Honest scope. Per validation batch this is milliseconds. Across a 5000-image validation split at batch 16 it is roughly 0.8 s an epoch — under 1% of a training epoch. The row is worth landing because it is bit-identical, strictly less work, and removes a per-image pattern that would otherwise be copied into the next accumulator; it is not worth landing on the strength of its epoch-level number, and this entry does not claim one.
Three candidates the measurement rejected.
Skipping the padding rows before the segmentation mask decode. The validation decoder hands decode_instance_masks a fixed 100 rows per image and decoded[keep] discards the padding immediately after, so filtering first is bit-identical and strictly less work — and it is slower on the accelerator: CPU improves 32–67% depending on survivor count, MPS regresses 40–86% at 40 repeats. Boolean indexing plus the data-dependent sync costs more than decoding 100 rows of a 160x160 grid in parallel. Rejected rather than adopted device-guarded: the guard would exist to serve the offline path at the accelerator's expense, for a proxy metric measured in milliseconds. Upstream made the same call in the other direction for one of its own changes, excluding MPS explicitly.
Hoisting torchmetrics' per-annotation score reads. CocoBackend._get_coco_format hoists boxes and labels once per image but reads area[image_id][k] one annotation at a time, and it accepts scores=None, so the scores can be assigned per image afterwards. Measured 26.3% and 26.6% faster at 32 and 128 images of 100 detections. Not adopted: reaching it requires calling a private method on a private attribute (metric._coco_backend._get_coco_format), which binds this project's validation path to torchmetrics internals that carry no compatibility promise, and the epoch-level saving is around 0.16%. The measurement is recorded so the trade can be re-taken if torchmetrics ever exposes it.
The uint8 / 255 divisor form. Upstream found that CUDA evaluates a division by the Python int 255 as a multiplication by its reciprocal, landing one ULP from the host result for 126 of 256 byte values, and switched to a 0-dim tensor divisor to restore parity. This repository's consumer conversion uses the same Python-int form, so the question was live — and it does not reproduce here: across all 256 byte values, both divisor forms match the host bit-for-bit on CPU and on MPS, 0 of 256 differing. A26's cross-platform divergence is not this mechanism. Recorded as a closed question rather than left as a plausible-sounding hypothesis.