WP-198 — overlap input preparation and fuse the colour restore¶
Engineering log · Phase 18 — CUDA graph replay and pipeline throughput
WP-198 — prepare the same training images with less exposed work¶
The CUDA owner profile at batch 64 and 640 px attributed 113.406 ms to HSV, 9.999 ms to quantization and 5.481 ms to dequantization, before the channels-last copy. The operator trace materializes full-size intermediate batches around the existing pointwise arithmetic.
The transfer hook now requests one Triton pixel pass for contiguous float32 CUDA images and stated float64 gains. It preserves expression order, precise tensor division, double addition before casting the saturation/value gains, nearest-even uint8 rounding and CUDA's scalar-divisor reciprocal on restore. Fused multiply-add contraction is disabled. A fresh channels-last output avoids both intermediate batches and the final layout copy. The 256-pixel launch was selected from a measured 128/256/512/1024-block matrix; it is a scheduling choice. R44 registers the optional compiler, not a method implementation.
The existing device_hsv opt-in remains the entrance. Missing compiler/backend, unsupported devices, capture/compile scopes, noncontiguous inputs and other consumer precision keep the native path. The release guard accepts explicitly declared consumer extras only when their imports handle absence; development-group and unguarded imports still fail. No tensor, output or gain is cached.
Without CUPTI, a two-epoch nano run spent 13.650 s waiting for the first training batches and 3.222 s across the other 62 batches. The consumer can hide part of that first-batch work: an opt-in epoch_prefetch loader resets its existing persistent iterator after fetching the final batch, so workers prepare the successor during the last step and validation. Each public iterator still ends at its own epoch. The module permits this only for complete, single-CUDA epochs with its dedicated random sampler; distributed sampling, limits, step bounds, overfit and loader reloads retain the ordinary loader.
The shuffle generator is separate from the model's global RNG. An unused prepared epoch carries a generator-state receipt so teardown and a fresh worker pool do not skip a shuffle. A synchronous reset failure restores that state and is reported when the next epoch is requested, after the already-fetched batch has been delivered. The callback keeps a weak reference to its Trainer, and the prepared marker weakly references the existing iterator so Lightning teardown can release unused prefetched batches after an early stop. A regression through the real CombinedLoader reset catches strong retention. It creates no additional pool or executor. Queued training batches can remain resident during validation, which is why the flag is opt-in.
The isolated L4 screening pair used 2048 public COCO2017 training images, 64 disjoint validation images, nano/B64/640/BF16 mixed, eight persistent workers and cached decoding. Each epoch is 32 training steps (2048 images at batch 64). Three complete epochs, including validation and checkpoints, took 131.179 ->119.761 s; whole processes took 138.139 ->126.883 s. All six model, optimizer, scheduler, EMA, loss and metric hashes matched. This is a screen, not the repeated headline. Five fresh-process pairs over ten epochs, alternating order with an empty compiler cache per fit, reduce median complete-fit time from 379.619 s (377.463–380.130) to 326.999 s (325.952–355.677), or 13.86%. Whole-process medians are 386.249 s (384.025–387.338) and 333.961 s (332.686–362.380), or 13.54%. The median CUDA-event steady window is 807.293 -> 727.509 ms per step. All ten fits retain identical six-state byte hashes. The slowest candidate is retained; paired fit reductions range from 6.43% to 14.09%. Strict determinism and allocation filling remain enabled. Full make gate was not run and this package remains open.
The 13.86% headline combines two features, each measured separately: colour fusion alone saved 7.15% in the ten-epoch screen, and epoch preparation alone saved 4.50% in its three-epoch screen. A CPU CopyPaste/rasterization combination reached 7.45% with fusion and was discarded. Epoch-boundary savings depend on dataset size and validation duration; these short-fit measurements are not a full-COCO training claim.