📦 WP-193f — perf(ptl): replay independent CUDA pose prediction stems¶
Engineering log · Phase 18 — CUDA graph replay and pipeline throughput
WP-193f — independent pose stem backwards preserve absent gradients¶
The small-batch CUDA profile still shows host-driven pose prediction launches after shared-feature replay. Record each of the existing eighteen box/class/point stems independently, using R43's graphed-callable API and the private prediction runner from WP-193e. The existing capture_network flag remains off by default. No model, assignment, loss or optimizer arithmetic changes.
Each stem has its own autograd node. If a batch's objective does not read its point output, no point backward runs: gradients remain absent and the optimizer skips the same parameters as native execution. A fixed joint head backward would instead materialize zero gradients. The point/uncertainty views and their concatenation retain their native order.
The shared recorder owns one bounded input signature per stem, including its input's requires_grad value. Differentiable feature samples retain that value when recorded. Existing buffer/RNG rollback, storage, hook, precision and pending-backward checks apply independently. The fit callback resets skipped-backward holds and clears the child holders on completion or failure. Deepcopy and pickle start with no recording. Replacing a head module retires the parent before feature replay, releasing obsolete pools. A stem's changed trainable set retires only its own recording; shared-feature freezing keeps its existing full-fit native fallback.
[MEASURED] Five alternating fresh-process pairs on an isolated g2-standard-8 (eight vCPUs, 32 GB), NVIDIA L4, driver 580.178.04, torch 2.9.1+cu129, Lightning 2.6.6 and Python 3.11.14. The workload is nano pose (one class, 17 points), batch 4, 640 px, BF16, five epochs/640 steps, seed 347, 512 original COCO val2017 training images and 15 disjoint person-annotated validation images. Training retains 226 images without person annotations. This is a mosaic-active training phase. Both arms enable prepared caching, fused device HSV, epoch prefetch, existing shared-feature replay, orthogonalization replay and optimizer-update replay.
The complete timer begins before torch import and ends after synchronized Trainer.fit return: construction, cache preparation, graph setup, augmentation, forward/backward, clipping, optimizer, schedules, EMA, validation and last-checkpoint writes are included. Diagnostic hashing and process teardown are excluded. OS/dependency caches are warm; each process rebuilds its prepared-image cache and CUDA recordings. No profiler, package installation or other training job runs during the cohort.
| Metric | Existing feature replay, median (range) | Independent stem replay, median (range) |
|---|---|---|
| Complete runtime, s | 127.302 (126.592–128.085) | 110.795 (110.309–112.551) |
| Post-warmup start-to-start step, ms | 159.569 (155.840–167.159) | 136.184 (132.954–137.304) |
| Peak allocated, MiB | 672.620 | 587.081 |
| Peak reserved, MiB | 1384.000 | 1730.000 |
Complete runtime falls 12.97% from the arm medians. Paired reductions are 11.59–13.50%. The model, optimizer, scheduler, EMA and complete per-step-loss byte hashes agree across the ten fits, as do validation metrics; the five baseline controls reproduce themselves. Sixteen complete start-to-start iterations are timed after ten warmup steps with CUDA synchronization at both boundaries. Memory peaks are identical within each arm. Reserved memory increases despite lower allocated peak; no OOM or capacity benefit is claimed.
Focused host/CUDA checks cover FP32, FP16 and BF16 values, input/parameter gradients, changing point supervision, real graph execution, native Trainer/scaler/optimizer behavior, temporary shape fallback, ownership replacement and changing stem trainability. A private mutation removing stem dispatch fails the regression rather than passing as two native runs.
A separate three-epoch batch-16 exploratory pair is 42.429 s before and 40.945 s after, 3.50% less time, with matching state hashes. It is one pair and does not clear this task's 8% floor. The claim is limited to the repeated batch 4 workload. Full-COCO throughput, other accelerators and long-run acceptance accuracy remain unverified. Stem replay ran on torch 2.9.1 only; the torch 2.4 FP32/FP16 checks named in TRAINING.md cover detection, oriented detection and shared-feature replay. Full make gate, aggregate offline coverage and accelerator goldens were not run; this package remains open.