🎞️ Phase 18 — CUDA graph replay and pipeline throughput

WP-183 — the box terms scored densely, so the step traces as one graph

A boolean gather is a shape the compiler cannot know. pred_boxes[fg_mask] lowers to aten.nonzero, whose output length is a property of the data, and torch.compile answers that with a graph break; the detection step carried seven of them, at the masked gathers. Recorded 2026-09-18. The replacement evaluates every term over the full (B, A) anchor grid — CIoU through a reshape(-1, 4) since its contract is two-dimensional, ProbIoU directly since it broadcasts, the L1 differences against the dense stride, the angle term over the whole branch — and selects the positives afterwards. The selection is a torch.where, not a multiply by the mask: a background anchor's target is the all-zero box, so the dense CIoU there is scored on a degenerate pair, and 0 * NaN is still NaN where where is an exact selection in both the value and the backward pass. What remains is a dense sum in which every off-positive term is an exact zero, and square_angle_loss already documents whole-branch input with q_i = 0 background as its intended second use.

Loss-identical, measured rather than argued. On a COCO batch the two forms differ by an absolute 0.0 on every term, the gradients stay finite on an image with no positives, and the pre-change CPU step snapshot holds at its existing tolerance; in eager the step is 2% faster, with the seven nonzero calls gone. The masked implementation is kept verbatim as a private oracle in each test module, and the dense form is held against it on random assignments — an image with no positives, an all-background batch, and degenerate predictions (zero-area, inverted, all-zero, both sides collapsed) planted on background anchors — to 1e-6 on values and gradients. The oriented oracle draws gt_index from [0, N) with N ≥ 2, so the gather over the padded instance axis is exercised on every column rather than only its first. WP-184 is what the one-graph step is for.

What the accelerator gate said afterwards (2026-09-19). On an A100-40GB, two of the five goldens/gpu/ rows moved out of band under this change alone: keypoints 0.5696 → 0.4919 and oriented 0.9783 → 0.9088, per-commit on the same machine, with WP-182's layout change accounting for −0.021 / −0.040 of the path and detection, segmentation and shapes_regression_det holding. The arithmetic is the same; the summation order is not, and the overfit gate is a 100-epoch trajectory that amplifies a 1e-7 per-step difference into a different endpoint — the same sensitivity the WP-177 escalation measured across seeds on unchanged code. The principal resealed both rows at the measured values (ESCALATION 2026-09-19, WP-186); the oriented band is now clamped_tolerance-narrow at ± 0.0088.

And what three seeds said (2026-09-20). The control the escalation asked for, run per commit on the same A100: the oriented move that looked monotone at HEAD splits into WP-182's −0.020 (3 of 3 seeds down) and this change's −0.014 (2 of 3), and keypoints span 0.49–0.73 across nine runs of the same objective. A layout change with no arithmetic in it producing the same signature as the loss rewrite is the evidence that the signature is the gate's, not the change's. Full table in ESCALATION 2026-09-19's follow-up.

WP-188 — notebooks as sources, the .ipynb as a build product

The format question was decided by what the gate can read. Recorded 2026-09-21. A notebook is worth publishing only while it still runs, and nothing about publishing one checks that; so the first decision was that every notebook is a .py the existing machinery already handles -- ruff lints it, license-headers scopes it, and tests/notebooks/test_notebooks_run.py runs it as a script under LUCID_NOTEBOOK_FAST=1 on every make gate. Jupytext's percent format is the one that costs nothing: a # %% line is a comment to Python and a cell to the converter. The .ipynb follows from the .py at build time in two places, docs/notebooks/ for the site and notebooks/<name>.ipynb on gh-pages for the Colab badge, and neither is ever committed to main; no-ipynb-tracked reads the index on every commit because .gitignore is advice and git add -f ignores it (D25). The script run ruled out one thing: the Colab bootstrap could not be a !git clone line, because jupytext writes a magic into the .py as a comment and a guarded block of comments alone is a SyntaxError when the file runs as a script -- so the bootstrap, and after it every console command the demo notebooks ran, went through subprocess.

The script run is withdrawn; the gate executes the notebook through a kernel. Revised 2026-09-21, once the four demo notebooks existed. Each of them is a sequence of console commands -- lucid-data download, lucid-yolo fit, lucid-eval, lucid-predict -- and as subprocess.run([...]) lists they read as argv arrays, not as the command lines docs/TRAINING.md spells. The user's ask was that they read as plain shell, and that decides the gate's mechanism rather than the other way round: each command is a %%bash cell -- # %% language="bash" in the .py with the script commented beneath, comments to a script run -- so the proof moves from "run the .py" to "convert the .py and execute the .ipynb with a kernel". The test now runs jupytext --from py:percent --to ipynb --execute --set-kernel python3 --run-path <repo root> per source, the same conversion make notebooks makes plus execution, and the subprocess reasoning above no longer applies because nothing runs the file as a script any more; the bootstrap is !git clone and %pip install -e lucid-YOLO under the same google.colab guard, with os.chdir and the sys.path insert keeping the block a block. Values cross into a shell cell through the environment -- the Python cell above does os.environ.update(DATA_ROOT=..., FIT_ARGS=...), the fast budget being one such string of flags -- and a switch the shell itself has to make reads $LUCID_NOTEBOOK_FAST, so the download the fast run skips and the lucid-predict loop only the fast run makes are if blocks in bash beside the command they guard. Three things were measured rather than assumed. A ! line does not fail the notebook -- !false followed by a print printed, under nbclient, with the cell green, IPython storing the status in _exit_code and continuing -- which is why the commands are %%bash cells and not ! lines: the script magic raises CalledProcessError on the cell's exit status and nbclient stops, as a probe with false confirmed; that status is the last command's, so a cell with more than one opens with set -e. --set-kernel -, jupytext's spelling for "the current interpreter", raises No kernel found that matches the current python executable: the kernelspec ipykernel's wheel installs into .venv/share/jupyter/kernels/python3 has argv[0] equal to the bare word python, and jupytext compares that to sys.executable with os.path.samefile. --set-kernel python3 names that spec directly, and jupyter_client.manager.KernelManager.format_kernel_cmd swaps a bare python for the launcher's sys.executable -- the venv's, since pytest runs jupytext as python -m jupytext -- which a probe confirmed from inside the kernel; so there is no ipykernel install step, and the test sets JUPYTER_PATH to the venv's share/jupyter so a user-level python3 spec, which the search order otherwise puts first, cannot take the name. And --from py:percent is not optional: jupytext's guess_format counts an indented # !git clone -- the Colab block's -- as an unescaped magic, because its escaped-magic test is not line.startswith("#") with no lstrip, and guesses hydrogen, a format that leaves every # ! line a comment. A jupytext header after the SPDX line and a [tool.jupytext] table in pyproject.toml were both tried and neither reaches the CLI reader, so the flag went into the test, the Makefile and docs.yml together; without it the published .ipynb would carry # !git clone as a comment and the badge would open a notebook that clones nothing. The dev group gains jupytext, nbconvert and ipykernel -- nbconvert and not nbclient, because jupytext --execute imports nbconvert.preprocessors.ExecutePreprocessor and declares neither as a hard requirement -- and the licence audit on the dev environment stayed clean at 152 distributions. Under LUCID_NOTEBOOK_FAST=1 the five sources take 18–32 s each through the kernel on an Apple-silicon CPU (the four-task wiring gate the 32 s), kernel start included, against the test's 60 s ceiling, which therefore did not move. The full runs' paths were settled at the same time: a dataset goes under the gitignored data/ of the checkout -- data/coco2017 for the three COCO demos, data/dota and data/dota_tiles for the oriented one -- and the run under results/demo_<task>, the Colab detect demo keeping its Drive guidance; the fast run keeps its temporary directory.

The demos moved off the shell cells; the kernel run stays. Revised again 2026-09-21, once the four demos were rewritten onto the library: every step is now a call on an object the kernel holds -- download_coco, DetectionLitModule and DetectionDataModule built from the tier's YAML values, Trainer.fit and Trainer.validate, lucid_yolo.predict and lucid_yolo._viz -- and the console command is quoted as prose, never run, so the %%bash cells, the os.environ hand-off and the $LUCID_NOTEBOOK_FAST test in bash are gone from them. The three measurements above still stand as measurements, they just no longer bear on the demos. What keeps the gate on jupytext --execute through a kernel rather than a script run is the bootstrap: !git clone and %pip install -e lucid-YOLO are magics, written into the .py as comments, and a script run would read them as comments and prove nothing about the two lines a Colab reader depends on; the --from py:percent reasoning is theirs as well. One consequence for a figure cell: its last expression is None if FAST else prediction_grid(640), because a bare name as the cell's last line is B018 to ruff and a call is not. Measured on an Apple-silicon CPU, the four demos take about 11 s each under the fast contract, against 18-32 s through the console scripts. The gate notebook went the other way for the same reason, one file later: the four overfit commands and the shapes regression are development tooling with no library entry point, so notebooks/development_gates.py runs them as the ! lines TRAINING.md spells, $PROBE appended under the fast contract -- --epochs 1 --batches 4, a probe override the overfit script gained on the shapes script's model, reporting PROBE with the score and enforcing no floor; 26 s for all five commands.

What the scratch-venv audit found. The docs group alone resolves to 43 distributions with jupytext added and 77 with mkdocs-jupyter, the 34 between them being nbconvert, nbclient, ipykernel and what they pull; pyzmq (BSD-3-Clause) is the one shipping a native binary the audit's wheel scan walks. Licence audit clean at both counts, nothing copyleft. docs.yml's build job is the only CI environment holding this tree, and its audit step already runs before the build, so the exposure is checked where it enters. The site is rendered with execute: false, so that job stays torch-free, which is what keeps the docs group small enough to audit by hand as well.

The --strict risk was checked with a throwaway notebook and did not fire. The plan flagged that mkdocs-jupyter might warn on a converted file missing kernel metadata, which --strict would turn into a failure. A percent-format source with one markdown cell and one code cell, converted by jupytext and listed in the nav, built strict with no warning and rendered its cells; jupytext writes the kernelspec. The source and its .ipynb were deleted after the build. Nothing in the shipped tree depends on this, since WP-188 lands zero notebooks -- it is recorded so WP-189 does not re-derive it.

Two checks moved from the site to the audit, and one rule changed shape. check_every_nav_entry_points_at_a_file_that_exists used to ask whether the nav target was a file under docs/; a notebooks/<name>.ipynb target now resolves against notebooks/<name>.py instead, whether or not the .ipynb has been written yet. The alternative -- accept the entry when the generated file is on disk -- would have answered differently on a warm checkout and a cold one, and a checker whose verdict depends on whether make notebooks was run is a checker that passes locally and fails in CI. check_every_notebook_source_has_a_nav_entry is the page rule (a register nobody can navigate to is one nobody reads) applied from the source side, and it tolerates a mkdocs.yml with no Notebooks section while no source exists, because MkDocs rejects a nav section with no children and the section therefore cannot arrive before the first notebook does.

The badge decided how the site is published. The first draft of this row added a publish-notebooks job writing the .ipynb files to an orphan notebooks branch, beside the existing Pages artifact deploy -- two publishing mechanisms for one site. The revision (2026-09-21) collapses them: the site itself now goes out as a branch deploy, ghp-import -n -p committing site/ onto gh-pages on top of that branch's history, and the converted notebooks are copied to site/notebooks/<name>.ipynb before the import so the badge's blob/gh-pages/notebooks/<name>.ipynb is a path on the branch Pages serves. mkdocs-jupyter's include_source places its own copy at site/notebooks/<name>/<name>.ipynb, beside the rendered page; the flat copy is the badge's, and both ride the same commit. ghp-import arrives with mkdocs, so the docs group gains nothing for it; deploy-pages, upload-pages-artifact, configure-pages and the pages/id-token permissions leave the workflow, and contents: write sits on the one job that remains -- a pull-request run holds it too, since a job's grant is fixed for the job, and never reaches the step that uses it. The Makefile and both workflow loops iterate a shell glob with an existence test rather than $(wildcard), since for f in ; is a syntax error under /bin/sh and an empty notebooks/ is the state this row ships in.

WP-189 — the drawing code moves into the package, and the dependency it already had is declared

Four notebooks would have carried four copies. Recorded 2026-09-21. WP-188 made notebooks/*.py gate-run sources, and the first four notebooks each want two figures: a run's curves and a checkpoint's predictions over a few images, labels beside them. Both existed — scripts/plot_training.py drew the first and scripts/draw_predictions.py the second — and a notebook cannot import a script, so each notebook would have re-spelled the drawers inline. That is the failure the WP-067 entry names from the other direction: a notebook nothing tests rots, and drawing code copied into four of them rots four ways at once, each copy free to draw the rotated box as its envelope again. So the functional core of both scripts moved to src/lucid_yolo/_viz/ — curves.py for the panels and the CSV reader, overlay.py for the four per-task drawers, the renderer table and the checkpoint-to-axes path — and each script is now its argument parse, its backend selection and its write. The notebook entry points are two: plot_curves(metrics) returns the report's figure titled by the run directory, and show_predictions(checkpoint, images, annotations) loads the checkpoint once, draws one panel per image, and lays the labels under the predictions when an annotation file is given; a notebook that already holds the trained module draws each panel with predict_and_draw(module, image, options, axes=ax) and draw_ground_truth(ax, targets, task) instead, the skeleton for a pose grid travelling in RenderOptions.skeleton so the four tasks share one call shape. Every public name the scripts published stays importable from them as a re-export, so scripts/_tests/test_draw_predictions.py keeps its command-level cases and the drawing cases moved to tests/viz/, where the code they assert now lives.

Ground truth is dashed, unfilled, and read by the training reader. The new drawer, draw_ground_truth(axes, targets, task), draws the label layer in the task's own geometry — boxes, instance rings, rotated quadrilaterals through the same rboxes_to_polygons the predicted rings use, boxes with a hollow ring at each visible point — and never fills or writes a score, so a prediction drawn solid on the same axes stays tellable from its target without a legend. What it takes is a Targets, which is what CocoDetectionDataset yields per image in the reading mode the task needs (oriented for obb, keypoints for keypoints); the file is not parsed a second time. The dataset needed two accessors for that — file_names, the companion of image_ids for a caller who arrives with a file rather than an id, and targets_at(index), the precomputed targets without the image decode __getitem__ performs — and both are the WP's one edit outside _viz, the scripts and the tests.

Private, and matplotlib is a runtime dependency — because it already was. The WP-067 paragraph kept the drawers out of the package on the claim that matplotlib sits in the dev group and "a wheel a consumer installs pulls no plotting stack". Checked here rather than repeated: pytorch-lightning[extra], in [project].dependencies, requires matplotlib>3.1 (importlib.metadata.requires("pytorch-lightning") lists it under extra == "extra"), so every consumer install has pulled matplotlib since that extra was declared, and torchmetrics imports it the moment it finds it installed. The claim was false before this row and the row changes nothing about what a consumer receives. What it does change is the declaration: matplotlib>=3.8 moves from the dev group to [project].dependencies, on the jsonargparse and tqdm precedent already in that list — a direct import is not a transitive dependency — and because WP-168's release guard reads declarations, not the transitive graph: check_dependency_tiers refused the tree with _viz importing a group-only distribution, and five scripts/_tests/test_release_guard.py cases (test_happy_path_passes, test_main_without_tag_falls_back_to_head, test_metadata_only_leaves_gate_command_unrun, test_certifying_run_invokes_gate_exactly_once, test_metadata_only_verdict_refuses_to_call_tag_shippable) failed on the live tree until it moved. The package is still _viz, underscore first, and the underscore is now an API contract rather than a dependency one: nothing else in lucid_yolo imports it, nothing outside it imports matplotlib, and the notebooks are its only callers, so its signatures can move with them. tests/viz/test_import_surface.py pins that from two sides. The dynamic side imports the shipped surface — lucid_yolo, lucid_yolo.predict, lucid_yolo.eval.checkpoint, lucid_yolo.data — in a fresh interpreter and asserts lucid_yolo._viz never entered sys.modules; the static side reads every shipped module's source and refuses an import line naming matplotlib or _viz outside the _viz subtree, which is the check that catches a function-local import no import of the surface executes. Neither asserts matplotlib absent from sys.modules after the import: torchmetrics loads it (torchmetrics/utilities/plot.py), and that is theirs.

No backend is selected in the package. The plan had _viz keep the scripts' matplotlib.use("Agg") at module top. It does not: use("Agg") is a process-wide setting, and a notebook importing a package that makes it is switched away from its inline display — figures returned from a cell would stop rendering. So the backend selection stays in the two scripts and the two test conftests, which are the headless callers, and the package imports pyplot unconditionally with no use() before it. For the same reason build_run_figure, plot_curves, predict_and_draw and show_predictions return their figure open: the script saves and closes, a notebook displays, and closing inside the package would serve the first by breaking the second. The svg.hashsalt moved with the backend line, since it is the committed figure's concern and the same kind of global.

WP-192 — the complete oriented evaluator uses the worker policy it already had

Timing evidence is host-specific and non-gated. The off-repository procedures were profiling the public scoring loop, comparing five-run medians with channels_last, traversing the loader, and calling public run; the warm-up and repetition procedure is recorded below where available. The benchmark harness and raw output are not committed.

The standalone path predates the image-size target and inherited-validation guard. The evaluator's explicit num_workers=0 arrived in 0fe08f4 on 12 August. The free-/dev/shm cap on automatic workers was already present from 9f1fa80 on 2 August; the image-size validation worker target (9d0fa88) and inherited-validation guard (d09ded9) arrived on 13 August, one day after the call site. 00af9e4 on 6 September renamed _val_worker_cap to _val_worker_target and clarified its documentation without changing the policy. Nothing revisited the call site, so automatic worker selection could only govern datamodules whose callers left worker selection open. Through the inherited-validation guard, the evaluator's explicit zero was raised to one validation worker wherever /dev/shm exists. All measured "existing route" baselines below, including 0.850 s, 9.757 s and 13.104 s, therefore used one validation worker; the current low-startup route runs zero workers and is unmeasured. A profile of the public scoring loop over 117 real DOTA tiles spent 3.60 of 5.10 s waiting for the loader. The rotated accumulator took 0.051 s, so even removing it entirely could save only 1.0% of that run. Converting the standalone module to channels_last, the other live candidate, moved the five-run median from 2.679 to 2.645 s and overlapped both ranges; it was rejected.

Completeness is not itself the crossover. Starting eight workers to score only eight tiles took a 1.784 s median against 0.850 s on the existing route. A positive --limit therefore keeps that route unconditionally — the measurement is the smallest size the diagnostic path runs at, not a sweep, so a large positive --limit keeps the same route by scope rather than by a remeasured crossover. With no limit, the evaluator asks the datamodule for its existing policy, then keeps it only when the loader has more batches than workers: otherwise no process can be reused after its first batch. That structural guard covers the measured one-batch regression and a complete but tiny custom split without introducing a dataset-size constant. No crossover measurement was taken on macOS / MPS, where workers spawn rather than fork, and the band of batch counts just above the worker count is also unmeasured. Nothing in the reader, letterbox, model, decoder, merge or metric changes. Tests walk run through both public branches and the tiny-complete fallback, while a separate real-pool case reads the same twelve tiles through one and two workers and compares images, boxes, labels, rotated boxes, difficult flags, order and the complete report.

The loader result was repeated on an isolated CPU. Five measured traversals after one warm-up on an n2-standard-8 Ice Lake VM (Python 3.12.3, torch 2.14.1) moved from a 9.757 s median (9.651–9.987 s) with one validation worker to 2.658 s (2.609–2.781 s) with the policy's eight, a 72.96% median reduction. Every traversal yielded the same 117 tiles, 2,691 instances and tensor checksum. That figure is the loader alone: CPU-device inference was not measured. The policy's worker count derives from os.cpu_count(), which is blind to container CPU quotas and process affinity, so a host that limits either may start more workers than it can run. A quota- or affinity-aware count would change the datamodule policy that training shares and belongs in its own work package, not in WP-192.

The complete pipeline keeps the gain. Five measured public run calls after one warm-up on an NVIDIA L4 (driver 580.178.04, torch 2.14.1+cu130, batch 8) included loading, model inference, rotated decoding and scoring, whole-image merge and scoring, and report construction. The existing route took a 13.104 s median (12.909–13.253 s); the policy route took 7.830 s (7.336–8.090 s), a 39.66% median paired reduction. Each pair used the same freshly initialised oriented nano model over the same official-DOTA-derived 117-tile slice; metrics and report payloads were equal in these runs after removing the recorded seconds and the intentionally different worker-count diagnostic. The in-repository, gated CPU evidence of report equivalence is tests/eval/test_eval_obb.py::test_run_through_real_workers_matches_run_through_low_startup_route, which compares a real worker pool with the zero-worker route after removing those same fields; it does not establish CUDA determinism.

The report now says which count ran. A complete run's worker count is no longer the fixed 0 the report used to imply; it is whatever the datamodule's shared-memory- and image-size-aware policy picked on the host that ran it. SplitScoring and the JSON report's info.workers field now carry that count, read off the same loader score_split already builds, so a report is self-describing about which route scored it without reaching into the datamodule's private state. A failure of the policy route's worker pool itself, such as a worker that dies mid-pass and surfaces as the loader's RuntimeError, is not one of the pairing ValueErrors run turns into a FAIL: line: it reaches the terminal as a traceback, lucid-eval exits with status 1, and no report is written.

That field is an additive schema change, and its consumers were checked. A repository-wide search over tests/, notebooks/, docs/ and src/ found no oriented-report golden and no consumer comparing the whole payload or the exact info key set. The tests and the other task reports read named keys, while demo_obb.py and the documentation only produce or name obb_report.json. The regression therefore pins the new field directly; downstream code that selects existing keys is unchanged, while a consumer enforcing an undocumented closed schema would need to accept the additive key.

The floor moved to its exact live count, not a one-row nudge. _WP_FLOOR only tracks additions correctly when it is raised to the roadmap's exact live count — the same ratchet rule WP-152b and WP-165b record, and the one _ASSUMPTION_FLOOR was caught violating at 47 rows of slack. This row found _WP_FLOOR at 177 against a live count already at 191 before it, 14 rows of slack that would have let most of Phase 17 be deleted without the ratchet firing; it is now 192, the exact count after this row, leaving zero slack for the next one to inherit.

WP-192b — decode once, augment many times

The input stalls, not the compiled loss, were the complete-path bottleneck. Profiling twelve real oriented DOTA samples at 1024 px attributed 0.791 of 1.233 CPU seconds to torchvision.io.decode_image; the four-source mosaic makes that repeated work the dominant single stage. Compiling the oriented objective was rejected after the five-repeat steady-state result proved noisy: its paired median-step reduction was 43.39%, but the pairs ranged from 26.68% to 46.08%, the baseline median had 31.08% relative spread, and the repeat-median mean start-to-start interval worsened 24.68%. The intended repeated Trainer.fit wall-time run yielded no admissible measured pair after its discarded warm-up because SSH failed. The selected change acts on the stage the complete path was actually waiting for, and earns its claim from the complete repeated fit below.

The cache is raw and opt-in. Both readers can eagerly retain one RGB uint8 tensor per distinct training file. The entry sits before mosaic and every random transform, and _load_image still allocates the float image each draw, so the cache removes decoding without freezing augmentation or handing mutable cache storage to a caller. DetectionDataModule.cache_train_images routes the switch to the training reader only; validation remains uncached. D27 keeps the default off because the memory bill is the uncompressed split, and limits the copy-on-write claim to forked workers.

Public-data values are unchanged. Two seeded pipelines, one cached and one uncached, produced bit-identical float-image int32 views and exactly equal target fields over 100 real DOTA draws carrying 7,083 instances. The slice held 41 distinct decoded tensors using 125,370,774 bytes.

The isolated CPU path establishes the stage ceiling. On an archbench C3 VM (8 vCPU, Intel Xeon Platinum 8481C, torch 2.9.1+cpu), five measured 1,640-draw traversals after one warm-up included dataset construction and returned the same checksum in both arms. The uncached median was 52.713 s (52.520–53.127 s); cached was 9.440 s (7.936–9.494 s), with a paired median reduction of 82.14% (81.99–84.95%). The VM was deleted and an explicit instance query returned no row. This is the reader-stage bound, not the training headline.

The training headline clears the complete-path bar on top of WP-193. WP-193 is merged (e8d632c); both arms of the L4 comparison ran on that post-backward head so adjacent gains are not double-counted. Five measured 257-step Trainer.fit pairs after one discarded warm-up, oriented nano at batch 16 / 1024 px / BF16 with six workers and EMA, moved from a 436.922 s median (435.429–441.999 s) to 332.337 s (332.170–343.212 s): 23.94% from arm medians. Every pair improved by 21.97–23.97%, with a 23.68% paired median. The ordinary step median was actually 2.42% slower; the interval mean fell because the multi-second loader stalls disappeared, which is the measured mechanism rather than a GPU-kernel claim. First input and first loss matched in every pair, and peak CUDA allocation was exactly 11,174,604,800 bytes in both arms. The isolated g2-standard-8 used an NVIDIA L4, driver 580.178.04, torch 2.9.1+cu129 and Python 3.12.3; archbench deleted it and both the helper cleanup and explicit inventory found no live instance.

The headline is a small-slice, decode-bound-host result and does not transfer unchanged to a full-size split. The L4 fit slice was the same 41 distinct tiles (125,370,774 bytes of cache) the parity paragraph cites, expanded to 4,100 records under fresh IDs per the PR description, so the cache build cost about 41 decodes rather than 4,100; the build sits inside the measured wall time and was nearly free. The host was an 8 vCPU g2-standard-8 with six workers, the regime where the measured mechanism, the loader stalls that disappeared, applies. A real split costs about one decode per distinct file to build, and its RAM bill follows sum(3*H*W) over those distinct files (TRAINING.md) rather than the 125 MB measured here; neither that build time nor that memory was measured on a full-size split, so the 23.68% and 23.94% figures say nothing about either.

Why the row is lettered 192b. It was lettered from WP-192 so that the then-open WP-193 post-backward contribution kept its identifier; WP-193 has since merged.

WP-192c — prepared source pixels and copy-paste masks

[THEORETICAL] Reuse immutable source work before random augmentation. cache_prepared_train_images is an opt-in float32 snapshot of the training reader's images and targets. Source conversion and subtraction of the existing affine fill happen once. Mosaic copies those centred values onto zero fill; the same resample restores the original fill afterwards. Preparation centres out of place, so the reader's tensor is never written and nothing depends on whether a reader hands out fresh or shared storage; the cost is one transient image per record during setup. The [MEASURED] timings below were taken at 672f0ba, which centred in place; the out-of-place subtraction adds only that setup-time allocation and was not re-timed. No second byte cache is retained. Fit setup reuses an existing snapshot; other stages release the datamodule's references, and failed preparation installs no partial cache. YOLO labels are snapshotted with the images.

[THEORETICAL] One source coordinate needs one copy. Copy-paste builds the union of its selected pixel masks and copies the immutable source once into the fresh destination clone. Overlapping writes already copied the same value. The instance targets remain separate and in their original order; the rasterisation, geometry and RNG draws are unchanged. Unlike the opt-in cache, the union changes the default (non-opt-in) copy-paste path; its output is byte-identical to the previous path, as TestCopyPasteUnion checks. Its inclusion in this WP awaits principal acceptance, because the screens do not carry it alone: cache-only reached 5.61% and union-only 4.32%. These changes follow from this repository's source, R21's existing fill interface and measured producer work.

[MEASURED] The complete short fit is the performance claim. Baseline d6f7ec8; candidate is d6f7ec8 plus the 672f0ba change. Five alternating pairs on an isolated NVIDIA L4, driver 580.178.04, torch 2.9.1+cu129 and Python 3.11.17 used detection n, batch 64, 640 px, BF16, four workers and decoded caching in both arms, with device_hsv=True. Each fresh-process fit covered five epochs and 80 steps over 1,024 distinct COCO val2017 training images and 32 held-out images. Cache construction, augmentation, forward/backward, optimiser, EMA, validation each epoch, checkpoints and final CUDA completion are inside the timer. Mosaic stayed enabled (close_mosaic=0), representing the augmentation-active part of training. Baseline median 116.184 s (114.011–116.872), candidate 97.579 s (96.728–98.340); paired reduction median 15.63% (13.93–16.75%).

[MEASURED] Correctness is byte-level. The five pairs and baseline-against-itself controls match model, optimiser, EMA, scheduler, loss and metric hashes. A separate 64-draw public-COCO check matches float-image bytes before quantisation, packed targets and RNG state with mosaic enabled and disabled, and preserves the source snapshots. Focused tests cover both readers, overlap/duplicate selections, non-contiguous images, source reuse, rollback, repeated setup and spawned workers. Direct affine checks cover exposed padding and identity parameters on fixed letterbox canvases, the training route, with byte parity. A standalone identity without letterbox can bypass the dependency resample and retain float32 centring round-off (observed maximum 5.96e-8 on RGB codes); a regression bounds its absolute error by float32 epsilon with zero relative tolerance. The declared torch 2.4 CPU floor passes the 148-test focused suite.

[MEASURED] The memory cost and scope stay explicit. This slice retains 3,375,379,188 float-image bytes, against 843,844,797 decoded bytes. Prepared images are per training record, including duplicates, plus target metadata; the option stays off by default. D27 governs the decoded uint8 cache only; the float32 shared-tensor cache is a different memory trade-off and awaits a principal decision. This is a small-slice, CPU-limited training result, not a claim for arbitrary worker counts, model sizes or full-size splits. Separate single-pair cache-only (eight epochs) and copy-paste-only (six epochs) fit screens reached 5.61% and 4.32%, respectively, so neither became a standalone claim above the investigation's eight-percent bar. A separate eight-worker, four-epoch single pair improved 4.48%, below that bar. Full make gate and accelerator goldens were not run; the row remains open.

WP-193 — post-backward state updates use fewer CUDA launches

The measured target is the complete interval, not either helper. A public COCO val2017 training harness measured from one completed batch to the next, including loader wait, host-to-device transfer, forward and loss, backward, clipping, MuSGD, EMA and synchronization. On an isolated g2-standard-8 NVIDIA L4 (driver 580.178.04, torch 2.9.1+cu129), variant n, batch 4, 640 px and BF16, five warm ABBA pairs moved from a 256.8 ms baseline median to 222.5 ms with this row. The paired reductions were 13.28%, 13.13%, 14.90%, 12.94% and 6.38%: median 13.13%, range 6.38–14.90%. The fifth candidate arm was the run's one timing outlier at 246.4 ms, but remained faster than the fastest baseline arm at 254.9 ms. Each arm discarded 15 warm-up intervals and measured the next 44. Peak allocated memory was 822.1 MiB in both arms. The paid VM was deleted after the run and an independent cleanup found no archbench instance alive.

Two launch-bound stages make one post-backward change. The existing optimizer already groups exact-shape CUDA matrices, but the torch.vmap layer around the scalar Newton--Schulz expression remains; it already lowered each product to one batched multiply, so the change drops that layer and folds the scale-and-add into the product epilogue. _orthogonalize_batched uses baddbmm for the same b*A + c*A@A and a*X + polynomial@X expressions, preserving each slice's normalization, coefficient triple, requested step count, gains, scaling and decay. EMA previously launched one multiply and one add for every floating parameter or buffer; grouping by (device, dtype) and calling torch._foreach_lerp_ once per bucket evaluates the same shadow + (1 - decay) * (live - shadow) recurrence. Singletons and non-CUDA optimizer paths remain scalar. Both transformations are scheduling changes derived from this tree's buckets and profiler counts; no training constant or imported implementation enters the diff.

Floating-point schedules are equivalent, not bit-identical. Native batching and interpolation reassociate operations, so the regression contract is a bounded trajectory against the prior scalar formulas rather than byte identity. The repository's multi-step trajectory tests run on CPU, over three (6, 4) float32 matrices for the optimizer and float32 parameters for the EMA. The 40-step fixed-gradient CUDA run on an L4 is a one-off measurement, not a repository test, and no multi-step trajectory test covers square, wide or tall shapes or a float64 bucket. A finite one-step public-COCO run starts from the same loss and checks model and momentum differences directly. This row does not claim an unchanged long-run stochastic trajectory: tiny per-step differences can change later discrete assignments, as the repository already records for the exact-shape batching it replaces.

The rejected queue was judged end to end. MuSGD's native batch alone reached only 7.48% median on its final five-pair L4 rerun and was rejected despite a larger exploratory result. EMA foreach was retained only after the combined five-pair run; no isolated repeat-level EMA claim is made. Five segmentation mask-loss compilation repeats had a 4.44% median but ranged from -10.88% to 6.05%, crossing zero. Packed host metadata was rejected from its structural stage bound, while loss logging and scaled-dot-product attention had only inconclusive single local A/B screens; none carries a quantified claim here. cuDNN benchmarking was slower in its exploratory complete-path run and also carries no repeat-level claim. None became a dossier. Combining the two adjacent post-backward launch reductions is the first candidate whose complete-pipeline median clears the requested 8% threshold.

The accelerator golden check is outstanding, not run clean. _orthogonalize_batched reassociates the same CUDA Newton--Schulz buckets the WP-177 entry (ESCALATION.md, 2026-09-15) already measured moving goldens/gpu/overfit_micro_kp.json's train_oks_ap 0.117 outside its stored band from float-scheduling alone, on a question that entry leaves unresolved. This row's own correctness evidence is the 40-step fixed-gradient CUDA trajectory and the one-step public-COCO check (both bounded-agreement, not golden-harness), plus the offline make gate (50/50, CPU-only). make gate-gpu / check_goldens.py --include-gpu was not run against this diff, so whether the additional reassociation here moves overfit_micro_kp or overfit_micro_obb further is unmeasured — the roadmap row stays 🔄 until it is.

WP-193b — compatible matrix batches and momentum dispatch

Compatible matrix batches and momentum dispatch. One diagnostic session after the native-batch and image-cache changes attributed a 49.28 ms median to the optimizer within a 159.60 ms complete detection interval on an isolated L4. Its 10.90 ms momentum-dispatch median suggested a 6.8% stage share; this is an exploratory opportunity estimate, not an independently repeated speedup or a universal ceiling. A single bounded-padding-only complete-path screen suggested 5.94%, without a run-level range; it did not establish a standalone improvement above the investigation's 8% bar. The combined candidate targets the remaining launch overhead within the same optimizer phase.

Zero extension changes the batch, not the quintic. For a wide matrix X, appending zero columns preserves its Frobenius norm and X @ X.T; every iterate of the existing polynomial therefore keeps those columns zero. CUDA matrices sharing the smaller dimension can run in one native batch, then crop and transpose back before their original dimension-dependent scale, gains and decay. Each padded gradient stack is bounded by the parameter and gradient bytes of the matrices in its own batch. A singleton, or a batch whose matrices all share one shape, keeps the exact-shape update; equal-shape matrices that share a smaller dimension with a matrix of another width join the padded batch. Parameters whose memory ranges overlap keep exact-shape keys, whichever storage object wraps them. No coefficient, iteration count or normalization rule changes.

Momentum keeps its rounded operations. Three foreach operations replace the CUDA per-parameter multiply, add and Nesterov add, with the original operation order. Device and dtype buckets keep independent state. Shared parameter/gradient/buffer storage and unusual restored state use the original scalar method; CPU and MPS keep their previous implementation. No public flag or checkpoint field is added.

The numerical contract is bounded agreement. Mixed-width GEMMs and reductions may round differently. A real one-step COCO run and its baseline-against-itself control have the same loss bytes, all 369 momentum buffers match bit for bit, and the largest model/buffer difference is 2.98e-8. A 60-step zero-warmup screen diverges in its BF16 trajectory, so this is not a claim of identical stochastic training or final accuracy. The mixed-shape scalar-oracle, storage-budget, alias, DLPack-overlap, dtype, checkpoint-resume and foreach recurrence tests cover the altered scheduling. The full accelerator goldens and long-run accuracy remain unverified; the existing WP-193 status stays open.

The padding region is measured, not universal. A five-repeat optimizer-only crossover screen found the unguarded x scale slower: 111.26 to 119.37 ms per step on the L4. Padding is therefore restricted to a smaller dimension of at most 256, the largest present in the measured nano workload; larger matrices retain exact-shape batches. The guarded screen improved the optimizer in all five scales, but those stage numbers are not the training claim. Parameter subclasses may legitimately refuse storage inspection: a baseline-compatible vector subclass exposed that regression before handoff. Such groups now keep their original keys and scalar momentum, including mixed groups where opaque storage may alias an ordinary parameter.

The final claim includes the complete short fit. Five alternating paired Trainer.fit runs on an isolated NVIDIA L4 (driver 580.178.04, torch 2.9.1+cu129, Python 3.11.17), after two discarded warm-up fits, used detection n, batch 4, 640 px, BF16 and four workers over 256 real COCO val2017 images. Each six-epoch / 384-step fit included cache setup, augmentation, forward/backward, the default three-epoch LR warmup and decay, alpha scheduling, EMA, mosaic closure, four validation batches each epoch and checkpoints. Baseline median 89.448 s (88.765–89.646); candidate 79.183 s (77.799–79.660). The paired reduction median is 11.19% (10.88–13.22%), with the same 809.9 MiB peak allocated. Timing brackets are synchronized and contain no profiler. This is a complete short fit, not a full-COCO or final-accuracy measurement. The author reports that the code measured here passed 65 focused tests on the L4 with two newer-runtime skips, and that those cases passed separately on CUDA. The pytest selection and the revision of that run were not recorded, so the count is not reproducible from this tree; the two optimizer test files collected 71 tests at the reviewed head b060de4. Full make gate and accelerator goldens were not run.

Why the row is lettered 193b. WP-193 had already merged (e8d632c) when this change arrived, and its row stays open on the accelerator-golden check, so the follow-up carries its own row, scope and gate instead of extending the merged one. The commit trailer check accepts digits only, so the commits keep the WP: 193 trailer, as the commits of WP-192b keep WP: 192.

WP-193c — replay the network, keep the dynamic objective eager

The source is the current training loop and R43's general-purpose API. This follow-up was profiled from 01b4d4c after compatible MuSGD batching merged, then rebased and remeasured on 50c2092 after the augmentation contribution merged. It changes how existing network work is dispatched, without rewriting any model, assignment, loss or optimizer arithmetic. capture_network=False is the default. Unlike the Inductor objective option, the recording contains only the detection network's forward and backward work; targets, loss, schedules, optimizer and EMA remain eager. Unlike the orthogonalization replay (WP-197b, already on main), it records no optimizer work.

One fit owns one recording. A private tensor-tuple view composes the same backbone, neck and head. PyTorch supplies the autograd bridge. Recording restores RNG and all buffers after warmup, and mixed precision disables the autocast cache. A partial batch stays eager. Changed storage, modes, hooks or precision retire the recording. Public forward, validation and prediction remain ordinary calls. The holder registers no parameters and serializes no CUDA state; fit completion and exceptions clear it. CUDA single-device automatic optimisation and detection are the measured scope; higher-order differentiation and compile_step are excluded.

BF16 has a measured runtime boundary. FP32 and FP16 replay pass the exact eager comparisons on torch 2.4.1. BF16 does not, so BF16 replay starts at the verified 2.9 runtime and older BF16 fits remain eager. This avoids choosing a tolerance to hide the failed exact oracle; it does not claim that every intervening release was tested.

Accumulation retains its own gradients. The CUDA recorder reuses backward gradient storage. If a later backward must add to live gradients, the holder first clones those values out of that storage. This also covers custom optimiser or zero-grad hooks. An analytic scalar probe and independent eager-network regressions expose the unprotected alias: two microbatches can otherwise produce 6 where the derivative is 4. Ordinary one-step optimisation keeps the uncopied route.

The claim is the complete short fit. Five alternating paired, fresh-process Trainer.fit runs after two discarded warm-up fits used detection n (depth 0.50, width 0.25, max channels 1024), batch 4, 640 px, BF16 mixed precision, four workers, persistent workers and decoded-image caching on 256 real COCO val2017 images. Each six-epoch / 384-step fit included cache setup, augmentation, network and loss, backward, MuSGD, LR warmup/decay, alpha scheduling, EMA, eight validation batches per epoch and checkpoints. The validation images are a 32-image subset of the training subset: this is throughput and trajectory evidence, never held-out accuracy. Persistent workers retain their augmentation state, so the mosaic-close callback is not evidence that worker-side mosaic actually closed.

On an isolated NVIDIA L4, driver 580.178.04, torch 2.9.1+cu129, Lightning 2.6.6, Python 3.12.3, synchronized fit time fell from median 92.589 s (91.614-93.691) to 77.330 s (76.234-78.265): 16.48% by the ratio of medians. Recording setup is inside each fit; imports and model construction are outside both arms. Baseline-against-itself and every paired arm produced identical SHA256 digests over the raw tensor bytes of losses, model, optimizer, scheduler, EMA and metrics. This certifies those deterministic short trajectories, not all precisions, devices or long-run accuracy.

Memory has two directions. Median peak allocated memory fell from 828.5 MiB (822.0-831.4) to 711.3 MiB (711.3-711.3), while median peak reserved memory rose from 1022.0 MiB (1022.0-1038.0) to 1260.0 MiB (1260.0-1262.0). The graph pool is a cost of the opt-in path; no avoided-OOM or batch-capacity improvement is claimed. Fit callbacks drop the holder's recording, rather than promising that every outstanding autograd reference or allocator reservation vanishes immediately.

The speed claim is limited to this small-batch workload. Batch-32 screens changed direction relative to the batch-4 result, so the gain above is not a general claim; the shipped detection recipes train at batch 16, which was not measured here.

Validation remains explicitly bounded. Full make gate, accelerator goldens and final accuracy have not been run; this row stays open on those checks. Focused regression tests and timing logs are retained with the contribution's review evidence.

WP-193d — replay shared pose features, preserve native head supervision

Replay the shared backbone and neck; keep the pose head native. The source is the current backbone/neck/head composition and R43's forward/backward recording API. Detection keeps its original recording. Pose records only backbone and neck, then runs its complete dual head and current objectives natively. A batch with no assigned positives leaves the native point stems' gradients absent. Recording those stems too materializes zero gradients, advancing optimizer state the native loop skips; the synthetic empty-batch regression (tests/ptl/test_pose_network_capture.py::test_pose_trainer_matches_native, layout="empty") therefore rejects that larger recording even though an ordinary public-data fit matched it. No model, assignment, loss or optimizer arithmetic changes.

One fit owns one signature. Pose uses the existing --model.capture_network true; its default remains false. Single-device CUDA automatic optimisation is required. Pose with gradient accumulation stays native: the shared recorder's accumulation checks diverge on the unchanged baseline. Frozen features, modified head topology or forwards, head hooks and compilation also stay native. Returning no features on an unsupported call lets the owner make its complete native call, preserving root and global hook observability. The recording core and detection tensor adapter retain their original execution; only the recorded stage factory and output adapter vary. Buffer/RNG rollback, precision/storage checks, first-order-only autograd and fit cleanup remain the existing contract.

The owner was measured before the patch. Four instrumented native training steps took about 1.541 s; backbone and neck forward calls accounted for 176.352 ms, or 11.45%, before their backward work. The candidate trace has no corresponding Python forward calls in steady replay and keeps the native head. Those instrumented shares diagnose launch overhead; they are not the end-to-end speed claim.

Five complete, alternating pairs. Fresh processes trained the registered nano pose model on 384 original COCO val2017 images with annotated persons and 32 disjoint validation images, five epochs, batch 4, 640 px, BF16, seed 123. The isolated g2-standard-8 used NVIDIA L4, driver 580.178.04, torch 2.9.1+cu129 and Lightning 2.6.6. Both arms enabled prepared-image caching, device HSV, epoch prefetch, orthogonalization replay and optimizer-update replay. Complete fit time includes model/trainer construction, cache/recording setup, augmentation, forward/backward, clipping, optimizer, EMA, validation and checkpoints. The cohort starts on a newly provisioned VM after focused tests; first-use fit costs remain in the measurements and process order alternates. No profiler ran in the final pairs.

Metric	Native median (range)	Feature replay median (range)
Complete fit, s	118.243 (117.791–121.846)	105.033 (104.321–105.394)
Whole process, s	123.826 (123.404–127.564)	110.666 (109.886–111.013)
Post-warmup CUDA step, ms	194.433 (193.751–196.155)	165.304 (164.386–166.928)
Peak allocated, MiB	913.041 (913.041–913.041)	687.645 (687.645–687.645)
Peak reserved, MiB	1178.000 (1178.000–1178.000)	1392.000 (1392.000–1394.000)

Complete-fit time falls 11.17%, with paired reductions 10.68–14.27%. In these five pairs, all model, optimizer, scheduler, EMA and per-step-loss byte hashes match; validation metrics agree. The native baseline also reproduces itself. CUDA events exclude the first 64 steps per run; the complete fit uses device synchronisation at both timing boundaries. The one-pair batch-16 screen changes fit time by 2.50% and preserves those same states; it is exploratory, not a repeated headline. These complete-fit measurement runs, made outside the test suite, are the only source for the scheduler, EMA, per-step-loss and validation parity: the in-repo pose regression (tests/ptl/test_pose_network_capture.py::test_pose_trainer_matches_native) compares only model, optimizer and scaler state after one epoch, with no validation, EMA or resume.

Compatibility and remaining evidence. All 27 focused cases pass on both torch 2.9.1 and torch 2.4.0, in one command per runtime with explicit UTF-8 configuration reading. BF16 on torch 2.4 stays native. The 17 CUDA cases carry the GPU marker selected by the accelerator gate; the head certificate also rejects instance and class overrides of forward_branch. Coverage includes empty and nonempty batches, FP32/FP16/BF16, clipped accumulation fallback, CLI construction, copied head certificates, branch-dispatch overrides, frozen features and head/global hooks. The branch-override regression fails with the previous guard reinstated in memory. Full pre-commit and strict mypy pass. Existing CUDA accumulation failures reproduce on the unmodified baseline and are outside this scheduling change; their test ids and runtime are not recorded here, and the decisions they raise are open in docs/ESCALATION.md (2026-10-20). Full make gate, accelerator goldens, other accelerators and long-run acceptance accuracy remain unverified; this package stays open.

WP-193e — preserve prediction gradient boundaries while replaying their work

A CUDA training trace on an L4 attributes 197.112 ms of four instrumented steps' host dispatch to the detection head's forward calls and 177.742 ms to its matching backward nodes, identified by their forward autograd sequence numbers. Together they occupy 28.40% of that 1.320 s host interval, before the separate prototype and semantic predictions. This diagnostic budgets submission work; complete-fit performance is measured separately below.

The opt-in capture_segment_heads records existing prediction calls through R43's graphed-callable API. Box/class stems retain independent input-gradient contributions. Each branch's coefficient routine, ProtoNet and the semantic predictor keep separate backward nodes, so an unused mask prediction leaves its parameters' gradients absent. The native per-level order, flattening, concatenations, tanh and prediction expressions remain unchanged. Backbone, neck, prototype fusion, assignment, losses, optimizer, schedules and EMA retain their existing execution.

Combining each branch's box/class predictions failed an exact state comparison, so the implementation retains separate stem contributions. FP32 replay also failed the semantic-bias gradient byte oracle. Replay is verified only for BF16 mixed precision on PyTorch 2.9 or newer; FP32 and FP16 retain native calls.

One fresh topology traversal feeds the guard checks. An attribute certificate retains bindings and checks mutable registry names and sequence elements, while current tensor storage, hooks, modes, precision and class dispatch remain independently checked. Changed or custom prediction stages, frozen prediction parameters, aliasing, logical negative/conjugate tensor views and unsupported runtime conditions use the native route. The native fusion is included in this guard without being recorded. Each prediction group retains one recorded input signature; a partial batch stays native. Transient holders keep original checkpoint keys and discard recordings on copying and at fit completion or failure.

Five alternating fresh-process pairs on an isolated NVIDIA L4 g2-standard-8, against main e4db46d after detection-loss and pose-feature replay landed, reduced complete nano segmentation fit time by 16.60%. Native/replay medians (ranges), seconds:

Host	Native	Replay	Time reduction
g2-standard-8	202.794 (201.536–204.709)	169.130 (168.167–169.786)	16.60%

Each fit uses 384 training and 32 disjoint validation COCO val2017 images with the original 80-class annotations, 10 epochs, batch 4, 640 px and BF16 mixed precision. Both arms enable prepared caching, device HSV, epoch prefetch, orthogonalization replay and update replay. The timer includes construction, cache preparation, graph recording, training, EMA, validation and checkpoints. Driver 580.178.04, torch 2.9.1+cu129, Lightning 2.6.6. Whole-process median time, including imports and verification, falls 16.16%.

Per-step-loss, model, optimizer, scheduler and EMA byte hashes and validation metrics match in the five pairs. A separate untimed ten-epoch pair matches eight dense prediction tensors at steps 0, 480 and 959 and matches the timed final state hashes. Runtime source SHA256: 1bc17340e8ede25574a4ad390e75790989dc58d9725f09d961b7cd115ab216f5. This page names neither the hashed artifact nor the hashing method, so a reader cannot reproduce the digest from it; until both are published, the digest records the author's measurement rather than checkable evidence. The two prediction-recorder test files report 134 passed on the L4 with torch 2.9.1 and 126 passed, 8 skipped on torch 2.4. Full pre-commit and the offline suite pass. Full make gate, accelerator goldens and long-run accuracy remain unverified.

Earlier cohorts are historical rather than measurements of this source: main 6eb6abb gave 16.42% on g2-standard-8 and 11.01% on g2-standard-4; the main 28549ba refresh gave 16.33% on g2-standard-8. Their original records remain in the earlier work-package revisions and detailed evidence; they are not a controlled comparison of CPU count or a current-source large-batch claim.

The speed claim is limited to the measured batch-4 workload. An earlier exploratory batch-16 L4 pair (640 px, BF16, three epochs / 72 steps) changed complete fit time from 40.413 s to 39.600 s, only 2.01%. Model, optimizer, scheduler, EMA and per-step-loss bytes and validation metrics matched. That screen used earlier source and a different harness revision; it is neither a repeated large-batch result nor a measurement of this current source. Larger-batch performance on the current source remains unverified.

The private prediction-runner seam in models/heads/detect.py changes dispatch around the existing expressions in their original order. Keeping independent box/class input-gradient additions requires recording the separate prediction calls: the earlier combined-branch screen failed the byte oracle. The default runner still invokes the native calls, and the recorder composes their adapters outside the head. This is a scheduling seam on a restricted surface, not a new method expression or prediction format.

The real automatic Trainer.fit regression uses MuSGD, EMA, valid instance masks and a validation epoch between two training epochs. It observes actual CUDA graph launches after recording and again after validation, instead of accepting populated entries as proof of replay. Forcing permanent native execution while retaining those entries makes this regression fail. The torch 2.4 route keeps its unsupported BF16 recorder native.

A _RecordedGroup's warmup shares the sibling network recorder's own hazard: a failed make_graphed_callables call can leave BatchNorm running-stat writes queued on its warmup stream, so the restore on the current stream must wait for a real synchronize first (test_network_capture_guards.py::TestFailedRecording::test_synchronizes_before_buffer_restore covers this for WP-193c). That case had no counterpart here; a new tests/ptl/test_segment_head_capture.py::test_recorded_group_synchronizes_before_buffer_restore injects the same failure into a real box stem's warmup on a local RTX 4060, asserts the device is synchronized before the BatchNorm buffer is restored, and confirms the failure still raises. Reverting the synchronize call in _RecordedGroup.__init__ makes it fail, so it is not vacuous. This is a local compatibility check, not a timed claim.

_SegmentHeadCapture.retire() shares _NetworkCapture.retire()'s disabled-guard contract too: only the first incompatibility of a fit warns, so a second cause in the same fit must stay silent (test_network_capture_guards.py::TestRetireWarning::test_later_cause_stays_silent covers this for WP-193c). A new tests/ptl/test_segment_head_capture.py::test_later_retire_cause_stays_silent calls retire() twice on a bare holder and confirms only the first call warns; reverting the if self.disabled: return guard makes it fail.

Two CPU-only adapter-call tests also distinguish active replay from a silent native fallback while retaining recordings. With an unchanged signature and no pending backward, the graphed callable must run; while a backward is pending, the native callable must run. The replay case fails under an always-native guard mutation.

Peak CUDA reserved memory medians are 6084→6474 MiB. Graph pools cost memory, and the result is specific to the measured small-batch workload. The implementation keeps first-order automatic optimization on one CUDA device without accumulation; unsupported modes remain native. Gradient accumulation stays native because this package validated replay only at accumulate_grad_batches=1, not because of a known defect in how the recorded backward protects live gradients.

Provenance: R43, this repository's native prediction calls and its own CUDA measurements; R12 supplies the public COCO data. No method arithmetic, coefficient, iteration count or method default changes. A15, A17 and A38 remain unchanged.

WP-193f — independent pose stem backwards preserve absent gradients

The small-batch CUDA profile still shows host-driven pose prediction launches after shared-feature replay. Record each of the existing eighteen box/class/point stems independently, using R43's graphed-callable API and the private prediction runner from WP-193e. The existing capture_network flag remains off by default. No model, assignment, loss or optimizer arithmetic changes.

Each stem has its own autograd node. If a batch's objective does not read its point output, no point backward runs: gradients remain absent and the optimizer skips the same parameters as native execution. A fixed joint head backward would instead materialize zero gradients. The point/uncertainty views and their concatenation retain their native order.

The shared recorder owns one bounded input signature per stem, including its input's requires_grad value. Differentiable feature samples retain that value when recorded. Existing buffer/RNG rollback, storage, hook, precision and pending-backward checks apply independently. The fit callback resets skipped-backward holds and clears the child holders on completion or failure. Deepcopy and pickle start with no recording. Replacing a head module retires the parent before feature replay, releasing obsolete pools. A stem's changed trainable set retires only its own recording; shared-feature freezing keeps its existing full-fit native fallback.

[MEASURED] Five alternating fresh-process pairs on an isolated g2-standard-8 (eight vCPUs, 32 GB), NVIDIA L4, driver 580.178.04, torch 2.9.1+cu129, Lightning 2.6.6 and Python 3.11.14. The workload is nano pose (one class, 17 points), batch 4, 640 px, BF16, five epochs/640 steps, seed 347, 512 original COCO val2017 training images and 15 disjoint person-annotated validation images. Training retains 226 images without person annotations. This is a mosaic-active training phase. Both arms enable prepared caching, fused device HSV, epoch prefetch, existing shared-feature replay, orthogonalization replay and optimizer-update replay.

The complete timer begins before torch import and ends after synchronized Trainer.fit return: construction, cache preparation, graph setup, augmentation, forward/backward, clipping, optimizer, schedules, EMA, validation and last-checkpoint writes are included. Diagnostic hashing and process teardown are excluded. OS/dependency caches are warm; each process rebuilds its prepared-image cache and CUDA recordings. No profiler, package installation or other training job runs during the cohort.

Metric	Existing feature replay, median (range)	Independent stem replay, median (range)
Complete runtime, s	127.302 (126.592–128.085)	110.795 (110.309–112.551)
Post-warmup start-to-start step, ms	159.569 (155.840–167.159)	136.184 (132.954–137.304)
Peak allocated, MiB	672.620	587.081
Peak reserved, MiB	1384.000	1730.000

Complete runtime falls 12.97% from the arm medians. Paired reductions are 11.59–13.50%. The model, optimizer, scheduler, EMA and complete per-step-loss byte hashes agree across the ten fits, as do validation metrics; the five baseline controls reproduce themselves. Sixteen complete start-to-start iterations are timed after ten warmup steps with CUDA synchronization at both boundaries. Memory peaks are identical within each arm. Reserved memory increases despite lower allocated peak; no OOM or capacity benefit is claimed.

Focused host/CUDA checks cover FP32, FP16 and BF16 values, input/parameter gradients, changing point supervision, real graph execution, native Trainer/scaler/optimizer behavior, temporary shape fallback, ownership replacement and changing stem trainability. A private mutation removing stem dispatch fails the regression rather than passing as two native runs.

A separate three-epoch batch-16 exploratory pair is 42.429 s before and 40.945 s after, 3.50% less time, with matching state hashes. It is one pair and does not clear this task's 8% floor. The claim is limited to the repeated batch 4 workload. Full-COCO throughput, other accelerators and long-run acceptance accuracy remain unverified. Stem replay ran on torch 2.9.1 only; the torch 2.4 FP32/FP16 checks named in TRAINING.md cover detection, oriented detection and shared-feature replay. Full make gate, aggregate offline coverage and accelerator goldens were not run; this package remains open.

WP-193g — replay common features without joining prediction gradients

The segmentation prediction recorder leaves backbone and neck native. An L4 Nsight Systems trace of eight training batches, with prediction and optimizer replay already enabled, attributes 151.230 ms of host dispatch to backbone and 127.376 ms to neck within 1,544.688 ms of batch ranges. These diagnostic ranges account for 18.04% before their backward work; they are not an end-to-end speed claim. The optimizer-step range encloses the forward/backward closure, so its total cannot be interpreted as optimizer-only cost.

The existing capture_network option now accepts segmentation and records these shared features through R43's existing graphed-callable adapter. All three feature levels already feed dense classification. Keeping coefficients, prototypes and semantic predictions outside that recording preserves their independent backward nodes and absent-gradient semantics. The option combines with capture_segment_heads, which records those predictions separately. Native model, assignment, loss, optimizer, schedule and EMA expressions are unchanged.

Single-device CUDA automatic optimisation and a fixed input signature remain required. The default stays off. Accumulation, frozen shared features, altered head topology/dispatch or registered head hooks, overridden or compiled segmentation forwards and the shared recorder's existing unsupported conditions retain native execution. Partial batches do not create additional recordings. Transient recordings are released at fit completion or failure and do not alter checkpoint keys.

Five alternating fresh-process pairs on an isolated NVIDIA L4 g2-standard-8 reduce complete nano segmentation fit time by 18.42% against main b293970, with segmentation prediction replay already enabled:

Seconds, median (range)	Native shared features	Replayed shared features
Complete fit	107.412 (106.764–109.816)	87.629 (87.043–88.462)
Whole process	112.721 (111.918–115.208)	92.937 (92.471–93.827)

Each fit uses 512 public COCO val2017 images for training and 16 disjoint images for validation, the original 80-class annotations, five epochs, batch 4, 640 px and BF16 mixed precision. Both arms enable prepared-image caching, device HSV, epoch prefetch with four persistent workers, segmentation prediction replay, orthogonalization replay and update preparation replay. Mosaic remains active throughout (close_mosaic=0); warmup is one epoch. The timer starts before model, data and trainer construction and includes cache preparation, graph recording, training, EMA, validation and last-checkpoint writes. Imports and output hashing are outside that fit timer; the whole-process row includes them and reduces by 17.55%. Runtime: Python 3.11.14, torch 2.9.1+cu129, Lightning 2.6.6, driver 580.178.04.

Model, optimizer, scheduler and EMA bytes, the 640 per-step loss bytes and validation metrics match within and across the ten fits. Losses, metrics and final model tensors are finite. Median whole-fit peak allocated memory is 1731 → 1511 MiB; peak reserved memory is 5496 → 5702 MiB. These are distinct allocator metrics, with extra reserved graph workspace rather than a capacity claim. The speed claim is limited to this batch-4 workload.

The SHA-256 of the measured UTF-8 bytes of src/lucid_yolo/ptl/module.py at the measured PR-head tree (8ebf10e) is 1800875a7c034e6567ff8848e87b7c96cc091004fc112e1b5734c0f2a1354306, computed with sha256sum src/lucid_yolo/ptl/module.py; later commits change module.py, so the hash does not describe later trees. The existing numerical stage expressions and all 41 restricted model/loss/assignment/optimizer/decode files are unchanged. Earlier measurements without the final head certificate are exploratory and do not supply this result.

A separate single batch-16 screen (three epochs over the same 512 training and 16 validation images) changed complete fit time from 45.828 to 44.498 s, only 2.90%. Model, optimizer, scheduler, EMA, per-step losses and metrics matched. This is an exploratory boundary case, not a repeated batch-16 performance claim; no batch-16 optimization is submitted on that number.

Focused checks pass 53 CPU-only cases with 43 GPU-marked cases deselected, 24 cases on torch 2.9.1 with CUDA available (including ten GPU-marked cases), and 27 cases including doctests on torch 2.4.0+cu124. Older BF16 replay retains native execution. The regressions exercise CPU routing, actual CUDA replay, every prediction and gradient byte, changing mask demand, empty supervision, partial batches, validation/EMA transitions, accumulation fallback and CLI construction. Removing replay deliberately fails the maintained route assertion. The full make gate, accelerator goldens and long-run accuracy were not run for this work and remain unverified; offline CI was green at the PR head 8ebf10e, which says nothing about the merged tree.

No new method constant or numerical assumption is introduced. The basis is this repository's native stage composition, our CUDA profiles and R43's registered runtime API.

WP-193h — omit only the snapshots discarded before backward

Network replay preserves live gradients at forward time because its recorded backward owns static gradient buffers (WP-208). That copy remains necessary under accumulation. R41's native automatic-optimization closure instead runs forward, zero-grad and backward in that order. With accumulation one and no zero-gradient observer, its copies become tensors that the following zero-grad discards.

The module now scopes the parameters in that real optimizer's live groups during the native closure's forward. The shared snapshot helper leaves those gradients in place. Zeroing, backward, callbacks and updates retain their original order and arithmetic. There is no new flag or checkpoint state. A context token resets on success or failure, and the original closure is restored in finally.

Only detection with active network replay, the native single-device strategy, native precision plugin and pass-through profiler, MuSGD step and zeroing, and the inspected Lightning 2.4–2.6 closure qualify. Direct forwards, accumulation, other tasks, custom objectives, model/strategy/precision hooks, zero-gradient callbacks, non-default or customized Lightning profilers, or custom optimizer wrappers and unknown runtime contracts keep the original snapshots. Optimizer parameters removed from a live group still receive a snapshot. Native optimizer profiler and LambdaLR wrappers qualify without accepting arbitrary functools.wraps wrappers.

No new method constant, numerical convention or assumption is introduced. The basis is the repository's own gradient ownership and CUDA profiles, R41's native closure ordering and R43's graph-owned gradient buffers. No external detection implementation was consulted.

Isolated L4 measurement

Five alternating fresh-process pairs per baseline, nano detection at batch 4, 640 px, BF16, 20 epochs, global/model seed 41, data-module seed 0 (its default), strict determinism (the CUDA CLI default) and cudnn.benchmark=False. The dataset is 384 public COCO val2017 training photographs and 32 disjoint validation photographs, with the original 80-class annotations (3,318/255 instances). Shared filesystem, driver and existing Triton caches are warm after the controls; each process recreates its image cache, model, Trainer and graph recordings. Each epoch consumes all training records and validates on all 32 validation records. EMA, gradient clipping at 10, mosaic, prepared-image cache, device HSV, epoch prefetch and the merged network/loss/orthogonalization/update replay options are enabled in both arms. Learning rate is 0.001, with three warmup epochs and four persistent data workers. The second baseline adds the norm replay/bookkeeping change at c5bc00ae6c23e364ac6348b505319e8a80f45a7d, an intermediate commit of the branch later squash-merged to main as ef020b2 (PR 28). The merged source differs from that commit, so re-measuring this baseline needs c5bc00ae itself.

Loggers, periodic Lightning checkpointing, progress bars and model summary are disabled; the final state checkpoint remains included.

Seconds below are median (minimum–maximum). The workflow clock includes CUDA-fenced model/data/Trainer construction, cache preparation, graph recording, every training and validation epoch, teardown and the final state checkpoint. The separate process clock also includes imports and CUDA context initialization.

Baseline	Original workflow	With this change	Time saved
Main 303a7e0	182.970 (182.441–184.103)	166.914 (166.530–167.857)	8.78%
Main plus norm replay/bookkeeping	164.251 (163.440–165.069)	149.493 (149.299–150.553)	8.99%

Whole-process times are 188.001 (187.487–189.245) → 172.121 (171.612–173.042) seconds on main, an 8.45% reduction; with norm replay/bookkeeping, 169.435 (168.550–170.203) → 154.508 (154.279–155.628), an 8.81% reduction. The last complete epoch averages 80.750 (79.773–83.055) → 72.227 (70.916–74.850) ms/step on main, and 70.810 (69.611–74.207) → 64.488 (63.508–64.920) ms/step on the second baseline. Allocated peaks after resetting at the second epoch are 781,228,544 → 771,284,480 bytes and 792,121,344 → 782,177,280 bytes, respectively, constant across five runs. Those peaks include later training and validation; no capacity benefit is claimed.

Runtime: NVIDIA L4, driver 580.178.04, CUDA 12.9, Python 3.11.18, torch 2.9.1+cu129, Lightning 2.6.6, g2-standard-8 with four CPU threads. One external NVML observer spans the matrix, outside each workflow/process clock. Raw commands, logs, per-step losses, runtime receipts and state fingerprints were recorded during the run but are not committed to the repository. The 1,920 losses per run and model, optimiser, EMA, scheduler and validation states match byte for byte across all runs within each baseline. Native controls also match each other. Scalar comparisons include IEEE-754 signed-zero bytes.

Profiling before implementation attributed 156.432 ms of 1,055.133 ms across six instrumented detection steps to the snapshot owner. Its 2,424 clones become 210, removing 369 per step. These instrumented durations explain the mechanism; the unprofiled complete workflows above provide the performance result.

Focused CUDA coverage passes 108 cases on the measured L4 runtime, including raw pre-clipping gradient bytes in FP32/BF16, and 35 new cases pass with Lightning 2.4.0 over torch 2.9.1. CPU/doctest checks pass 46 cases with three explicit skips; pre-commit and strict typing pass. The closure-shadow, profiler-observer and precision-dispatch regressions fail before their respective fixes and pass afterward.

This establishes a small-batch, replay-enabled detection result, not a universal training gain. One exploratory batch-16, three-epoch pair saved 4.86%; it changes both batch size and run length and does not isolate either effect. Larger datasets, other hardware, multi-GPU, MPS, torch 2.4, full make gate, aggregate coverage, accelerator goldens and long-run accuracy remain unverified. Other tasks and unrecognised closure contracts retain the original snapshots.

WP-194 — static checks leave pytest, test names lose their filler words

Pytest owns behaviour; hooks own file content. A scan of tests/ and scripts/_tests/ found four tests in tests/ that read a file as text and exercised no code, and fourteen in scripts/_tests/ that asserted a hook passes on the live tree. The second group repeated what lint.yml and make gate already run, since both call pre-commit run --all-files, so those tests are deleted rather than moved. The first group moved: the matplotlib and _viz import boundary under src/ is ruff TID251 banned-api with per-file ignores for _viz, scripts/, tests/ and notebooks/. A probe module holding two function-local imports showed the rule fires on both, which is the case the regex test existed for. The top-k decoder's no-suppression text pin and the two packaged-recipe pins (no deterministic:, task: outside det_*) became the content-pins hook, scripts/lint/audit_content_pins.py, with its functional core in scripts/_tests/test_audit_content_pins.py. The runtime sys.modules probes in tests/viz/test_import_surface.py stay in pytest because they import code. Two kept groups in test_check_commit_trailers.py also read the real register, TestPlaceholderRows and TestAssumptionRegister; they stay because they test how the parser reads that register. Removed alongside: the register-contiguity assertion, which docs-present already checks, and the docstring-path assertion on the trailer script.

Names drop a an the its their is are does that. 1057 test functions were renamed mechanically, and no two names collided in any file or class. The offline collection count is 2910 both before and after the rename. One name that read badly after the drop was renamed by hand: test_rejects_row_naming_class_absent_from_data_yaml. References in ROADMAP, ASSUMPTIONS and this log were rewritten to the new names. Gate: trailers in commit history still name the old ones, because history cannot be rewritten. This reverses the sentence-style naming that audit_flat_test_groups.py describes, at the principal's direction, and the flat-group hook stays green on the renamed tree. No guard hook stops filler words from returning, so it stays a convention.

WP-195 — rotated augmentation shares clipping and refitting dispatch

The profiler selected the whole operation. An isolated one-thread CPU pass over 32 public DOTA augmentation draws at 640 pixels spent 1.15 s of its 5.67 s input interval in rotated clipping and refitting, about 20.3%. Optimizing only the half-plane helper did not move the enclosing input path enough; grouping the clips but leaving each refit scalar also fell below the training target. This row batches both parts. At 1024 pixels the same stage was only 6.8% of the input interval, so that profile does not support a comparable claim there.

The batching follows the existing arithmetic. Each of the four half-plane passes groups rings by their current vertex count, evaluates the scalar clipper's existing pointwise expressions over the group, and splits the surviving vertices back in their original ring order. Refitting groups the surviving polygons the same way: vmap applies the unchanged _polygon_area and _fit_corners, and the existing polygons_to_rboxes converts the resulting quadrilaterals together. There is no polygon padding, vertex cap, replacement fit, changed threshold or imported geometric convention. Results scatter back to their original instance rows. The original scalar clipper remains the reference and the route for single clips, accelerator inputs, other dtypes, gradients and non-finite inputs.

The scope is CPU input work during oriented training. It changes no model, loss or optimizer arithmetic. The public entry point and return layout are unchanged. Tests compare raw storage bytes with the scalar route, exercise changing vertex counts and empty parts, retain non-contiguous input ownership and check the fallback conditions. Full make gate and accelerator goldens are not claimed for this contribution; validation is focused on the changed geometry and a complete CUDA training comparison.

The new test follows the ordinary staging path. The notebook dataset ignore is anchored to /data/, matching its documented root-directory scope. The unanchored data/ also ignored new files below tests/data/ and src/lucid_yolo/data/, so git add -A skipped them. The root notebook-download exclusion remains; already tracked files keep their behaviour.

The complete CPU input path confirms the mechanism. Five paired input epochs on an isolated n2-standard-8 (8 vCPUs, Intel Xeon at 2.80 GHz, torch 2.9.1+cpu, one thread) traversed all 117 distinct public DOTA tiles and 2,691 annotations, including image-cache construction. Median wall time moved from 22.758 s (21.629–23.423) to 20.870 s (20.783–21.168); paired median reduction 8.27%, range 3.51–9.91%. Clipping/refitting accounted for 17.98% of the baseline input interval and its median cost fell from 6362.72 to 2856.17 microseconds per real-data call. Input images and every packed-target field matched at byte precision in all pairs. The instrumented input epoch includes checksum work; the separate complete CUDA fit carries the training claim.

The training result is measured after the merged cache and state updates. Five counterbalanced complete 20-epoch fits on an isolated g2-standard-8 with an NVIDIA L4 (driver 580.178.04, torch 2.9.1+cu129, Python 3.12.3, Lightning 2.6.6) used nano OBB, 640 pixels, batch 16, BF16, six persistent workers, cached images, mosaic and EMA. The first 20 sorted official DOTA validation images supplied 117 distinct lossless tiles and 2691 annotations; they are training-throughput inputs, with no held-out accuracy claim. Every fit traversed 2340 augmented samples and 160 optimizer steps; validation, logging and checkpointing were disabled. Baseline median 165.056s (164.518–169.835) became 149.772s (146.751–152.531); paired median reduction 10.19%, range 8.68–12.25%. Timing synchronizes CUDA around the whole fit and includes cache construction, worker startup, data and augmentation, transfer, forward/loss/backward, optimizer, EMA and Trainer teardown. Final model, EMA, optimizer and scheduler storage hashes, first input and sampled losses match in every pair. Peak CUDA allocation is unchanged at 3,583,640,576 bytes. One discarded two-epoch warmup precedes the five fresh-process pairs. No comparable 1024-pixel, larger-scale, MPS, DDP or convergence benefit is claimed. The paid VMs were deleted and independently verified.

WP-197 — the input workers sample colour and CUDA applies it in a batch

An isolated CPU profile attributed 5.476 of 12.668 seconds to the colour transform over 128 public COCO augmentation draws. That selects an enclosing input stage rather than a small helper.

device_hsv is opt-in for CUDA with float32 or mixed precision. Workers keep the existing HSV draw before the flip draw and carry the stated gains with the flipped image. A pointwise colour transform commutes with that pixel permutation. The consumer batches the existing RGB/HSV expressions with vmap, applies the same gains, then quantizes and restores the consumer dtype at the existing boundary. Float32 images cost four times the default uint8 image IPC and transfer bytes; the worker cap already budgets float32. Masks, validation and the default route keep their existing behavior. Precision plugins that cast floating inputs before the data hook retain worker-side HSV.

CUDA's host-scalar division lowered the hue divisor to a reciprocal multiply and changed 72 of 314,572,800 real augmentation pixel codes by one. R40 explains that boundary. A device-tensor six keeps the CPU division convention; the captured pixel is a regression test, and changing the divisor back to a host scalar makes it fail. No gain, recipe, target geometry or model arithmetic changes.

Five paired complete six-epoch detection-n fits on 256 public COCO images (640 px, batch 8, BF16, four workers, cached decoding) reduced the median from 102.053 s (99.831–103.663) to 79.468 s (78.859–81.501). The median paired reduction was 22.22%, range 18.72–22.79%, on an isolated four-vCPU NVIDIA L4 host, driver 580.178.04 and torch 2.9.1+cu129. Loss, model, optimizer, EMA, scheduler and metric byte hashes matched in each pair and across repeated baselines. Peak allocated CUDA memory was 1,595,021,312 versus 1,596,232,192 bytes.

With the batched MuSGD change, merged as 01b4d4c, applied to both arms (the measured row ran on that change's pre-merge revision, not on the squashed commit), five batch-4 fits (384 updates each) reduced the median from 115.651 s (115.136–116.204) to 95.898 s (95.501–96.696). The additional median paired reduction was 16.93%, range 16.38–17.82%, with the same six state/loss hashes equal. This is a separate workload row, not a percentage to add to the batch-8 result.

The final real-input replay compared 1,258,291,200 consumer image bytes over all 256 COCO images: zero differences, with equal packed geometry and a stable repeated baseline. The L4 passed 172 focused regressions, 28 changed-code doctests and one targeted augmentation-producer doctest; ten pre-existing doctest examples were explicitly skipped. A scalar-divisor mutation fails the captured CUDA pixel and the CPU-only divisor contract; restored code passes. Float32 and FP16 mixed precision each completed a finite one-batch real-data compatibility smoke with deferred mode active. Full pre-commit passed on CPU torch 2.14 with 149 focused tests and three CUDA skips.

Focused tests cover RNG order, ordinary and deferred transport, masks, consumer precision, the CPU fallback, validation and CUDA rounding. _host_sample lets the grid and golden helpers accept a deferred sample without quantizing it (unit-tested); both scripts build the default pipeline, so their outputs are unchanged. The complete-fit evidence uses public COCO, includes loading, augmentation, forward/backward, optimization, EMA, validation and checkpoints, and compares loss and final-state bytes. Full make gate was not run; this work package remains open.

WP-197b — static matrix iterations keep their arithmetic and lose host dispatch

The captured region is pure. capture_orthogonalization, off by default, records the existing scalar and batched Newton--Schulz functions rather than the optimizer step. Each new matrix is copied in and each returned result is copied out. Momentum, parameter writes, learning-rate schedules, gradient scaling and EMA stay outside the graphs. Neither function's arithmetic changes. CPU, MPS, autograd, autocast, non-contiguous and offset inputs keep the eager functions; an enclosing capture or compiler also keeps the original operations.

The optimizer owns the workspace. The cache is bounded by its matrix-parameter count. Function, shape, stride, dtype, device, step count, epsilon, CUDA stream, GEMM precision, BLAS backend and determinism select a recording. New signatures fall back to eager when that bound is reached. Separate graph pools allow skipped gradients and changing bucket order. A failed recording publishes no entry; loading optimizer state drops old recordings. Legacy state without the new field honours the constructor's requested mode.

The evidence is complete fits rather than a helper timer. [MEASURED] Five paired six-epoch Trainer.fit runs per starting tree on an NVIDIA L4, driver 580.178.04, PyTorch 2.9.1+cu129 and Lightning 2.6.6. Nano detection, 640 px, batch 4, BF16 mixed, four persistent workers and decoded-image caching use the first 256 sorted public COCO val2017 images (1,834 annotations). The 32 validation images overlap the training slice: this is a throughput and numerical-trajectory workload, not held-out accuracy or a new acceptance tier. Timing includes graph setup, loading, augmentations, backward, optimizer, EMA, validation and checkpoints; process imports and the CPU module constructor precede the measured fit.

Starting tree	Eager fit seconds, median (range)	Replay fit seconds, median (range)	Time saved
Main before the compatible-batching merge (01b4d4c), no device HSV	99.092 (98.506–99.243)	79.515 (79.141–80.062)	19.76%
Compatible batching and device HSV applied, expected benefit on current main	88.009 (87.149–88.255)	76.960 (76.671–78.001)	12.55%

[MEASURED] Each of the ten pairs has identical byte hashes for model, optimizer momentum state (not the full optimizer state_dict: the capture_orthogonalization flag sits in param_groups and differs by design), scheduler, EMA, loss sequence and validation metrics; the baselines match themselves too. Separate source hashes bind the runs to the executable code. Neither original orthogonalization body changes. Replacing replay with eager execution fails the committed Python-work-count regression.

[MEASURED] Main peak allocated CUDA memory is 829.8 (821.0–829.8) to 880.8 (880.8–880.9) MiB; peak reserved memory is 1078.0 (1070.0–1086.0) to 1298.0 (1298.0–1306.0) MiB. [MEASURED] The composition peak allocated CUDA memory is 809.7 (806.1–817.8) to 867.1 (863.1–872.4) MiB; peak reserved memory is 1082.0 (1082.0–1090.0) to 1270.0 (1268.0–1278.0) MiB. These are allocator metrics, not an OOM or capacity benefit.

[MEASURED] Five separate two-update real-data probes time the 41 initial recordings at median 188.1 ms (188.0–192.7); this setup is also included in each full-fit result above. An earlier batch-8 prototype was screened only once and was not advanced without repeated complete-fit evidence. Larger batches, other models, DDP and other accelerators have no performance claim; retained workspace and startup make the flag opt-in.

Scope and checks. [MEASURED] The focused CPU/doctest runs pass 228 cases. The final replay test module passes 35 cases (30 CUDA and 5 CPU/doctest) on both PyTorch 2.4.0+cu124 and 2.9.1+cu129 on the L4; full precommit passes. Focused CPU and real-CUDA regressions cover independent returned storage, fresh inputs, bounded signatures, precision changes, skipped gradients, resume, eager fallbacks and failed capture. Reusing a cached integer signature for a float step count is specifically refused through the original eager error. test_cli.py::test_model_flag_reaches_hparams mirrors WP-184's compile_step precedent and proves --model.capture_orthogonalization reaches DetectionLitModule.hparams through the real parser rather than a constructor fixture. The full development and accelerator golden gates are outside this run's authorized validation scope; this row therefore remains open.

WP-197c — replay the surrounding optimizer work

Existing preparation, a different submission schedule. capture_updates, off by default, records MuSGD's update preparation and momentum per group. Each step copies new gradients into owned inputs; the current learning rate and parameter writes stay outside capture. The native iterator still writes immediately in its original order. No coefficient, iteration count or optimizer expression changes. Warmup restores gradient identities and momentum, and loading state invalidates recordings. Layout copies preserve values while matching parameter strides for the final foreach write.

A recording certifies its inputs. Dense independent float32 CUDA storage, live identities, layouts, scalar settings and the CUDA execution context gate reuse. Scalar keys preserve Python types and float bits; float iteration counts keep the native error, and signed-zero changes keep native momentum bits. Missing gradients, aliases, changed signatures, other devices, autograd, autocast, inference mode, compilation and enclosing capture use native preparation. Subclass iterators stay native because an override could write parameters during recording; the sibling pure orthogonalization wrapper does not depend on that iterator. Setup and retained workspace make this optional.

Complete public-data fits. [MEASURED] Five alternating paired ten-epoch Trainer.fit runs after one discarded pair: nano detection, 640 px, batch 4, BF16 mixed, seed 42, four persistent workers and four torch threads. The first 256 sorted official COCO val2017 images train; the next 32 validate without overlap. Both arms enable decoded-image caching, device HSV and the already-merged orthogonalization replay. The isolated NVIDIA L4 uses driver 580.178.04, PyTorch 2.9.1+cu129 and Lightning 2.6.6. Graph setup, caches, loading, augmentation, transfer, forward/loss/backward, clipping, optimizer, EMA, validation every epoch and checkpoints are timed. Imports and CPU module construction precede the synchronized fit interval.

Complete-fit result	Native preparation	Update replay
Seconds, median (range)	120.850 (120.542–122.793)	108.750 (108.123–110.967)
Peak allocated MiB, median (range)	877.8 (871.6–877.8)	924.6 (924.6–924.6)
Peak reserved MiB, median (range)	1204.0 (1202.0–1212.0)	1292.0 (1286.0–1294.0)

[MEASURED] Complete-fit time falls 10.01%; paired reductions span 9.62–10.33%. Every model, learned optimizer, scheduler, EMA, loss-sequence and validation-metric byte hash agrees within pairs; independent baselines agree too. Only the execution flag is excluded from optimizer metadata and recorded separately. Actual state tensors from a preselected pair are saved and independently re-hashed. This is throughput and numerical-trajectory evidence, not convergence or accuracy. Added memory is a cost, not a capacity benefit; one batch-32 screen showed no benefit.

Checks and scope. [MEASURED] 169 focused CPU/CUDA/doctest checks pass with 4 expected skips on PyTorch 2.9.1; 165 pass with 8 skips on the exact 2.4.0 floor. Layout, inference-guard and scalar-key mutations fail the corresponding regressions. The CUDA class in tests/optim/test_update_replay.py (every replay byte-parity, reload, AMP, layout, alias and signed-zero check) is marked gpu and skipped without CUDA, and make test deselects gpu: those checks run only on a CUDA host, and CI certifies the CPU-native route, the reload flag and the CLI wiring alone until gate-gpu.yml has an armed runner. Optimizer sources are byte-identical between e5d7a85 and 623edc0, so a CUDA run recorded against either commit covers both. Full precommit and strict mypy pass. The full development gate and accelerator goldens were not run, so the row stays open. Other tasks, models, accelerators, DDP and long convergence carry no performance claim. R42 governs the CUDA API; all preparation expressions come from this repository and our own profiles.

WP-198 — prepare the same training images with less exposed work

The CUDA owner profile at batch 64 and 640 px attributed 113.406 ms to HSV, 9.999 ms to quantization and 5.481 ms to dequantization, before the channels-last copy. The operator trace materializes full-size intermediate batches around the existing pointwise arithmetic.

The transfer hook now requests one Triton pixel pass for contiguous float32 CUDA images and stated float64 gains. It preserves expression order, precise tensor division, double addition before casting the saturation/value gains, nearest-even uint8 rounding and CUDA's scalar-divisor reciprocal on restore. Fused multiply-add contraction is disabled. A fresh channels-last output avoids both intermediate batches and the final layout copy. The 256-pixel launch was selected from a measured 128/256/512/1024-block matrix; it is a scheduling choice. R44 registers the optional compiler, not a method implementation.

The existing device_hsv opt-in remains the entrance. Missing compiler/backend, unsupported devices, capture/compile scopes, noncontiguous inputs and other consumer precision keep the native path. The release guard accepts explicitly declared consumer extras only when their imports handle absence; development-group and unguarded imports still fail. No tensor, output or gain is cached.

Without CUPTI, a two-epoch nano run spent 13.650 s waiting for the first training batches and 3.222 s across the other 62 batches. The consumer can hide part of that first-batch work: an opt-in epoch_prefetch loader resets its existing persistent iterator after fetching the final batch, so workers prepare the successor during the last step and validation. Each public iterator still ends at its own epoch. The module permits this only for complete, single-CUDA epochs with its dedicated random sampler; distributed sampling, limits, step bounds, overfit and loader reloads retain the ordinary loader.

The shuffle generator is separate from the model's global RNG. An unused prepared epoch carries a generator-state receipt so teardown and a fresh worker pool do not skip a shuffle. A synchronous reset failure restores that state and is reported when the next epoch is requested, after the already-fetched batch has been delivered. The callback keeps a weak reference to its Trainer, and the prepared marker weakly references the existing iterator so Lightning teardown can release unused prefetched batches after an early stop. A regression through the real CombinedLoader reset catches strong retention. It creates no additional pool or executor. Queued training batches can remain resident during validation, which is why the flag is opt-in.

The isolated L4 screening pair used 2048 public COCO2017 training images, 64 disjoint validation images, nano/B64/640/BF16 mixed, eight persistent workers and cached decoding. Each epoch is 32 training steps (2048 images at batch 64). Three complete epochs, including validation and checkpoints, took 131.179 ->119.761 s; whole processes took 138.139 ->126.883 s. All six model, optimizer, scheduler, EMA, loss and metric hashes matched. This is a screen, not the repeated headline. Five fresh-process pairs over ten epochs, alternating order with an empty compiler cache per fit, reduce median complete-fit time from 379.619 s (377.463–380.130) to 326.999 s (325.952–355.677), or 13.86%. Whole-process medians are 386.249 s (384.025–387.338) and 333.961 s (332.686–362.380), or 13.54%. The median CUDA-event steady window is 807.293 -> 727.509 ms per step. All ten fits retain identical six-state byte hashes. The slowest candidate is retained; paired fit reductions range from 6.43% to 14.09%. Strict determinism and allocation filling remain enabled. Full make gate was not run and this package remains open.

The 13.86% headline combines two features, each measured separately: colour fusion alone saved 7.15% in the ten-epoch screen, and epoch preparation alone saved 4.50% in its three-epoch screen. A CPU CopyPaste/rasterization combination reached 7.45% with fusion and was discarded. Epoch-boundary savings depend on dataset size and validation duration; these short-fit measurements are not a full-COCO training claim.

WP-199 — the existing dynamic loss runs in a bounded CUDA workspace

Our L4 CPU/CUDA profile identified repeated assignment and loss launch work after the already captured network. The new option replays that remaining detection objective, using this repository's existing expressions and R43's neutral runtime API.

capture_loss, off by default, replays the existing detection assignment and loss forward/backward through R43's graphed-callable API. It composes with network and orthogonalization replay and records no optimizer update. The private assignment entry point separates validation from unchanged arithmetic; the ordinary public assigner continues to validate exactly as before. Both branches share a label-space check when their class counts agree. Different class counts retain both checks, and invalid current labels still raise before input copying.

A ground-truth capacity grows to the next power of two of an observed count, replacing the previous graph rather than retaining a family of pools. Padding is zero boxes, zero labels and false masks. Added rows select no anchor and contribute zero to conflict counts and normalization maxima; original anchor rankings, ties and target order stay unchanged. Shrinking counts clear every stale row. An all-empty batch keeps the native empty-target route, including finite inputs whose intermediate area arithmetic overflows. This is a scheduling choice derived from actual sizes, with no method constant changed.

Masked padding needs finite arithmetic and an epsilon that remains positive in float32. Built-in assigners with subnormal/non-finite epsilon or invalid exponents keep the native route, including constructor-valid positive Python epsilons that underflow on the device. Current non-finite tensor inputs retain the native loss, including its finite empty-target classification result when only decoded boxes contain NaNs. The extra check runs outside recording.

Each result owns its scalar and assignment storage. An outstanding backward leaves another forward eager; the next automatic batch clears a hold left by a skipped backward. Fit completion, reuse, exceptions, deepcopy and pickle discard the recording. Global module hooks, partial batches, another CUDA stream, unsupported layouts, inference, anomaly detection and frozen predictions and unverified older BF16 runtimes retain native execution. Changed criterion implementations, scalar settings or precision retire recording. Capture failures clear the partial owner and raise. Recording collects unreachable graph cycles before opening capture, suspends automatic cycle collection until the first replay completes, then restores the caller's GC mode on success or failure.

Focused checks cover real CUDA scalar/assignment/gradient bits across target-count growth, shrinkage and empty batches; current-label errors; old-result ownership; partial batches; changed gains and criteria, including the native empty-target result; keyword-only custom losses; failed recording; the real CLI and a real automatic Trainer. Mutating away stale-row clearing or replacing replay with eager execution fails the dedicated regression suite. The full development gate and accelerator goldens are not part of this validation; this package remains open.

[MEASURED] Five alternating fresh-process pairs on an isolated g2-standard-8 (eight vCPUs, 32 GB), NVIDIA L4, driver 580.178.04, torch 2.9.1+cu129 and Lightning 2.6.6: complete fit time is 125.272 (124.536–126.679) s native and 108.811 (108.582–109.466) s with replay, a 13.140% reduction (median and range). The workload is nano, batch 4, 640 px, BF16, five epochs on 1,024 COCO val2017 images used for training and 32 disjoint validation images, with four persistent training workers. Prepared caching, device HSV with its fused restore, epoch prefetch, network replay, orthogonalization replay and optimizer update replay are enabled in both arms. Timing includes construction, cache preparation, graph setup, training, schedules, EMA, validation and checkpoints. Dependency compiler and OS caches are warm; every fit rebuilds its image cache and CUDA recordings.

The model, full optimizer state, scheduler, EMA and complete loss tensor byte hashes match in each pair; validation metrics match too. The five native controls match each other. Actual replay plus native empty-batch counts cover every training batch. A separate batch-16 exploratory pair was 7.767% slower. The claim covers the measured small-batch workload. Long-run accuracy and full-COCO throughput remain unverified.

The final loss replay suite passes 59 cases on L4 torch 2.9.1 and 58 with one intentional BF16 skip on torch 2.4.0. Eight optimizer-flag CLI cases passed before the test-only composition addition, on the unchanged API. CUDA cases carry the GPU marker used by the project's test target. Real automatic Trainer coverage includes accumulation, clipping, FP16 overflow recovery, checkpoint plus assign=True restoration and simultaneous network/loss replay with both optimizer replay options. Seven mutation controls fail their dedicated regressions. The normal project environment passes the complete pre-commit set, including strict typing. Historical preliminary alpha compilation-counter and copied torch2.9 stub diagnostics reproduced on unchanged pre-merge main; these were not rerun on the merged source and neither is a green full-suite claim. Full development and accelerator-golden gates were not run; this package remains open.

WP-199b — one opt-in pose loss execution path

capture_loss, off by default, also accepts pose. Its unrotated detection objective uses the existing recording, and the current assignments feed the native point gathers. During training, a certified native RLE flow scores the two branches' selected rows together. Visibility selection precedes arithmetic, the split uses the selected point count, and each branch retains its own mean and alpha weight. The density expressions, shared parameters and optimizer remain the existing ones.

The flow has six row-independent coupling conditioners, each with three linear calls. Batching reduces those forward calls from 36 to 18. Source and runtime identities, hooks and compiled modules are checked before pairing; altered scoring, custom densities and validation retain their native calls. Different branch input dtypes/devices and tensor subclasses also keep separate calls, preserving each branch's logarithm precision. Existing detection recorder guards and fit cleanup stay in force for the detection objective; they do not gate the density batching, which keys on the option being active for the fit and so continues after the recorder retires, under a customized criterion and under the BF16 fallback for torch older than 2.9. Laplace-only pose keeps its native point objective.

This is not a byte-identical training option. Changing the batch can change matrix-kernel selection and the order of parameter-gradient sums. CPU double comparisons establish the separate branch objective and derivatives; CUDA comparisons measure the rounding difference. Default execution is unchanged. Long-run convergence and acceptance accuracy remain unverified.

Initial profiles on an L4 identified host dispatch rather than a dominant device kernel: median kernel duration was about 1.4 microseconds, and the eager detection loss and RLE forward scopes together occupied about 15% of the instrumented step window before backward. Those initial diagnostic runs used the module's 0.34-depth example, not the registry nano model, and are not a nano throughput result. Detection-loss replay alone reduced the exploratory fit by 4.54%; density batching alone by 7.54%. Neither was sufficient alone. Complete-fit validation uses registered nano (depth 0.50), batch 4, 640 px and BF16 on one L4 with Torch 2.9.1+cu129, five epochs over 384 public COCO person images and 32 disjoint validation images. Both arms enable the existing applicable training options. Five alternating fresh-process pairs include model/trainer and cache setup, graph recording, training, EMA, validation and checkpoints. Native median (range) is 100.586 (99.890–101.894) s; pose loss execution is 87.058 (86.201–87.297) s: 13.45% less complete-fit time. Whole-process medians are 106.025 and 92.389 s. Peak reserved CUDA memory rose from 1404 to 1496 MiB on the measured arms; as with the timing, the submitted tree has no memory measurement. The native arm is main at 0c84549. The commit of the pose loss execution arm is not recorded and is pending the contributor. The submitted tree builds on 22d8df4 (pose prediction-stem replay, WP-193f) and main has since gained 5756ea0 (segmentation replay); neither that tree nor the merged one has been re-measured, so the percentage is a result for the measured arms only. One complete batch-16 pair measured 48.357 to 46.338 s (4.18% less time), below the investigation's 8% target, so the repeated claim stays at batch 4. Same-weight public COCO samples have equal forward loss values but up to 1.45% relative gradient L2 difference in BF16; the five-epoch terminal training losses are 38.89 and 42.14, but which arm ended at which value is not recorded and is pending the contributor, so the accuracy effect cannot be read from them; short-run throughput does not establish accuracy preservation.

Focused regression tests cover separate visible means, asymmetric and empty subsets, non-finite unlabelled annotations, gradient agreement, an executable 36-to-18 call count, altered dispatch, real automatic CUDA training and composition with the existing pose network replay (capture_network). The tests proving this are tests/losses/test_rle_pair.py::test_pair_keeps_separate_visible_means_and_gradients, tests/losses/test_rle_pair.py::test_pair_halves_native_linear_calls, tests/losses/test_rle_pair.py::test_pair_certificate_refuses_altered_dispatch, tests/ptl/test_pose_loss_execution.py::test_pose_host_execution_keeps_loss_and_gradient_values, tests/ptl/test_pose_loss_execution.py::test_pose_custom_scoring_keeps_native_dispatch, tests/ptl/test_pose_loss_execution.py::test_pose_cuda_execution_keeps_objective_and_gradient_direction and tests/ptl/test_pose_loss_execution.py::test_pose_cuda_execution_composes_with_network_replay. The full development gate, aggregate offline coverage and accelerator goldens remain unverified. This work package stays open.

WP-200 — six native outputs, one recorded network

The oriented head is part of the recording. The existing capture_network option now accepts task: obb. Backbone, neck and both head branches run their existing expressions through R43's graphed-callable API. The adapter retains the four dense detection fields and both angle tensors, rebuilding them by name. Assignment, rotated decoding, classification, ProbIoU, rotated L1, angle scoring, schedules, clipping, optimizer, EMA and validation stay in their current loop. No method arithmetic, gain, convention or frozen golden changes.

The complete native objective owns the gradient contract. Its empty-target path retains zero gradients to both angle stems. An overridden objective can instead leave those gradients absent, so a changed training consumer, classification loss or oriented helper keeps the complete native call. Accumulation also stays native, as it does for pose: the existing shared recorder has no established accumulation contract. The ordinary network storage, hooks, precision, single-device CUDA and first-order constraints remain unchanged. One image signature is retained per fit; a temporary partial batch stays native and can return to the existing recording. The default stays off.

The stage was priced before implementation. Four native CUDA steps occupied a 1.466 s instrumented window, with about 24.1% covered by recorded device activity. Network forward dispatch took 378.113 ms, including 174.956 ms in backbone and neck and 202.463 ms in the head. A separate complete fit with lightweight host timers took 68.263 s: network forwards took 15.943 s, or 23.35%, across 208 training and 32 validation calls. That includes validation, which remains native, and is an upper bound rather than a speed claim. The 21.261 s backward timer includes loss work as well as network work and is not wholly removable. Profiler runs are separate from the final timing comparisons.

The smaller recording did not clear the investigation's target. A shared-backbone/neck-only screen preserved model, optimizer, scheduler, EMA and loss bytes, but changed complete-fit time from 68.277 to 63.800 s, a single-pair 6.56% reduction. It was set aside before the final cohort. The complete six-output screen preserved those same states and validation values and changed 68.742 to 59.853 s, a preliminary 12.93%; the repeated cohort, rather than that screen, determines the final claim.

Workload. The registered nano oriented model trains on 104 public DOTA-v1.0 tiles with 2,075 annotations and validates on 13 tiles with 616 annotations. Training and validation use disjoint original source images, 16 and 4 respectively. Eight epochs, batch 4, 1024 px, BF16, seed 123, deterministic kernels, four persistent loader workers and one NVIDIA L4 on a g2-standard-8. Both arms enable prepared source caching, device HSV, epoch prefetch and the existing orthogonalization/update-preparation replay. The comparison uses separate upstream and candidate source trees in fresh processes. Timing brackets include model/trainer construction, cache and recording setup, augmentation, training, EMA, validation and checkpoint writes, with device synchronization at both boundaries.

[MEASURED] Five alternating fresh-process pairs. One excluded warmup pair precedes the five measured pairs. Native and replay ranges do not overlap; every paired reduction is positive (11.669–13.572%). The exact model, optimizer, scheduler, EMA and per-step loss byte hashes agree in each pair and across native repeats, as do validation values. The package import path and hashes bind each arm to its separate source tree. The complete-fit median drops 12.011%.

Metric, median (range across five runs)	Native	Replay
Complete fit, seconds	68.642 (68.213–69.333)	60.397 (59.924–60.784)
Whole process, seconds	73.795 (73.379–74.561)	65.608 (65.169–65.987)
Warmed training step, ms	183.275 (179.156–184.821)	139.500 (138.453–141.266)

Warmed step diagnostics discard the first 64 steps and use CUDA events, synchronized before reading. Peak allocated memory is 2,572,018,688 bytes native and 1,889,886,208 bytes replay; peak reserved memory is 4,167,041,024 and 4,508,876,800 bytes, respectively, constant across the five runs. Extra graph-pool reservation is a tradeoff, not a capacity improvement. Dependency/compiler and OS caches are warm; each fit reconstructs its prepared image cache and CUDA recording. Package import and post-fit byte hashing are outside the fit timer and included in the separately reported process time. Runtime: Python 3.11.17, torch 2.9.1+cu129, Lightning 2.6.6, L4 driver 580.178.04.

The removed work is observed at its owner. Separate four-step CUDA traces show 31,193 native cudaLaunchKernel calls and 12,609 with replay, while graph launches rise from 4 to 12, adding the network forward/backward recordings beside the existing optimizer replay. Device-event union is 353.374 and 358.045 ms; these instrumented windows support the dispatch diagnosis, not a kernel-speed claim.

A larger-batch probe limits the claim. One batch-8 pair, with the same eight epochs and 1024 px inputs, changes complete-fit time from 60.494 to 58.014 s, only 4.099%. State and validation parity still hold, but that one pair is below the investigation's 8% target and is not a repeated performance claim. The five-pair headline remains scoped to batch 4.

Validation. Thirty oriented/constructor cases passed on both torch 2.9.1 and 2.4.0; the floor retains its existing BF16 eager fallback, while FP32 and FP16 exercise real replay. A silent eager reversion and an unsafe objective-guard bypass each make the committed Trainer tests fail. Five additional helper-replacement cases pass on CPU after review. Local CPU checks and source doctests, with those cases included, report 17 passed and 19 CUDA skips. Strict mypy and the full pre-commit hooks pass. No frozen golden is changed.

Coverage and limits. Focused tests compare all six fields, actual CUDA launches, empty-target gradient presence, temporary shape fallback, gradient clipping, accumulation retirement (native fallback), FP32/FP16/BF16, scaler overflow and recovery, native-objective fallback, EMA validation and fit cleanup. FP16 fixtures use a unit initial scale plus one injected overflow so at least one real optimizer update must occur. A silent native reversion must fail the launch assertions; bypassing the objective guard must be rejected by the absent-angle-gradient case. Full make gate, accelerator goldens and long-run acceptance accuracy are outside this validation run.

WP-200b — retain the expressions, reduce dense intermediates

Dense assignment remains visible after network replay. Matched CUDA profiles on the nano oriented model attribute 80.969 ms of summed GPU work per training update to candidacy, alignment and conflict resolution. That is roughly 9.234% of complete time per update in the measured recipe, a non-overlap opportunity estimate rather than a critical-path bound. Native target construction is observed separately and is not part of the optimized-stage numerator. Actual repeated complete training decides value.

One opt-in request, three device families and four passes. fused_assignment=False is a keyword-only model and assigner option. When enabled, the route writes cross IoU directly, gathers class scores and applies the existing mask, tests rotated containment and resolves contested anchors without the large broadcast intermediates. PyTorch still supplies IoU powers and rotated sine/cosine. Top-k, the small-target and one-to-one filters, target construction/normalization, gains, epsilon and method settings stay native. Native target maxima retain their reduction order and signed-zero behavior. Conflict reduction returns the first owning index, including NaN/tie cases. Multiply-add contraction is disabled; explicit round-to-nearest PTX division preserves subnormal inputs/results. No model/optimizer arithmetic or frozen golden changes.

Unsupported execution stays native. The request is scoped to one assignment and restored after nesting or failure. CPU/MPS/non-NVIDIA devices, tensor subclasses, unsupported dtype/layout, compiler tracing and oversized index/grid/reduction configurations keep the original expressions. Triton 3.x on SM80 or newer is optional; importing either module without it remains safe for CPU-only tooling. A first launch failure warns once and disables that family; verified execution errors propagate. First eager launches finish before their signatures become capture-ready; unseen signatures stay native inside capture. Only compiler readiness metadata is cached, never inputs, outputs or graph pools. Recording certificates include the option, delegated helpers and kernels.

Public workload. DOTA-v1.0 has 104 training tiles / 2075 annotations and 13 validation tiles / 616 annotations, split by disjoint original source images 16/4. Nano OBB depth 0.5 / width 0.25 / max_channels 1024, 15 classes, 1024 px, batch 16, BF16, seed 739; four persistent workers/four Torch threads, one warmup epoch, clipping 10 and mosaic throughout (close_mosaic=0). L4/g2-standard-8, Torch 2.9.1+cu129, Triton 3.5.1, Lightning 2.6.6, driver 580.178.04. Both arms enable merged prepared-image caching, device HSV, epoch prefetch, network replay, orthogonalization replay and update preparation.

Five alternating 50-epoch pairs use fresh processes/contexts, prepared caches, recordings and unique empty Triton caches in both arms. Earlier excluded runs warm OS/dependency/driver caches. The complete clock begins before model/data/trainer construction and ends after synchronized fit/checkpoint completion, including cache building, recording/compilation, validation and EMA. Imports and post-fit receipt hashing are outside it and included in whole-process time.

Measurement, median (range)	Native	Fused	Time reduction
Complete fit, seconds	306.911 (300.181–311.299)	273.342 (273.152–279.940)	10.94%
Whole process, seconds	312.132 (305.372–316.486)	278.525 (278.367–285.148)	10.77%
Full-batch CUDA-event step, ms	446.689 (445.190–447.819)	360.010 (359.133–360.173)	19.40%

All 350 loss byte strings per run, model/optimiser/EMA/scheduler state hashes and validation-metric byte serializations match across all ten runs. Baselines agree against themselves. Step diagnostics use identical full-batch indices after startup, excluding compile-preparation steps in either arm. Every epoch retains an active network recording; all fused arms execute the three families with none disabled. The complete/process clocks decide value; events explain the mechanism. Peak allocator statistics are recorded separately, with no OOM/capacity claim.

Validation and limits. Frozen-runtime VM checks pass 372 CUDA cases, excluding two exact alpha-NaN recorder failures reproduced on clean main. Torch 2.4 / Triton 3.0 passes 89 focused cases with two expected old-BF16 replay skips. CPU/doctests pass 39 cases with 48 GPU skips; 12 compiler cases retain one graph. Full pinned pre-commit passes. Public zero-score regression cases preserve native zero-weight sign bits; the missing-compiler regression covers direct module import as well as the production constructor. Real graph replay covers changed values and unaligned views. Full assignment/graph memcheck and isolated new-pass/graph initcheck pass; whole-assignment initcheck is not clean in native Torch radix top-k, reproduced by a bare dependency control. No false-positive diagnosis is made. Source/tests/extra compose with the published gradient-norm proposal and retain exact four-epoch states; that check is integration proof, not a performance claim against it.

After review, both graph tests also witness non-native results during capture, and the CUDA marker follows the backend’s SM80/compiler eligibility. The final assignment file passes 86 cases on an RTX 4060 with Torch 2.14.0+cu130/Triton 3.8.0; forcing only capture to stay native makes both graph tests fail. These are local compatibility checks. All 97 runtime files still match the five-pair VM snapshot; the strengthened assertions were not rerun on the older VM runtime lanes.

Full make gate, the offline 96% aggregate coverage gate, accelerator goldens and long-run/full-dataset acceptance accuracy were not run. Other tasks, devices, runtimes, smaller batches and short fits have no performance claim. Startup costs can dominate short fits; verify complete training time before enabling the option for another workload.

WP-200c — current metadata and a recorded gradient reduction

The complete CUDA training loop still performs repeated module enumeration and scalar-attribute filtering before replay. On a real COCO keypoint training profile, network replay checks account for 0.211 of a 0.682 s four-step cProfile window (31.0%); signatures account for 0.115 s (16.9%). These interpreter-instrumented shares identify the mechanism and are not absolute throughput evidence. An independent initial segmentation Nsight timeline shows a large host idle budget, while its largest standalone kernel family contributes only 2.61% of that window, so a compute-only rewrite of that family has too little device-time budget for the intended complete-training saving there. Reducing submissions can also remove host gaps; that larger budget motivates recording the native reduction.

The membership walk now avoids unused names under native enumeration, shares the current parameter/buffer lists between checks and retains the recorded parameter list for backward protection. A custom enumeration route retains the original calls. Cached scalar metadata is used only while every ordered attribute binding and class forward remains identical. Tensor storage, shape, stride, dtype, device, trainability and hooks are still read independently, and changed contracts keep the existing retirement policy.

The optional capture_gradient_norm flag records the dependency's existing get_total_norm call on private CUDA gradient inputs. Current gradients are copied in before replay. The native clip_grads_with_norm_ call applies the current clipping limit to the live parameters; aliases and duplicate writes keep that call's semantics. The flag requires torch 2.9 and single-device CUDA automatic optimisation. Default off, value clipping, offset views, unsupported layouts and custom clipping keep their original route. One recording is retained per fit, and input/policy/stream drift retires it once. Successful fits, failed fits and copying release the recording. A callback retained by Trainer for a later unrelated model is a no-op, including the exception hook. Model, loss, assignment and optimizer expressions are unchanged.

This package derives from the current repository, our own measurements, R42 and R45. It introduces no method constant or numerical assumption. CUDA buffers and setup cost are explicit costs of the opt-in flag.

Five alternating fresh-process pairs against main 4dc5ddd on an isolated NVIDIA L4 reduce complete nano pose fit time by 9.55%, including setup and validation. Whole-process time decreases by 9.13%. These are the final corrected implementation, including warmup-failure ordering, offset-view fallback, unrelated-model lifecycle fallback, live public training-mode checks, post-warmup metadata certification with the original retirement stamp, and native clipping-conflict validation. Earlier production cohorts remain supporting evidence only.

Median (range)	Main 4dc5ddd	Patch
Complete fit, s	98.342 (96.295–100.413)	88.946 (88.575–88.961)
Whole process, s	103.682 (101.545–105.764)	94.218 (93.900–94.437)
Steady train step, ms	108.671 (106.886–109.805)	95.709 (94.944–96.188)

The training-only step row fences the CUDA stream at epoch boundaries before validation or EMA weight swapping. It discards the first three epochs and divides the remaining seven training windows by their 448 steps. The complete-fit row retains every warmup/setup cost. Each fit uses 256 person-containing public COCO val2017 images for training and 32 disjoint images for validation, original keypoint annotations, nano, batch 4, 640 px, ten epochs and BF16 mixed precision. Four intraop threads and four persistent loader workers are used in both arms. Prepared-image caching, device HSV, epoch prefetch, applicable network/loss/orthogonalization/update replay and all-mosaic training are enabled in both. The timer starts before model/data/trainer construction and includes preparation, recording, training, optimizer, EMA, validation and checkpoint writing. Imports and output hashing are outside the complete-fit row and inside the whole-process row.

Model, optimizer, scheduler, EMA, all 640 step losses and validation metrics have matching value-byte hashes across all ten fits. Every loss is finite, every candidate batch observes the live norm recording, and every completed fit releases it. This is trajectory and execution validation on a public subset; it does not establish convergence or full-COCO accuracy.

The median training peak allocated memory is 587.10 to 587.10 MiB, and reserved memory is 1836.00 to 1846.00 MiB. These are separate allocator metrics, not a capacity claim. A separate single batch-16/three-epoch pair changes complete fit time by 3.60%; it is a boundary screen, not a repeated batch-16 claim. It changes both batch size and fit length, so it does not isolate batch scaling.

The measured source hashes are:

9ef4b97ff31077bdb7ff54591693972989bf75ab512bd12bb221d54276c92ad5  src/lucid_yolo/ptl/network_capture.py
0affdbc304d6be79723a92f7649ce4d7db1a3788babe9097d74d828edb67e986  src/lucid_yolo/ptl/module.py
300d5cf1314af0793effc758b7d5c7ad8864cbb44a1e044084a74c9b8a186b04  src/lucid_yolo/ptl/gradient_norm_capture.py

Python 3.11.17, torch 2.9.0+cu129, CUDA 12.9, Lightning 2.6.6, driver 580.178.04. The 57 focused cases include real public Trainer reuse across models, a later foreign optimizer update and preservation of its original training failure. The callback ignores a missing holder, following this repository's network-callback convention. Real stem retention is observed before three backward passes and the expected accumulated gradients; removing parameter retention fails that regression. Focused regressions also cover CUDA reduction/scaling for four dtypes, native aliases and duplicate writes, current limits, actual owner drift/retirement, copy/fit cleanup, controlled failed warmup/capture and a bounded linear FP16 Lightning fit with accumulation, one injected overflow and real optimizer updates. The FP16 fixture isolates the norm/precision hook; it does not certify the separate full-network FP16/accumulation/clipping interaction investigated in the controls. CPU fallback, older-runtime routing, custom precision behavior and the live metadata checks are also exercised. No new arithmetic or method constant is introduced. The original submission corrected the checkpoint-assignment CUDA test to compare the eager gradient pattern for a classification-only loss. After synchronization with main, that regression instead supervises all four detection outputs and requires every parameter gradient, retaining main's stronger oracle. At the measured revision, full make gate, aggregate offline coverage, accelerator goldens and long-run acceptance were not run.

Integration on 28 October 2026. This package is renumbered from WP-201 to WP-200c because main now owns WP-201. The merge retains main's pre-forward live-gradient copy and shared recording, tensor-signature and cleanup helpers. The parameter cache supplies the existing copy helper before replay; the regression follows that timing. The performance table and source hashes above describe the original measured revision, not this integrated tree. No performance measurement of the integrated tree was made.