Opt-in CUDA performance flags

Opt-in, mostly CUDA only, mostly in progress. Every flag on this page is off by default (or, for fused_hsv, on only where its toolchain exists). Most need a CUDA device and belong to a roadmap row that is still open (ROADMAP rows 192c to 200). Exceptions: the host-side image cache cache_train_images is built in the data module and needs no accelerator, and the rows for it (WP-192b) and compile_step (WP-184) are done. None of them is needed to train: the per-tier recipes in Launching a training run work without any. Each gain below was measured on one workload and one device; check complete-fit throughput and memory on your own batch size before enabling a flag.

The flags

The measured-gain column carries only figures stated in the sections below or in the linked work-package pages; a dash means no figure is stated there.

Flag	Task	Default	Measured gain	WP
--data.device_hsv	all (CUDA; 32-true, 16-mixed, bf16-mixed)	false	— (can help a CPU-limited run; image IPC and host-to-device bytes are four times the uint8 transport)	WP-197
--data.fused_hsv	all (needs device_hsv; Linux NVIDIA compute capability 8.0 or newer, Triton and a host C compiler)	on where available	—	WP-198
--data.epoch_prefetch (with --data.persistent_workers true)	all (single CUDA device, full epochs)	false	— (overlap measured on one L4 only)	WP-198
--data.cache_train_images	all	false	—	WP-192b
--data.cache_prepared_train_images	all	false	— (3.30 MB per image measured on a 1,024-image slice, about 390 GB per process extrapolated for COCO train2017)	WP-192c
--model.capture_segment_heads	segment (with --trainer.precision bf16-mixed)	false	— (a small-batch result does not establish a large-batch speed-up)	WP-193e
--model.capture_network	detect	false	small-batch only (batch 4); batch-32 screens changed direction	WP-193c
--model.capture_network	keypoints	false	12.97% saved, nano/640/BF16, batch 4, L4, five complete pairs, against shared-feature replay; reserved memory 1384 to 1730 MiB	WP-193f, WP-193d
--model.capture_network	obb	false	4.10% on a single batch-8 pair (Nano OBB, BF16, L4); peak reserved memory 3974 to 4300 MiB	WP-200
--model.capture_network	segment	false	—	WP-193g
--model.capture_gradient_norm	all (single CUDA device, norm clipping, torch>=2.9)	false	— (flag-only share unmeasured)	WP-200c
--model.capture_loss	detect, keypoints	false	13.14% saved, nano/640/BF16, batch 4, L4, five complete-fit pairs; a batch-16 exploratory pair was 7.767% slower	WP-199, WP-199b
--model.capture_orthogonalization	all (MuSGD)	false	—	WP-197b
--model.capture_updates	all (MuSGD, eligible dense float32 CUDA groups)	false	—	WP-197c
--model.compile_step	detect	false	x1.23 on the training step (74.7 ms eager, 60.9 ms compiled; RTX PRO 6000, variant n, COCO, batch 16, deterministic); 110–140 s compile per process	WP-183, D24 in Decisions

Two combination rules apply across the table. capture_network and capture_loss both refuse compile_step, and capture_network raises at construction for a task outside detect, keypoints, obb and segment. capture_segment_heads and capture_network can be combined for segmentation, and capture_loss can combine with capture_network, capture_orthogonalization and capture_updates.

What happens off CUDA, or off the supported case

Each row falls back with a UserWarning rather than failing, except where the last column says it raises. The first two rows are not flags: they apply to every run.

Feature	CUDA	CPU	MPS	When the case is not met
channels_last memory format (D23)	applied automatically to parameters and images, no flag	default layout	default layout	not applicable; nothing to configure
deterministic: true	strict True	strict True	"warn_only" (some backward kernels have no deterministic MPS implementation)	see What deterministic: true and seed actually pin
compile_step	detect only	trains eager, nothing to gain	trains eager, Inductor crashes in convolution_backward	warns once at setup and trains eager; the alpha scalar lifted for the compiled region (_alpha_tensor) stays on the host on MPS, which has no float64
device_hsv	with 32-true, 16-mixed or bf16-mixed	worker-side HSV	worker-side HSV	warns once and keeps worker-side HSV, as do precision plugins that cast the images or gains before the data hook
fused_hsv	Linux, compute capability 8.0 or newer, Triton and a host C compiler	native restore	native restore	native CUDA restore; a failing first compile or launch warns once and keeps it for the process
epoch_prefetch	single device, full epochs, module's own shuffle generator, persistent workers	ordinary loader	ordinary loader	ordinary loader; warns once naming the first unmet condition
replay flags (capture_network, capture_loss, capture_segment_heads)	single device, automatic optimisation, supported task, BF16 needs a final PyTorch 2.9 or newer	warn and stay eager	warn and stay eager	other strategies warn and stay eager; an unsupported task or compile_step raises at construction; a recording failure raises
capture_orthogonalization, capture_updates	eligible dense float32 groups	original functions / native step	original functions / native step	native step; a recording failure raises (capture_updates)

CUDA colour augmentation

--data.device_hsv true moves training HSV and quantization to CUDA with 32-true, 16-mixed or bf16-mixed precision. The workers still sample the same gains and horizontal flips, and validation keeps its existing path. The default is false; other backends and precisions warn once and keep worker-side HSV. Any precision plugin whose input conversion casts the float32 images or float64 gains before the data hook (true-half and double, and the FSDP and DeepSpeed plugins under every accepted precision) keeps the original path as well, with the same one-time warning.

On Linux NVIDIA GPUs with compute capability 8.0 or newer, the float32 consumer restore can fuse the existing colour operations and quantization into one channels-last write. This is on by default wherever Triton and a host C compiler are present — torch's CUDA wheel already selects a Triton, so an existing device_hsv run there takes the fused path without installing anything (uv pip install -e ".[cuda-hsv]" adds Triton where it is missing). --data.fused_hsv false keeps device_hsv on the native CUDA restore, to step around a toolchain or bisect a difference between the two. The first call compiles the kernel and needs a host C compiler. The extra also installs the existing Setuptools build dependency needed by older Triton SDKs. A Triton ImportError, a missing compiler, other devices, noncontiguous buffers and other consumer precisions retain the native restore. So does a first compile or launch that fails: it warns once with its cause, and the native restore runs for the rest of the process. Gain sampling, rounding and the uint8 boundary are unchanged. Direct unpack_batch calls keep their existing path.

This can help a CPU-limited training run, including one with decoded images cached by --data.cache_train_images true. It transports float32 images until the colour transform finishes, so image IPC and host-to-device bytes are four times the default uint8 transport. The shared-memory worker cap budgets float32 images only when the module chooses the worker count (--data.num_workers unset) in one process; segmentation masks still need their own memory allowance. A named --data.num_workers is neither capped nor warned: batch 64 at 640 px with 8 workers and prefetch_factor 2 queues about 5.0 GB of /dev/shm in float32, against 1.26 GB in uint8. Each DDP rank budgets the free /dev/shm on its own, so ranks sharing one host can jointly overcommit it — a known limit. The cap cannot shrink a batch: with a small /dev/shm, enlarge it or use --data.num_workers 0. Measure the complete run on the intended host.

Prepare the next epoch during validation

--data.epoch_prefetch true with --data.persistent_workers true lets the existing workers prepare up to num_workers x prefetch_factor batches of the next training epoch during the last step and validation. It is opt-in because the training queue can remain resident alongside validation inputs: num_workers x prefetch_factor batches of host memory, about 1.26 GB of uint8 images for the batch 64, 640 px, 8 worker, prefetch_factor 2 example above, or about 5.0 GB of float32 with --data.device_hsv true. With pinned host memory, which is on by default when CUDA is available, queued batches can sit in page-locked RAM that the /dev/shm worker cap does not budget. Use it when dataset and augmentation configuration remain stable between epochs. Every packaged config breaks that: its CloseMosaicCallback switches mosaic off for the final epochs through set_mosaic_prob, which persistent workers never see, because each keeps the pipeline copy it started with — those epochs would still train on mosaics. The module warns once when such a change arrives under epoch_prefetch; keep --data.persistent_workers false for these configs. It applies to full, single-CUDA epochs with the module's own shuffle generator; batch limits, a step bound, overfit, loader reloads and distributed training keep the ordinary loader, as do non-persistent workers, and the run warns once naming the first unmet condition. CUDA is the boundary because the overlap was measured only on one CUDA GPU (an L4); other accelerators are unmeasured, not known to fail. The configured final epoch does not prepare another one. Measure the complete fit: the benefit is greatest when first-batch preparation is a meaningful fraction of the epoch.

Decoded training-image cache

--data.cache_train_images true decodes each distinct training file name during datamodule setup and keeps the RGB uint8 tensor in host RAM. Mosaic, affine, colour and flip augmentation still run on fresh float storage for every draw; only the repeated JPEG/PNG decode disappears. COCO and YOLO training readers support the flag, and validation is deliberately not cached.

Budget the option from the uncompressed images, not their files: approximately sum(3 * height * width) bytes for 8-bit sources, before Python/tensor metadata — a 16-bit PNG decodes to uint16 and costs two bytes per channel, so twice that — and once per device: a multi-GPU run spawns one process per rank, each building its own DetectionDataModule and therefore its own cache, so the single-process budget above multiplies by trainer.devices. The cache is keyed by file name (by path for the YOLO reader), so two records naming one file share an entry, while the same picture stored under two names is decoded and held twice. It defaults off.

Whether DataLoader workers share the cache depends on the multiprocessing start method, not on the operating system, and the data module leaves the start method at the platform default. Under fork the workers read the parent's cache through copy-on-write pages, so it costs one copy of RAM however many workers there are. Under spawn and forkserver — the default on macOS and Windows, and on Linux from Python 3.14 — the cache is pickled to every worker, torch moves each cached tensor into shared memory to do it (one segment or file descriptor per image), and with --data.persistent_workers false, the default, the hand-over repeats every epoch. The values are identical either way; a non-fork start method does not promise the one-copy cost or the speedup, and a large split can meet file-descriptor or /dev/shm limits at a size nobody has measured. The pipeline never writes to the cache — every draw converts into fresh float storage. Leave it off when the split does not fit comfortably beside the loader queue, model, optimizer and operating-system cache.

Prepared training-source cache

--data.cache_prepared_train_images true also reuses float conversion and affine centring before random augmentation. It takes precedence over cache_train_images, keeps one float32 source per training record and snapshots YOLO labels with their images. Mosaic, geometry, colour, targets and RNG draws keep their existing results. The option defaults off.

Budget about 12 * sum(height * width) bytes for the retained RGB pixels, plus target and reader metadata, per training process. Extrapolated from the 1,024-image slice measured in WP-192c (3.30 MB per image), the full COCO train2017 split would need about 390 GB per process; this is an extrapolation, not a measured run. Every DDP rank is a separate process and holds its own copy, so ranks on one host multiply that figure. Each float32 pixel channel uses four times the bytes of an 8-bit channel. The prepared cache retains separate images for duplicate COCO records, whereas the decoded cache deduplicates file names. COCO reports an estimate before decoding; YOLO has no annotation sizes and reports the retained pixel bytes afterwards. Under spawn or forkserver (see above), workers receive cached tensors through shared memory, so budget the cache alongside the queued batches. Forked workers share its read-only pages. Fit-stage setup builds the cache once, repeated fit setup reuses it, and other stages release the datamodule's references. Measure the complete fit on the intended host.

Independent segmentation predictions

--model.capture_segment_heads true --trainer.precision bf16-mixed records the existing segmentation prediction routines, including their first-order backward work. The default is false. Box/class stems remain independent; coefficients, prototypes and semantic logits have separate backward nodes so an empty mask objective keeps absent gradients. With this option alone, common features, prototype fusion, assignment, losses, state updates and schedules stay native. Enable capture_network separately to replay the common features too.

Byte-exact agreement with native training is verified only on NVIDIA L4 (sm_89) with torch 2.9.1+cu129; other GPUs and torch builds are unverified. The route requires PyTorch 2.9 or newer, one CUDA device and automatic optimisation. Gradient accumulation replays too (WP-208): a live gradient is copied out of a group's recorded gradient buffers at forward time, before the replay that would overwrite them, as network replay does. CPU, MPS, FP32, FP16 and older BF16 runtimes warn and use native calls; evaluation stays native too. Custom or changed prediction dispatch, hooks, frozen prediction parameters, aliased storage and retained feature gradients retire replay for the fit. Each prediction group records one input signature per fit (shape, stride, dtype, device, gradient flag and CUDA stream); a call with any other signature, a partial batch included, runs the native head without adding another graph, and the recorded graph pools stay reserved until the fit ends, so a run whose batch shapes keep changing pays that memory without replaying. Copying, checkpoints and fit teardown retain the original parameter names and no recording.

A recording failure raises after restoring running buffers and RNG state; disable the flag to keep that run native. The flag retains CUDA graph pools, so check reserved memory and throughput on the actual batch size. A replayed step's mask coefficients, prototypes and semantic logits live in CUDA graph memory that the next replay overwrites in place; box and class predictions are copies. Anything that keeps those three tensors, or a gradient adopted from their backward, beyond the training step that produced them has to clone it. Measurements and rejected alternatives belong to WP-193e; a small-batch result does not establish a large-batch speed-up.

Opt-in CUDA network replay

--model.capture_network true records the detection backbone, neck and dual head with PyTorch's make_graphed_callables, including their backward pass. Assignment, losses, optimizer, LR/alpha schedules and EMA keep the ordinary loop. It is separate from compile_step and cannot be combined with that flag. The default is false. The measured speed-up is limited to a small-batch workload (batch 4); batch-32 screens changed direction, so check throughput on your own batch size before enabling it.

For task: keypoints, the same flag records backbone and neck plus eighteen independent box/class/point stems through the private prediction runner. Each point stem has its own backward node: a batch with no assigned positives keeps its point-parameter gradients absent. The current RLE/OKS or Laplace objectives stay native. A fully frozen backbone and neck, changed head topology or dispatch, and head hooks/compilation keep pose native; a partial shared-feature freeze still replays. A fully frozen start or a later change to the shared-feature trainable set keeps the rest of the fit native after one warning. Changing a stem's trainable set or module mode retires that stem's recording, with a warning naming the stem; replacing a head module retires the parent before feature replay. The measured benefit is for nano/640/BF16, batch 4 on L4: five complete pairs save 12.97%, including setup, validation and checkpoints, against the existing shared-feature replay. Reserved memory rises from 1384 to 1730 MiB. Check time and memory on another batch or device. WP-193f records the measurements and unverified limits; WP-193d retains the earlier feature-only result.

For task: obb, the flag records backbone, neck and the complete dual head, including both angle outputs. The native classification and oriented losses stay outside capture. Their empty-target expressions retain zero angle gradients. Overridden training consumers, classification losses or oriented helpers keep the complete native call. The oriented speed-up is likewise limited to a small-batch workload (Nano OBB at batch 4, BF16, one L4): a single batch-8 pair changed fit time by only 4.10%, and replay raised peak reserved memory from 3974 to 4300 MiB, so check throughput and memory on your own batch size. The measurements and remaining limits are in WP-200.

This path requires single-device CUDA automatic optimisation and task: detect, task: keypoints, task: obb or task: segment. BF16 replay requires a final PyTorch 2.9 or newer release; 2.9 pre-release and NGC a0 builds are excluded by design. 2.9 is the first verified runtime: older BF16 runs warn and stay eager because the 2.4 comparison did not meet the exact eager oracle. FP32 and FP16 replay of detection, oriented detection and pose shared features are checked on 2.4 as well; pose stem replay has run on 2.9.1 only (WP-193f). CPU, MPS and other strategies warn and stay eager; other tasks or compile_step raise at construction. One image signature is recorded per fit. A partial batch uses eager execution without growing a graph cache. Changes to the recorded stages’ parameter/buffer storage, module modes, hooks or precision retire replay for the rest of that fit. The supplied network must keep its ordinary stage implementations and static Python configuration. Root forward overrides, stage instance-forward overrides and registered hooks use the eager path. Anomaly detection (--trainer.detect_anomaly true) also stays eager so backward NaN checks run normally. A failure while recording raises instead of falling back to eager (RNG state and buffers are restored first), so disable --model.capture_network if recording fails on your model. Higher-order differentiation is not supported.

With detection and accumulation one, the module can omit snapshots that the native Lightning automatic-optimization closure clears immediately after forward. The version gate admits Lightning 2.4–2.6; 2.4.0 and 2.6.6 were executed and 2.5.0 was read from source only. Only the inspected native dispatch and pass-through profiler qualify; customized or unsupported contracts retain the snapshots. The module logs once per fit, at rank-zero info level, whether this snapshot-skip fast path is active and, if not, the first check that failed. This is an internal refinement of network replay; there is no additional flag. WP-193h records the measured workload and verification limits.

Recording warmup restores RNG and BatchNorm buffers before the first real batch. Accumulation or custom optimiser/zero-grad hooks preserve live gradients outside the graph-owned backward buffers before they can be overwritten. AMP uses an uncached autocast scope as the PyTorch API requires. Validation, prediction and public forward calls stay eager. Checkpoints retain the ordinary parameter names and the flag, with no graph state. Each fit records afresh and drops its recording on completion or an exception. CUDA graph pools can increase reserved memory even when peak allocated memory falls; measure both on the intended workload. The measured scope and remaining validation gaps are in WP-193c.

For segmentation, the same --model.capture_network true option records only backbone and neck. Prediction groups, prototype fusion and both losses keep their existing arithmetic and gradient boundaries. It can combine with --model.capture_segment_heads true, which records the prediction groups separately. Overridden or compiled segmentation forwards stay native. The default remains false; a recording is transient and follows the same fit cleanup and input-signature rules above. See WP-193g for the measured workload and limits.

Network replay also shares one current membership traversal between its eligibility and storage checks. Certified scalar module metadata is seeded after recording warmup and reused only while module identities, class forwards, live training modes and ordered attribute bindings are unchanged. Live tensor storage, layout, dtype, gradient flags and hook checks remain independent. This host-bookkeeping change applies whenever capture_network replays, independently of capture_gradient_norm, without changing the recorded model arithmetic.

Optional CUDA gradient-norm recording

--model.capture_gradient_norm true records PyTorch's global L2 gradient-norm reduction on single-device CUDA automatic optimisation with torch 2.9 or newer. The default is false. It is independent of network/loss/update replay and uses the trainer's existing norm clipping configuration, such as --trainer.gradient_clip_val 10. Gradient scaling stays in PyTorch's native call and reads the current limit on each step. Disabled clipping (a limit of None or at most zero) and value clipping retire recording once with the reason "clipping is disabled or not by norm" and keep the native call. Ordinary 16-mixed GradScaler training can capture the reduction after unscaling; optimizers that unscale inside their own step (_step_supports_amp_scaling, such as fused optimizers) keep the native call, including Lightning's clipping rejection under mixed precision. One gradient shape/stride/dtype/device and stream signature is retained per fit. Offset views and other incompatible gradients or changed public clipping implementations retire recording for that fit. Precision plugins are matched through their type's MRO against stock Precision/MixedPrecision clipping methods, with each method checked by identity: subclasses inheriting those methods remain eligible; customized module or precision-plugin clipping retires capture with a reason naming the plugin type and keeps the native call. CPU, MPS and unsupported runtimes warn and stay native; the torch gate says the norm decomposition is "verified only on torch>=2.9". Fit completion, failure and copying release the transient recording, which adds no model tensor keys to checkpoints.

Setup and private CUDA buffers have a cost. The 9.55% complete-fit reduction in WP-200c covers the whole patch on a pre-integration revision (nano pose/640/BF16, batch 4, L4, five complete pairs), including the network-replay bookkeeping change above. The flag-only share is unmeasured, and the integrated tree was not re-measured. Long-run accuracy remains unverified.

Optional CUDA detection and pose loss execution

--model.capture_loss true replays the existing detection assignment and loss forward/backward with fresh predictions, targets and the current alpha tensor. Default false: capture costs setup time and retains CUDA workspace, so short or compute-bound runs may not benefit. It accepts task: detect or task: keypoints, and requires single-device CUDA automatic optimisation and compile_step: false. It can combine with capture_network, capture_orthogonalization and capture_updates; capture_loss itself records no optimizer update, so the optimizer step is whatever those two optimizer options select (the existing eager path when both are off).

For pose, the unrotated detection objective uses that recorder while point gathers and OKS stay native. During training, the built-in RLE densities can score both branches in one batch, retaining separate visible-point means and branch weights. That batching is gated on the option being active for the fit, not on the recorder: it continues after the recorder retires and on steps whose detection loss takes its native call, such as a customized criterion or BF16 on torch older than 2.9. This reduces dispatch but can change matrix-kernel selection and parameter-gradient summation rounding: pose execution retains the mathematical objective, but training is not byte-identical. Hooked, overridden or compiled densities, altered pose scoring and validation use their original calls. Branches with different input dtypes/devices or tensor subclasses keep separate density calls. Laplace-only pose keeps native point scoring. See WP-199b for the measured scope and unverified accuracy checks.

One masked target workspace grows from observed instance counts and replaces the previous recording. Returned scalars and assignments own their storage. Current label bounds are checked before replay. Global module hooks, unsafe custom epsilon/exponent settings, entirely empty targets, partial batches, non-finite inputs, unsupported layouts or autograd contexts, another CUDA stream and older BF16 runtimes keep the native loss. FP32 and FP16 have focused CUDA coverage on torch 2.4.0 and 2.9.1; BF16 replay starts at the verified final 2.9 runtime. Changed criterion implementations, gains or precision retire the recording. A capture failure raises after clearing the partial owner. Higher-order differentiation and multiple backward passes through one recorded result are outside this automatic-training option.

On the measured L4 nano/640/BF16 workload, five batch-4 complete-fit pairs saved 13.14%. A separate batch-16 exploratory pair was 7.767% slower. Treat this as a small-batch launch-overhead option and measure the complete fit before enabling it for another workload.

WP-199 records the measured workload, discarded screens and remaining validation gaps. Check complete-fit throughput and memory on your own batch size before enabling it.

CUDA graph replay of orthogonalization

--model.capture_orthogonalization true records MuSGD's existing Newton--Schulz matrix calls and replays them with each new gradient. The default is false: private graph pools retain extra device workspace, and setup may outweigh the gain on short or compute-bound runs. Momentum, parameter updates and the learning-rate schedule remain eager. CPU and MPS keep the original functions. WP-197b records the measured workload and compatibility envelope.

CUDA update preparation replay

--model.capture_updates true additionally records MuSGD's update preparation, including momentum, for eligible dense float32 CUDA groups. Learning-rate changes and parameter writes remain outside capture. This option can be used with or without capture_orthogonalization; the two recordings do not nest. CPU, MPS and unsupported CUDA layouts or execution contexts retain the native step. A failure while recording raises instead of falling back to the native step (gradients and momentum are restored first), so disable --model.capture_updates if recording fails on your model. Loading optimizer state discards old recordings. The default is false because capture adds setup and retains workspace. WP-197c reports five complete public-data fits, numerical checks and measured memory cost; other workloads need their own measurement.

torch.compile (CUDA, detect)

--model.compile_step true (or compile_step: true under model:; det_nano_smoke.yaml carries it at false) compiles the detection objective with Inductor. What is compiled is one region, DetectionLitModule._detect_objective: the forward, both branch decodes, the task-aligned assignment and the dual loss, traced as a single graph with no breaks — tests/ptl/test_compile.py pins that on CPU for a batch with ground truths and for one with none. Compiling the model half alone was measured first and was not worth a flag: x1.09–1.16 on the step, because the assignment and the loss are where the small kernels and the launch overhead live. With WP-183's dense box terms (no data-dependent gather) and the label check skipped under tracing, the whole region compiles: on an RTX PRO 6000, variant n, COCO, batch 16, deterministic, the eager training step is 74.7 ms and the compiled one 60.9 ms, x1.23. A bf16 run was recorded at x1.46 on the same bench and is not enabled by this flag — precision is a separate decision the fidelity gate has not been asked about.

What it costs: 110–140 s of compilation per process before the first step, and a handful of recompiles over a run: the training graph, the first no_grad evaluation step, and — because Dynamo specialises sizes 0 and 1 whatever is marked — one each for the first batch with no ground truths (tal.py's own branch), the first with exactly one padded instance, and the first with one image, if the loader ever produces it. Measured on CPU with the eager backend over five batch shapes: four graphs. Batch size and padded instance count are marked dynamic on the images and the ground truths before every compiled call, so the last, smaller batch of an epoch and a batch with more instances than the one before share one graph; without the marks Dynamo would specialise until cache_size_limit (8) and then drop to eager without saying so, and with them the assigner's F.one_hot(best_gt, num_gt) — a concrete class count — had to become an arange comparison, since the enforcing marks refuse a pinned dim rather than silently recompiling. TORCH_LOGS=recompiles shows the count.

Where it refuses, it says so once at setup and trains eager. Off CUDA: the MPS Inductor backend crashes in convolution_backward, and CPU has nothing to gain. Under task: segment, obb or keypoints: their extra terms read tensors on the host inside _task_extra_loss, outside the region, so compiling the detection part alone would buy a fraction of the x1.23 for the same compile cost; they stay eager until their terms are made traceable. The compile wraps the bound method, not the module, so parameter names, the state dict, the EMA shadow, checkpoints and export are exactly what they are without the flag — a compiled module would prefix every key with _orig_mod..

The default is off on cost, not on doubt. A78 — whether the compiled objective trains as eager does — was validated on an A100 (paired 4-epoch run: 1 recompile, step 180.6 → 144.2 ms, loss parity 3.6e-5 with TF32 off), but compilation added 210 s of wall to a 4-epoch run of 313 steps, a break-even near 19 epochs at that size and under one epoch on COCO train; turn it on per recipe when the run is long enough. When comparing the two arms, compare mAP, or set torch.set_float32_matmul_precision("highest") first: under TF32 the per-batch train/loss differs by up to 1e-1, because the task-aligned assignment is discrete and TF32 rounding moves its top-k set. D24 carries the numbers.

Lifetime of the replay flags

Each opt-in replay flag is kept or retired on its own record, under the principal's rule P5 (D32). A flag stays while its make gate-gpu parity is green and its documented speed-up holds. A flag is retired when its parity stays red past one phase, or when its gain on its documented workload is under 3%. No flag becomes the default in 0.x without a principal call. Nothing on this page is removed by the table below: a flag whose stated gain is already under 3% would be marked as a retirement candidate for the principal, and none is, since every documented-workload figure is well above it. The batch-16 and batch-32 screens quoted elsewhere on this page (2.01% and 2.90% for segmentation, 2.50% for pose features, 4.10% for oriented) are exploratory boundary cases, not the documented workload, and do not trigger the rule.

A known failing CUDA test is a red main until it is fixed, and a [GPU] work package does not close without a green make gate-gpu run (decision 3 of the 2026-10-20 entry in ESCALATION). A retirement trigger is therefore a call for the principal, not an automatic removal.

Flag (task)	Work package	Documented workload and measured gain	Keep condition	Retire trigger
capture_network (detect)	WP-193c	Nano detection on one L4, torch 2.9.1+cu129: complete-fit median 92.589 s to 77.330 s, 16.48%.	make gate-gpu parity green; the gain holds on this workload.	Parity red past one phase, or gain under 3% on this workload.
capture_network (keypoints)	WP-193d, WP-193f	Feature replay (WP-193d), five pairs: complete-fit time falls 11.17%. Stem replay (WP-193f), nano/640/BF16, batch 4, L4, five pairs: complete runtime falls 12.97%.	Same.	Same.
capture_network (obb)	WP-200	Nano oriented, 104 DOTA-v1.0 tiles, eight epochs, batch 4, 1024 px, BF16, one L4, five pairs: complete-fit median drops 12.011% (68.642 s to 60.397 s).	Same.	Same.
capture_network (segment)	WP-193g	Nano segmentation, 512 COCO val2017 images, five epochs, batch 4, 640 px, BF16, L4, five pairs, prediction replay already on: complete fit 107.412 s to 87.629 s, 18.42%.	Same.	Same.
capture_loss (detect)	WP-199	Nano, batch 4, 640 px, BF16, five epochs on 1,024 COCO val2017 images, L4, five pairs: complete fit 125.272 s to 108.811 s, 13.140%.	Same.	Same.
capture_loss (keypoints)	WP-199b	Registered nano, batch 4, 640 px, BF16, five epochs over 384 COCO person images, L4, five pairs: 100.586 s to 87.058 s, 13.45% less complete-fit time.	Same.	Same.
capture_segment_heads (segment)	WP-193e	Nano segmentation, 384 training COCO val2017 images, 10 epochs, batch 4, 640 px, BF16, L4, five pairs: complete fit 202.794 s to 169.130 s, 16.60%.	Same.	Same.
capture_orthogonalization (MuSGD)	WP-197b	Nano detection, first 256 COCO val2017 images, six epochs, batch 4, 640 px, BF16, L4, five pairs: time saved 19.76% on the tree before compatible batching, 12.55% with compatible batching and device HSV applied (expected benefit on current main).	Same.	Same, read against the 12.55% row.
capture_updates (MuSGD)	WP-197c	Nano detection, first 256 COCO val2017 images, ten epochs, batch 4, 640 px, BF16, L4, five pairs: complete fit 120.850 s to 108.750 s, 10.01%.	Same.	Same.
compile_step (detect)	WP-184 (roadmap row, D24 in Decisions)	Measured x1.23 on the detect step, RTX PRO 6000, variant n, COCO, batch 16; 110–140 s compile per process. WP-184 has no page of its own, so this is the roadmap row's figure, a step-time ratio rather than a complete-fit time.	Same; the compile cost is paid per process.	Same, with the step-time ratio converted to a complete-fit gain.

Every row's "gain" is the figure its page states for the workload shown, on one device; none is a claim for another batch size, device or full-COCO run. The pose-feature row is not a complete-fit figure for the stem replay alone: WP-193f measures stem replay against feature replay already enabled.