WP-205 — task registry and module split

Engineering log · Phase 19 — Codebase audit remediation

🔗 Depends on WP-203 · Needed by WP-206

⬅️ WP-204 · Roadmap · WP-206 ➡️

WP-205 — one task registry, three oversized files split by responsibility, and the small duplications removed, consolidated in one commit

Batch 5 of the codebase-audit roadmap, the CPU part only. The audit of 2026-10-21 found the closed task set detect|segment|obb|keypoints restated as name constants and per-task tables in five modules, ptl/module.py (3042 lines), ptl/datamodule.py (2065) and eval/coco_eval.py (1763) each holding several responsibilities, and a handful of local duplications inside them. The gate is make gate, as for every package (AGENTS.md sec. 1, D30); it will be run before the commit is made, and this page claims no result for it.

Task registry. lucid_yolo/tasks.py is a leaf module (standard library only; a fresh interpreter importing it loads neither torch nor numpy). Task is a StrEnum of the four tasks in the order DetectionLitModule already listed them, and Task.spec returns a frozen TaskSpec holding the default image side (640, and 1024 for oriented boxes) and the default lucid-eval batch size (32, and 8 for oriented boxes). StrEnum rather than a plain (str, Enum) because on Python 3.11 the latter formats as Task.DETECT, which would poison any f-string a leaked member reached. The tables that used to restate these facts now derive from the registry: DEFAULT_IMG_SIZE and DEFAULT_BATCH_SIZE in cli/eval.py (same keys, values and key order, oriented first, because two refusal messages print that order), DEFAULT_ORIENTED_IMG_SIZE and the four _*_TASK constants and entry-point table in predict.py, the three constants in cli/predict.py, the renderer and ground-truth tables and the image-size lookup in _viz/overlay.py (which no longer imports a CLI module), and _TASKS in ptl/module.py. The plain string stays at every boundary. hparams["task"] stays a str in the saved checkpoint, including when a subclass forwards a Task member to DetectionLitModule.__init__: Lightning lets a subclass's constructor locals overwrite the base's, so a normalisation placed before save_hyperparameters() was undone, and the member then failed torch.load(..., weights_only=True). The base class now rewrites the recorded value and retakes the initial snapshot after the call (_record_plain_task), and six tests cover the named-parameter and **kwargs forms. The residual: a subclass that calls save_hyperparameters() again itself, after super().__init__(), recollects its own frame and can record the member; the base constructor cannot reach that call.

ptl/module.py. The padding and batch helpers (pad_targets, pad_rboxes, pad_keypoints, StepBatch, _host_split, _split_batch and their constants) moved to ptl/_padding.py, and the module-level keypoint-loss helpers (normalize_keypoints_to_box, _StepContext, _KeypointPositives, the OKS sigma lookup) to ptl/_task_terms.py; each was sliced verbatim and re-imported, __all__ is unchanged, and ptl/__init__.py imports normalize_keypoints_to_box from its new home because mypy strict does not follow an implicit re-export. Two local simplifications: _pad_instance_axis replaces the five zero-then-fill loops over boxes, labels, oriented boxes, keypoint coordinates and visibility (the if count guard stays, since assigning an empty (0, 0, 2) keypoint set into a (0, K, 2) slot raises), and _blend replaces the expression alpha * o2m + (1.0 - alpha) * o2o at eight sites, written verbatim in the helper. setup and validation_step were shortened by extracting private helpers (about 99 and 74 lines before, 64 and 61 after, each measured from its def to the next); the capture-ignored warning keeps its order relative to the activation flag and its stacklevel, both pinned by tests. The per-task loss-term methods stay on the class: replay certification compares method identity (_ORIENTED_METHODS, _EAGER_POSE_SCORING) and _oriented_helpers() compares the module's own bindings of decode_ltrb, decode_rboxes and oriented_branch_terms, which three tests patch, so moving the callers would turn those patches into no-ops. They belong with the replay consolidation.

ptl/datamodule.py. The IPC transport (PackedTargets, pack_targets, unpack_*, the collate functions, unpack_batch) moved to ptl/_transport.py and the worker pipeline (_DetectionSource, _init_worker, _PreparedTrainingSource, _TrainPipeline) to ptl/_pipeline.py; datamodule imports from both and re-exports every name a test or script reads, so lucid_yolo.ptl.PackedTargets is the same object as before. Names the datamodule no longer looks up (_jitter_hsv_batch, get_worker_info) are deliberately not re-exported, so a stale patch on datamodule raises AttributeError instead of passing silently; the tests that patched them now target the module that looks them up. The pack_targets docstring said eleven dense tensors and the dataclass has twelve without a default (DP-21); it now says twelve and names the two optional fields the collate fills. Targets._shared_keypoint_count is the one place a batch's keypoint count K is read and checked; Targets._concat_keypoints and the transport's _concat_keypoints both call it. Behaviour change: a batch whose images carry different K now raises ValueError naming both counts from pack_targets and collate_detection; it used to raise RuntimeError from torch.cat. No test pinned the old type, and the output for every input the old code accepted is unchanged. pad_keypoints in the padding module still carries its own check.

eval/coco_eval.py. The split carried over from WP-204 landed: the streaming scorers and the hotcoco path moved to eval/_coco_streaming.py, the OKS keypoint scoring to eval/_coco_keypoints.py, by line-range slices checked contiguous and unduplicated. coco_eval keeps the bbox and segm functions and the orchestrator, __all__ is byte-identical, and the names tests and callers read are re-exported. MeanAveragePrecision is deliberately not re-exported: nothing in coco_eval uses it after the move, so a patch of coco_eval.MeanAveragePrecision fails loudly (it did, on the unretargeted test) rather than becoming a silent no-op; that test now patches _coco_streaming. gather_keypoints stays in coco_eval: it is a decode gather delegating to gather_by_anchor, not keypoint scoring, and leaving it keeps the references in the keypoints model card, two docstrings and a patch site exact. docs/report/keypoints.md now points evaluate_keypoints at eval/_coco_keypoints.py.

Size. Line counts at HEAD and in the working tree, from git show HEAD:<file> | wc -l and wc -l:

File	Before	After
ptl/module.py	3042	2697
ptl/datamodule.py	2065	1234
eval/coco_eval.py	1763	873
ptl/_padding.py	new	337
ptl/_task_terms.py	new	238
ptl/_transport.py	new	588
ptl/_pipeline.py	new	323
eval/_coco_streaming.py	new	522
eval/_coco_keypoints.py	new	451
tasks.py	new	132
data/targets.py	523	564

The three split files lose 3,636 lines between them and gain 2,458 in the six new modules and 41 in targets.py (the helper and its doctest); headers and import blocks account for most of the difference from a pure move.

Verification the batch's agents recorded. Each moved block was compared against the original rather than trusted: the padding and transport helpers and the coco_eval blocks by script against the pre-change text, _pad_instance_axis and _blend by old-against-new execution (160 random ragged batches torch.equal for the padding, and a two-image training step with every loss term and every parameter gradient torch.equal for detect, segment, oriented and keypoints, including the paired pose path). make golden ran 50/50 on the final trees of the module and datamodule steps. Every patch site on a moved name was enumerated by grep, and the retargeted ones were shown to fail when their guard was disabled.

Coverage (T8). The audit measured CI coverage at 94.65% against fail_under = 96. The local gate at the previous commit measured 96.56%. This batch adds regression tests for what it changed and no tests aimed at coverage, and it moves code between files without adding branches, so no change is expected; the figure on CI's ubuntu cell has not been measured for this tree and is not verified without a push. The batch does not claim to have closed T8.

Limits and deferred.

• Batch 5b is not part of this row. The replay consolidation (C-REPLAY, with the guarded lifecycle D5 and the stream-drain contract D6, and the P5 per-flag criteria) needs the principal's answer on gradient accumulation across the replay flags (the 2026-10-20 entry in ESCALATION) and CUDA parity runs. It has no number yet.
• Moving the capture tests (T6) to a tests/ptl/capture/ directory waits on the same consolidation.
• C-TASKS step 2 is not done. The per-task strategy objects, and with them the task == "obb" style comparisons that remain in ptl/module.py, cli/eval.py, cli/train.py and export.py, stay as they were.
• A fifth task is not yet one registry row. lucid-eval derives its size and batch tables from the registry, so a fifth Task member would be accepted there and reach the final dispatch branch, detect_eval.run, which has no task guard and would score it as detection. No current input triggers it; a test pins exactly four members and its docstring lists every site a fifth task must extend. A guard in cli/eval.py belongs with step 2.
• lucid_yolo.tasks is not documented. It has no docs/api entry and is not re-exported from the top-level package, so this batch adds no CHANGELOG line for it.
• Loss-input records (CS-17, DP-18) are batch 6, and decode/common.COCO_IMG_SIZE, models/build._DEFAULT_IMG_SIZE and the restated subset in eval/coco_score.py still hold their own copies of the registry's facts, pinned equal by tests where one exists.
• The oriented detections.cpu() copy stays in validation_step. preds is still built under the obb task, where val/mAP is never updated, to keep behaviour identical; removing the device-to-host copy is a performance change for a later batch.
• Not run on this host. The two fused-HSV tests whose patches were retargeted to _transport (test_literal_pixels_and_single_restore, test_transfer_hook_uses_fusion_and_keeps_targets) need CUDA and Triton and were only exercised through a CPU stand-in, and no CUDA-marked capture test ran (243 skips in the module step's run), so make gate-gpu is the run that covers the replay paths and those two tests.
• Moved docstrings keep their relative cross-references. A few Sphinx roles in the new private modules name a sibling that no longer resolves module-locally; none of those modules is rendered on a docs/api page.

No frozen golden is touched.