🔎 Inference¶
Turning a trained model's raw outputs into detections: one-call prediction for a single image per task, and the decoders behind it, NMS-free top-k beside the NMS baseline. Generated from the docstrings in src/lucid_yolo/predict/ and src/lucid_yolo/decode/.
Prediction¶
predict_image
¶
predict_image(module: DetectionLitModule, image: Path, img_size: int = 640, decoder: DecodePath = 'e2e', conf_threshold: float = DEFAULT_CONF_THRESHOLD, device: device | None = None) -> Tensor
Detect objects in one image file and return them in original-image coordinates.
The image is letterboxed into an img_size square canvas, run through the module
once, decoded by the selected path, and mapped back onto the original pixel grid by
the exact analytic inverse of the letterbox that produced the canvas (A10). Rows
below conf_threshold are dropped, which also removes the score-zero padding rows
both decoders emit to keep their output shape fixed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
module
|
DetectionLitModule
|
An eval-mode |
required |
image
|
Path
|
Path to the image file to read. |
required |
img_size
|
int
|
Letterbox side the model sees. Defaults to |
640
|
decoder
|
DecodePath
|
|
'e2e'
|
conf_threshold
|
float
|
Detections at or below this score are dropped. Defaults to
:data: |
DEFAULT_CONF_THRESHOLD
|
device
|
device | None
|
Device to run on. Defaults to CPU; the command resolves |
None
|
Returns:
| Type | Description |
|---|---|
Tensor
|
A CPU tensor of shape |
Tensor
|
|
Tensor
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If the module's task is not |
Examples:
predict_segmentation
¶
predict_segmentation(module: DetectionLitModule, image: Path, img_size: int = 640, decoder: DecodePath = 'e2e', conf_threshold: float = DEFAULT_CONF_THRESHOLD, device: device | None = None) -> SegmentedPrediction
Segment objects in one image file and return boxes and masks in original coordinates.
:func:predict_image plus Eq. 7, in the order
:class:~lucid_yolo.eval.coco_eval.DualPathEvaluator established and for its reason:
the model runs once, the selected path decodes boxes and reports the anchor each
row came from, each surviving detection's mask coefficients are gathered by that
index, the masks are assembled while the boxes are still on the letterboxed canvas
— which is the frame
:func:~lucid_yolo.eval.segment_decode.decode_instance_masks crops in — and only
then are boxes and masks each landed on the original pixel grid. Cropping after the
inverse letterbox would leave every box right and every mask quietly wrong.
Rows below conf_threshold are dropped before the masks are assembled, which
is where this differs from the evaluator: the evaluator needs a fixed-length output
per image and pays for a mask per padding row, while a caller looking at one picture
wants a mask per object. At 640 px the difference is 300 canvas-resolution masks
instead of a handful, which is measured in hundreds of megabytes of transient float.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
module
|
DetectionLitModule
|
An eval-mode |
required |
image
|
Path
|
Path to the image file to read. |
required |
img_size
|
int
|
Letterbox side the model sees. Defaults to |
640
|
decoder
|
DecodePath
|
|
'e2e'
|
conf_threshold
|
float
|
Detections at or below this score are dropped, with their masks.
Defaults to :data: |
DEFAULT_CONF_THRESHOLD
|
device
|
device | None
|
Device to run on. Defaults to CPU; the command resolves |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
A |
SegmentedPrediction
|
class: |
SegmentedPrediction
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If the module's task is not |
Examples:
predict_oriented
¶
predict_oriented(module: DetectionLitModule, image: Path, img_size: int = DEFAULT_ORIENTED_IMG_SIZE, decoder: DecodePath = ORIENTED_DECODE_PATH, conf_threshold: float = DEFAULT_CONF_THRESHOLD, device: device | None = None) -> Tensor
Detect oriented objects in one image file and return them in original coordinates.
:func:predict_image's shape with R1 Eq. 13 in it, and the same pieces the per-tile
oriented evaluator (:mod:lucid_yolo.eval.rotated_eval) is built from rather than a
second reading of them: :func:~lucid_yolo.models.heads.obb.decode_rboxes turns the
one-to-one branch's ltrb distances and raw angles into canonical rotated boxes,
:func:~lucid_yolo.models.heads.obb.o2o_rotated_topk ranks them, and
:func:~lucid_yolo.decode.common.rboxes_to_letterboxed_original lands them on the
original pixel grid. Nothing here computes a pad, a ratio or a corner.
The column convention is A45, the one the oriented evaluator and
:func:~lucid_yolo.eval.dota_eval.rotated_detections_to_predictions already consume:
[cx, cy, w, h, theta, score, class], which is the long-edge rotated box of
:mod:lucid_yolo.data.rotated_geom followed by the two columns the A9 tuple ends
with. It is not the A9 tuple with an angle appended — the four box columns are a
centre and two extents, not corners — so a caller that slices [:, :4] expecting
xyxy gets a plausible, wrong rectangle. :data:~lucid_yolo.decode.common.RBOX_COLUMNS
names where the box columns end and the score begins.
Canonicalization is post-decode, and that is the right seam (A23). Two places
could hold it and both are worse. In the head, it would normalize an angle the loss
is still training against a raw target, and R1 Eq. 13 explicitly emits the
pre-activation. Here, after the letterbox inverse, it would run once per entry point
— this one, the evaluator, an export graph — and each copy would be free to drift.
Inside :func:~lucid_yolo.models.heads.obb.decode_rboxes it runs on the dense
(B, A, 5) output, so every consumer of a decoded rotated box inherits w >= h
and theta in [-pi/4, 3*pi/4) unconditionally, including one that never ranks
and never letterboxes. What this function then adds is the proof that the guarantee
survives the trip home: a letterbox inverse is one isotropic scale plus a translation,
which scales both extents by the same positive number and turns no angle, so the
canonical form is invariant under it and is not recomputed here.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
module
|
DetectionLitModule
|
An eval-mode |
required |
image
|
Path
|
Path to the image file to read. |
required |
img_size
|
int
|
Letterbox side the model sees. Defaults to
:data: |
DEFAULT_ORIENTED_IMG_SIZE
|
decoder
|
DecodePath
|
|
ORIENTED_DECODE_PATH
|
conf_threshold
|
float
|
Detections at or below this score are dropped. Defaults to
:data: |
DEFAULT_CONF_THRESHOLD
|
device
|
device | None
|
Device to run on. Defaults to CPU; the command resolves |
None
|
Returns:
| Type | Description |
|---|---|
Tensor
|
A CPU tensor of shape |
Tensor
|
|
Tensor
|
score-descending, canonical ( |
Tensor
|
measured from |
Tensor
|
contiguous class index (see the module docstring). |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the module's task is not |
Examples:
predict_keypoints
¶
predict_keypoints(module: DetectionLitModule, image: Path, img_size: int = 640, decoder: DecodePath = 'e2e', conf_threshold: float = DEFAULT_CONF_THRESHOLD, device: device | None = None) -> KeypointPrediction
Locate objects and their points in one image file, in original-image coordinates.
:func:predict_image plus the point branch, composed the way
:func:predict_segmentation composes the mask branch and for the same reason: the
point stem is dense over anchors, so the selected path must report the anchor
each surviving row came from and the points are gathered by that index. The
decode-then-gather order is :class:~lucid_yolo.export.KeypointExportGraph's and
the training module's own val/oks_mAP's, so all three read one composition.
Points are mapped back with :meth:~lucid_yolo.data.letterbox.Letterbox.inverse_map
-- the same exact analytic inverse the boxes take (A10), applied to the point axis
flattened into a plain point list, so a point and the box around it cannot land on
two different grids.
K is read from the decoded tensor, never assumed. The head's point count is a
constructor argument (A64) and COCO's 17 is one instantiation of it; a function that
assumed 17 would silently mis-shape every other schema.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
module
|
DetectionLitModule
|
An eval-mode |
required |
image
|
Path
|
Path to the image file to read. |
required |
img_size
|
int
|
Letterbox side the model sees. Defaults to |
640
|
decoder
|
DecodePath
|
|
'e2e'
|
conf_threshold
|
float
|
Detections at or below this score are dropped, with their
points. Defaults to :data: |
DEFAULT_CONF_THRESHOLD
|
device
|
device | None
|
Device to run on. Defaults to CPU; the command resolves |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
A |
KeypointPrediction
|
class: |
KeypointPrediction
|
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If the module's task is not |
Examples:
Decoders¶
TopKDecoder
¶
Bases: Module
Score-based top-k end-to-end decoder for the one-to-one branch (A9).
Turns the one-to-one branch's raw dense outputs into a fixed-size batch of
score-ranked detections without any IoU computation or non-maximum
suppression (R3 sec. 4). The module owns no parameters — it is a pure
functional transform wrapped as an :class:~torch.nn.Module so it composes
into an inference graph and traces/exports cleanly with a static output
shape.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
k
|
int
|
Maximum detections kept per image; also the fixed output length. Defaults to 300 (R1 sec. 3.2.1, R3 sec. 4). |
_DEFAULT_TOPK
|
conf_threshold
|
float
|
Detections whose confidence is strictly below this value
have their score zeroed (the row is kept so the output shape is
fixed). Defaults to |
0.0
|
Examples:
>>> import torch
>>> from lucid_yolo.assign.grid import make_anchor_points
>>> points, strides = make_anchor_points([(2, 2)], [8])
>>> cls_logits = torch.full((1, 4, 3), -5.0)
>>> cls_logits[0, 1, 2] = 5.0 # one confident anchor, class 2
>>> raw_ltrb = torch.ones(1, 4, 4)
>>> decoder = TopKDecoder(k=2)
>>> detections = decoder(cls_logits, raw_ltrb, points, strides)
>>> detections.shape # fixed (B, k, 6) even though only 4 anchors exist
torch.Size([1, 2, 6])
>>> int(detections[0, 0, 5]) # class of the top detection
2
forward
¶
Decode raw one-to-one outputs into fixed-size top-k detections.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cls_logits
|
Tensor
|
Raw one-to-one class logits of shape |
required |
raw_ltrb
|
Tensor
|
Raw one-to-one ltrb distances of shape |
required |
anchor_points
|
Tensor
|
Anchor-centre |
required |
strides
|
Tensor
|
Per-anchor level stride of shape |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
Detections of shape |
Tensor
|
|
Tensor
|
score-zero padding rows filling any shortfall below |
Examples:
decode_with_indices
¶
decode_with_indices(cls_logits: Tensor, raw_ltrb: Tensor, anchor_points: Tensor, strides: Tensor) -> tuple[Tensor, Tensor]
Decode as :meth:forward does, and report the source anchor of each row.
The whole decode lives here and :meth:forward is its box-only view, so
the two cannot report different selections. The extra return is what any
per-anchor quantity outside the A9 tuple must be gathered by — the
segmentation decode's mask coefficients (WP-053b): a coefficient row taken
from a differently ranked selection would yield a plausible mask of the
wrong object beside a correct box.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cls_logits
|
Tensor
|
Raw one-to-one class logits of shape |
required |
raw_ltrb
|
Tensor
|
Raw one-to-one ltrb distances of shape |
required |
anchor_points
|
Tensor
|
Anchor-centre |
required |
strides
|
Tensor
|
Per-anchor level stride of shape |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
A pair |
Tensor
|
meth: |
tuple[Tensor, Tensor]
|
anchor each row came from, with |
tuple[Tensor, Tensor]
|
data: |
Examples:
>>> import torch
>>> from lucid_yolo.assign.grid import make_anchor_points
>>> points, strides = make_anchor_points([(1, 2)], [8])
>>> cls_logits = torch.tensor([[[-5.0], [5.0]]]) # anchor 1 is the confident one
>>> raw_ltrb = torch.zeros(1, 2, 4)
>>> detections, anchors = TopKDecoder(k=3).decode_with_indices(
... cls_logits, raw_ltrb, points, strides
... )
>>> detections.shape
torch.Size([1, 3, 6])
>>> anchors # anchor 1 ranks first; the third row is padding
tensor([[ 1, 0, -1]])
NMSDecoder
¶
Bases: Module
Confidence-threshold + class-wise NMS decoder for the dense branch (A9).
Turns the one-to-many branch's raw dense outputs into a fixed-size batch of
score-ranked detections via a confidence threshold and class-wise
non-maximum suppression (the non-E2E path, blueprint sec. 5.5). Emits the
same (B, max_det, 6) A9 tuple as
:class:~lucid_yolo.decode.topk_e2e.TopKDecoder so both paths feed one
evaluator. The module owns no parameters — it is a pure functional transform
wrapped as an :class:~torch.nn.Module.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
conf_threshold
|
float
|
Anchors whose best-class score is strictly below this
value are dropped before NMS. Defaults to |
_DEFAULT_CONF_THRESHOLD
|
iou_threshold
|
float
|
Class-wise NMS IoU threshold; a lower-scoring box is
suppressed when its IoU with a kept higher-scoring box of the same
class exceeds this. Defaults to |
_DEFAULT_IOU_THRESHOLD
|
max_det
|
int
|
Maximum detections kept per image; also the fixed output length.
Defaults to |
_DEFAULT_MAX_DET
|
Examples:
>>> import torch
>>> from lucid_yolo.assign.grid import make_anchor_points
>>> points, strides = make_anchor_points([(2, 2)], [8])
>>> cls_logits = torch.full((1, 4, 3), -5.0)
>>> cls_logits[0, 1, 2] = 5.0 # one confident anchor, class 2
>>> raw_ltrb = torch.ones(1, 4, 4)
>>> decoder = NMSDecoder(max_det=100)
>>> detections = decoder(cls_logits, raw_ltrb, points, strides)
>>> detections.shape # fixed (B, max_det, 6)
torch.Size([1, 100, 6])
>>> int(detections[0, 0, 5]) # class of the top detection
2
forward
¶
Decode raw dense outputs into fixed-size NMS-filtered detections.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cls_logits
|
Tensor
|
Raw dense class logits of shape |
required |
raw_ltrb
|
Tensor
|
Raw dense ltrb distances of shape |
required |
anchor_points
|
Tensor
|
Anchor-centre |
required |
strides
|
Tensor
|
Per-anchor level stride of shape |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
Detections of shape |
Tensor
|
tuple |
Tensor
|
with score-zero padding rows filling any shortfall below |
Examples:
decode_with_indices
¶
decode_with_indices(cls_logits: Tensor, raw_ltrb: Tensor, anchor_points: Tensor, strides: Tensor) -> tuple[Tensor, Tensor]
Decode as :meth:forward does, and report the source anchor of each row.
The whole decode lives here and :meth:forward is its box-only view, so
the two cannot report different survivor sets. The extra return exists for
the same reason as its one-to-one twin
(:meth:~lucid_yolo.decode.topk_e2e.TopKDecoder.decode_with_indices): the
segmentation decode (WP-053b) gathers each detection's mask coefficients
from the dense (B, A, K) map by these indices. On this path the kept
set is chosen by the threshold and the suppression, not by a ranking any
caller could reproduce, so the indices have to be carried out through the
same keep/order selection that built the boxes — recomputing a
second selection outside is exactly how a mask ends up describing another
anchor's object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cls_logits
|
Tensor
|
Raw dense class logits of shape |
required |
raw_ltrb
|
Tensor
|
Raw dense ltrb distances of shape |
required |
anchor_points
|
Tensor
|
Anchor-centre |
required |
strides
|
Tensor
|
Per-anchor level stride of shape |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Tensor
|
A pair |
|
batch |
Tensor
|
meth: |
tuple[Tensor, Tensor]
|
naming the anchor each row came from, with |
|
tuple[Tensor, Tensor]
|
data: |
Examples:
>>> import torch
>>> from lucid_yolo.assign.grid import make_anchor_points
>>> points, strides = make_anchor_points([(1, 2)], [8])
>>> cls_logits = torch.tensor([[[-5.0], [5.0]]]) # only anchor 1 clears the threshold
>>> raw_ltrb = torch.full((1, 2, 4), 0.25) # a real 4 px box on each anchor
>>> decoder = NMSDecoder(conf_threshold=0.5, max_det=3)
>>> detections, anchors = decoder.decode_with_indices(cls_logits, raw_ltrb, points, strides)
>>> detections.shape
torch.Size([1, 3, 6])
>>> anchors # anchor 1 survives; the remaining rows are padding
tensor([[ 1, -1, -1]])
A box has to enclose an area to be a detection: zero ltrb distances put
x2 exactly on x1, and such a row is dropped with the sub-threshold
ones rather than spending one of the max_det slots.
>>> flat = torch.zeros(1, 2, 4)
>>> _, none_kept = decoder.decode_with_indices(cls_logits, flat, points, strides)
>>> none_kept
tensor([[-1, -1, -1]])
An empty batch decodes to empty outputs of the documented width rather
than raising, matching :class:~lucid_yolo.decode.topk_e2e.TopKDecoder:
RotatedNMSDecoder
¶
Bases: Module
Confidence-threshold + class-wise rotated-NMS decoder for the dense branch (A45, A61).
Turns the one-to-many branch's dense oriented outputs into a fixed-size batch of
score-ranked oriented detections, suppressing by exact rotated overlap rather than by
the overlap of the boxes' upright envelopes. Emits the same (B, max_det, 7) A45
tuple as :func:~lucid_yolo.models.heads.obb.o2o_rotated_topk, so the suppression
column and the suppression-free column are the same kind of number. The module owns no
parameters — it is a pure functional transform wrapped as an
:class:~torch.nn.Module.
This is the o2m path's decoder and a comparison baseline. It is not a repair to the one-to-one path, whose freedom from NMS is the architecture's claim rather than a limitation to work around.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
conf_threshold
|
float
|
Anchors whose best-class score is strictly below this value are
dropped before suppression. Defaults to |
_DEFAULT_CONF_THRESHOLD
|
iou_threshold
|
float
|
Rotated-IoU threshold; a lower-scoring box is suppressed when its
rotated IoU with a kept higher-scoring box of the same class exceeds this.
Defaults to :data: |
ROTATED_NMS_IOU_THRESHOLD
|
max_det
|
int
|
Maximum detections kept per image; also the fixed output length. Defaults
to |
_DEFAULT_MAX_DET
|
Examples:
>>> import torch
>>> from lucid_yolo.assign.grid import make_anchor_points
>>> points, strides = make_anchor_points([(2, 2)], [8])
>>> cls_logits = torch.full((1, 4, 3), -5.0)
>>> cls_logits[0, 1, 2] = 5.0 # one confident anchor, class 2
>>> raw_ltrb = torch.ones(1, 4, 4)
>>> angles = torch.zeros(1, 4, 1)
>>> decoder = RotatedNMSDecoder(max_det=100)
>>> detections = decoder(cls_logits, raw_ltrb, angles, points, strides)
>>> detections.shape # fixed (B, max_det, 7)
torch.Size([1, 100, 7])
>>> int(detections[0, 0, 6]) # class of the top detection
2
forward
¶
forward(cls_logits: Tensor, raw_ltrb: Tensor, angles: Tensor, anchor_points: Tensor, strides: Tensor) -> Tensor
Decode raw dense oriented outputs into fixed-size rotated-NMS-filtered detections.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cls_logits
|
Tensor
|
Raw dense class logits of shape |
required |
raw_ltrb
|
Tensor
|
Raw dense ltrb distances of shape |
required |
angles
|
Tensor
|
Raw dense orientation angles in radians, shape |
required |
anchor_points
|
Tensor
|
Anchor-centre |
required |
strides
|
Tensor
|
Per-anchor level stride of shape |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
Detections of shape |
Tensor
|
|
Tensor
|
|
Tensor
|
filling any shortfall below |
Examples:
>>> import torch
>>> from lucid_yolo.assign.grid import make_anchor_points
>>> points, strides = make_anchor_points([(1, 2)], [8])
>>> cls_logits = torch.zeros(1, 2, 1)
>>> raw_ltrb = torch.zeros(1, 2, 4)
>>> angles = torch.zeros(1, 2, 1)
>>> RotatedNMSDecoder(max_det=5)(cls_logits, raw_ltrb, angles, points, strides).shape
torch.Size([1, 5, 7])
>>> # Nothing above the threshold still returns the fixed shape, all padding.
>>> empty = RotatedNMSDecoder(conf_threshold=0.99, max_det=2)
>>> empty(cls_logits, raw_ltrb, angles, points, strides)
tensor([[[0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0.]]])
>>> # An empty batch decodes to an empty result of the documented width.
>>> zero = torch.zeros(0, 2, 1)
>>> empty(zero, torch.zeros(0, 2, 4), zero, points, strides).shape
torch.Size([0, 2, 7])