Types and targets¶

The high-level data_keys interface keeps supported auxiliary targets on the same sampled geometric path as the image. Low-level target functions are also public for applying a known matrix or sampling grid directly.

Data keys¶

data_keys describes positional arguments. The first entry must be "input".

Key	Expected shape	Output behavior
`input`	`(B, C, H, W)`	Image transformed by every segment
`mask`	`(B, C, H, W)`	Nearest-neighbor by default; bilinear for floating soft masks
`bbox_xyxy`	`(B, N, 4)`	Transforms corners and returns an enclosing axis-aligned box
`bbox_xywh`	`(B, N, 4)`	Converts to/from xyxy internally
`keypoints`	`(B, N, 2)`	Applies the forward homogeneous pixel matrix

Unknown keys are passed through with a warning. Duplicate auxiliary keys are rejected.

Mask interpolation¶

Nearest-neighbor sampling preserves discrete class values, but the package deliberately detaches the entire nearest-mask path. Its output has no autograd path with respect to either mask values or the sampling grid. This is package policy; direct PyTorch nearest grid_sample can propagate gradients to floating input values even though nearest coordinates are nondifferentiable almost everywhere. mask_interpolation="bilinear" supports differentiable soft masks and requires floating-point mask input; class values mix at boundaries.

Coordinate-changing passthrough¶

A passthrough transform normally changes only the image. That is safe only for an operation known to preserve coordinates, such as a verified blur or pointwise color transform. When auxiliary targets are present, the pipeline raises for passthroughs whose exact class name appears on its finite coordinate-changing refusal list.

Unknown spatial transforms are not structurally detected. Unsupported crops, resizes, and custom callables can still modify the image while targets remain unchanged. Treat every Unknown ... SPATIAL_KERNEL barrier warning as unsafe with data_keys; use a registered transform or route all targets through a native target-aware pipeline.

Output conversion¶

With output_backend="numpy", image and mask targets become channel-last NumPy arrays. Boxes and keypoints stay as tensors because image channel layout does not apply to coordinate arrays.

Segment descriptors¶

fusion_plan_descriptors is available before the first forward call. Each immutable descriptor exposes the segment kind, transform names, saved-application count, backend name where available, and structured boundary/refusal reasons.

SegmentDescriptor `dataclass` ¶

SegmentDescriptor(kind: SegmentKind, transforms: tuple[str, ...], n_warps_saved: int, backend: str | None = None, barrier: str | None = None, split_reason: str | None = None, refused: str | None = None)

Structured description of one segment in a fused augmentation pipeline.

Returned by :attr:FusedCompose.fusion_plan_descriptors <fuse_augmentations.compose.FusedCompose.fusion_plan_descriptors>. Each instance describes exactly one segment — a fused geometric group, a lossless exact segment, a projective segment, or a passthrough barrier — and is frozen and JSON-serialisable via :meth:to_dict.

Parameters:

Name	Type	Description	Default
`kind`	`SegmentKind`	Segment type. One of `"fused"`, `"exact"`, `"projective"`, `"color"`, `"crop_resize"`, or `"passthrough"`.	required
`transforms`	`tuple[str, ...]`	Class names of the transforms in this segment, in execution order.	required
`n_warps_saved`	`int`	Number of `grid_sample` interpolation passes eliminated by fusing this segment. Zero for passthrough and single-transform segments.	required
`backend`	`str \| None`	Adapter class name used for this segment (for example `"KorniaAdapter"`, `"AlbumentationsAdapter"`, `"TorchVisionAdapter"`), or `None` for backend-free pipelines created via :meth:`FusedCompose.from_params <fuse_augmentations.compose.FusedCompose.from_params>`.	`None`
`barrier`	`str \| None`	Machine-readable reason this segment ends a fusion run and forces a boundary, or `None` when the segment is fused/does not act as a barrier. `"spatial_kernel"` for blur/noise passthrough, `"pointwise"` for non-linear pixel passthrough, `"coordinate_change"` for geometric distortion passthrough (elastic/grid/optical), `"crop_resize"` for a :class:`CropResizeSegment`. This is the structured counterpart to the human-readable :attr:`fusion_plan` string.	`None`
`split_reason`	`str \| None`	Machine-readable reason a run that could otherwise fuse was split at this segment, or `None`. Currently `"backend_boundary"` when a mixed-backend group break created the split, else `None`.	`None`
`refused`	`str \| None`	Machine-readable reason an op stayed on the passthrough path instead of being fused, or `None` for fused segments. `"not_fusible"` for ops with no fusion representation (default for passthrough), or `"substitution_unavailable"` when substitution was requested but the target backend was not importable.	`None`

Examples:

>>> d = SegmentDescriptor(
...     kind="fused",
...     transforms=("RandomRotation", "RandomHorizontalFlip"),
...     n_warps_saved=1,
...     backend="KorniaAdapter",
... )
>>> d.kind
'fused'
>>> d.n_warps_saved
1
>>> d.barrier is None
True
>>> d.to_dict()  # doctest: +SKIP
{'kind': 'fused', 'transforms': ['RandomRotation', ...], ...}

to_dict ¶

to_dict() -> dict[str, object]

Return a JSON-serialisable dict representation of this descriptor.

Returns:

Type	Description
`dict[str, object]`	Dict with keys `"kind"`, `"transforms"`, `"n_warps_saved"`,
`dict[str, object]`	`"backend"`, `"barrier"`, `"split_reason"`, and `"refused"`.
`dict[str, object]`	The `"transforms"` value is a `list` of strings (not a `tuple`)
`dict[str, object]`	for JSON compatibility.

Examples:

>>> d = SegmentDescriptor(
...     kind="passthrough",
...     transforms=("RandomGaussianBlur",),
...     n_warps_saved=0,
...     backend="kornia",
...     barrier="spatial_kernel",
...     refused="not_fusible",
... )
>>> d.to_dict()["barrier"]
'spatial_kernel'
>>> d.to_dict()["refused"]
'not_fusible'

Transform categories¶

TransformCategory ¶

Bases: Enum

Category of an augmentation transform for fusion classification.

Attributes:

Name	Type	Description
`GEOMETRIC_INTERP`		Fusible geometric op requiring interpolation (rotate, scale, shear).
`GEOMETRIC_EXACT`		Fusible only when INTERP is present; lossless alone (flip, 90-deg rot).
`POINTWISE`		Reorderable per-pixel op; not fusible (color jitter, normalize).
`SPATIAL_KERNEL`		Barrier; not fusible and not reorderable (blur, noise, erase).
`PROJECTIVE`		Fusible projective (perspective) op requiring full 3x3 homography.
`POINTWISE_LINEAR`		Reorderable per-pixel linear op; self-fusible as 4x4 color-space affine matrix (brightness, contrast, channel mix). Consecutive runs are fused into a single `FusedColorSegment` via `build_color_matrix`; adapters that do not support `build_color_matrix` for a given transform fall back to passthrough.
`POINTWISE_LUT`		Reorderable per-pixel non-linear scalar op whose per-channel intensity map is a pure lookup (gamma, solarize, posterize). Consecutive runs are composed into a single per-channel lookup table via `build_lut` and applied once by :class:`~fuse_augmentations.affine.segment.FusedLUTSegment`; adapters that do not support `build_lut` for a given transform fall back to passthrough. Distinct from `POINTWISE_LINEAR` (which composes into a colour matrix) and from `POINTWISE` (cross-channel ops such as saturation/hue that never fuse into a lookup table).
`CROP_RESIZE_FIXED`		Fixed-output-size crop followed by resize. Each op produces a `CropResizeSegment` with a single `grid_sample` call at the target `(H, W)` dimensions. The output shape differs from the input shape.
`SPATIAL_LINEAR`		A Gaussian blur that can fold with adjacent Gaussian blurs and, after a safe axis-aligned upscale, move to the end of an affine chain.

Sampling enums¶

InterpolationMode ¶

Bases: IntEnum

Interpolation modes ordered by quality (higher = finer).

Examples:

>>> InterpolationMode.BICUBIC > InterpolationMode.BILINEAR
True

PaddingMode ¶

Bases: IntEnum

Padding modes ordered by quality (higher = fewer artifacts).

Examples:

>>> PaddingMode.REFLECTION > PaddingMode.ZEROS
True

Low-level target functions¶

These functions expect matrices and grids that already follow the package's pixel-coordinate and sampling conventions. Most users should prefer data_keys.

transform_keypoints ¶

transform_keypoints(keypoints: Tensor, mtx_forward: Tensor) -> Tensor

Transform (batch_size, num_points, 2) keypoints by a (batch_size, 3, 3) forward homography.

Converts each keypoint to homogeneous coordinates [x, y, 1], multiplies by the forward matrix, and returns the first two components of the result per point::

keypoints'[batch_size, num_points] = (mtx_forward[batch_size] @ [x, y, 1]^T)[:2]

Unlike bounding boxes, keypoints are transformed exactly (no AABB widening). The operation is differentiable with respect to both keypoints and mtx_forward.

Parameters:

Name	Type	Description	Default
`keypoints`	`Tensor`	Keypoints in pixel coordinates. Shape `(batch_size, num_points, 2)`, columns `[coord_x, coord_y]`, dtype `float32`.	required
`mtx_forward`	`Tensor`	Forward (not inverse) affine or projective matrix in pixel coordinates. Shape `(batch_size, 3, 3)`, dtype `float32`.	required

Returns:

Type	Description
`Tensor`	Transformed keypoints. Shape `(batch_size, num_points, 2)`.

Examples:

Identity matrix leaves keypoints unchanged:

>>> import torch
>>> keypoints = torch.tensor([[[10.0, 20.0], [30.0, 40.0]]])  # (batch_size=1, num_points=2, 2)
>>> mtx_identity = torch.eye(3).unsqueeze(0)
>>> out = transform_keypoints(keypoints, mtx_identity)
>>> torch.allclose(out, keypoints)
True

transform_bbox_xyxy ¶

transform_bbox_xyxy(boxes: Tensor, mtx_forward: Tensor) -> Tensor

Transform (batch_size, num_boxes, 4) xyxy boxes by a (batch_size, 3, 3) forward homography.

Computes all four corners of each box, transforms them through the forward matrix using homogeneous multiplication, then returns the axis-aligned bounding box (AABB) that tightly wraps the transformed corners.

The AABB wrapping step means output boxes are always axis-aligned and may be larger than the true rotated box. This is the standard trade-off for box transforms that must remain in xyxy format.

Parameters:

Name	Type	Description	Default
`boxes`	`Tensor`	Bounding boxes in xyxy format. Shape `(batch_size, num_boxes, 4)`, columns `[x1, y1, x2, y2]` in pixel coordinates, dtype `float32`.	required
`mtx_forward`	`Tensor`	Forward (not inverse) affine or projective matrix in pixel coordinates. Shape `(batch_size, 3, 3)`, dtype `float32`.	required

Returns:

Type	Description
`Tensor`	Transformed AABB boxes. Shape `(batch_size, num_boxes, 4)`, xyxy format.

Examples:

Identity matrix leaves boxes unchanged:

>>> import torch
>>> boxes = torch.tensor([[[10.0, 20.0, 50.0, 80.0]]])  # (batch_size=1, num_boxes=1, 4)
>>> mtx_identity = torch.eye(3).unsqueeze(0)
>>> out = transform_bbox_xyxy(boxes, mtx_identity)
>>> torch.allclose(out, boxes)
True

transform_bbox_xywh ¶

transform_bbox_xywh(boxes: Tensor, mtx_forward: Tensor) -> Tensor

Transform (batch_size, num_boxes, 4) xywh boxes by a (batch_size, 3, 3) forward homography.

Converts boxes from [x, y, w, h] to [x1, y1, x2, y2], delegates to :func:transform_bbox_xyxy (4-corner transform + AABB), then converts back to [x, y, w, h].

The output width and height reflect the AABB after rotation, so they will be larger than the input for non-axis-aligned transforms.

Parameters:

Name	Type	Description	Default
`boxes`	`Tensor`	Bounding boxes in xywh format. Shape `(batch_size, num_boxes, 4)`, columns `[x, y, w, h]` where `(x, y)` is the top-left corner, dtype `float32`.	required
`mtx_forward`	`Tensor`	Forward (not inverse) affine or projective matrix in pixel coordinates. Shape `(batch_size, 3, 3)`, dtype `float32`.	required

Returns:

Type	Description
`Tensor`	Transformed boxes in xywh format. Shape `(batch_size, num_boxes, 4)`.

Examples:

Identity matrix leaves boxes unchanged:

>>> import torch
>>> boxes = torch.tensor([[[10.0, 20.0, 40.0, 60.0]]])  # x, y, w, h
>>> mtx_identity = torch.eye(3).unsqueeze(0)
>>> out = transform_bbox_xywh(boxes, mtx_identity)
>>> torch.allclose(out, boxes)
True

transform_mask ¶

transform_mask(mask: Tensor, grid: Tensor, mode: MaskInterpolationStr = 'nearest') -> Tensor

Apply a precomputed affine grid to a segmentation mask.

The default mode='nearest' preserves integer class labels without fractional mixing and retains the historical no-gradient behavior. Use mode='bilinear' for differentiable float soft masks; labels may mix at boundaries. Out-of-bounds samples are filled with 0 via padding_mode='zeros'.

Parameters:

Name	Type	Description	Default
`mask`	`Tensor`	Segmentation mask. Shape `(batch_size, channels, height, width)`, typically `channels=1`. dtype: Any floating or integer dtype. Integer masks are automatically cast to a floating dtype for `grid_sample` and cast back to the original dtype afterward. Value range: integer class indices (e.g. 0, 1, 2, …). Channel convention: channel-first (PyTorch).	required
`grid`	`Tensor`	Sampling grid from `torch.nn.functional.affine_grid`. Shape `(batch_size, height, width, 2)`. Any floating dtype (`float16`, `float32`, `float64`) is accepted; integer masks are cast to `float32` internally regardless of the grid dtype to avoid fp16/bf16 rounding while keeping memory usage and bandwidth lower than `float64`. Note: `float32` exactly represents integer class IDs up to `2**24 - 1` (16777215); larger integer IDs may be rounded. Coordinates in normalised `[-1, 1]` space with `align_corners=True`.	required
`mode`	`MaskInterpolationStr`	Mask sampling mode, either `"nearest"` (default) or `"bilinear"`. Bilinear mode requires a floating-point mask.	`'nearest'`

Returns:

Type	Description
`Tensor`	Warped mask with the same shape and dtype as `mask`.

Examples:

Identity grid leaves the mask unchanged:

>>> import torch
>>> import torch.nn.functional as F
>>> mask = torch.zeros(1, 1, 4, 4)
>>> mask[0, 0, 1, 1] = 1
>>> eye2 = torch.eye(2, 3).unsqueeze(0)  # identity theta (1, 2, 3)
>>> grid = F.affine_grid(eye2, [1, 1, 4, 4], align_corners=True)
>>> out = transform_mask(mask, grid)
>>> out.shape
torch.Size([1, 1, 4, 4])
>>> bool(out[0, 0, 1, 1] == 1)
True

Types and targets¶

Data keys¶

Mask interpolation¶

Coordinate-changing passthrough¶

Output conversion¶

Segment descriptors¶

SegmentDescriptor dataclass ¶

to_dict ¶

Transform categories¶

TransformCategory ¶

Sampling enums¶

InterpolationMode ¶

PaddingMode ¶

Low-level target functions¶

transform_keypoints ¶

transform_bbox_xyxy ¶

transform_bbox_xywh ¶

transform_mask ¶

SegmentDescriptor `dataclass` ¶