Vision and media

Threshold against a local Gaussian-weighted mean minus c.

Threshold against the local arithmetic mean minus c.

Blend two equal semantic images using a * (1-alpha) + b * alpha.

Average an odd rectangular neighborhood up to 31×31.

Apply a bilateral neighborhood filter.

Apply value * contrast + brightness.

Return whether this build can decode codec.

Return whether this build can encode codec.

Extract a centered region with the requested exact extent.

Reorder one to four channels and explicitly assign the resulting format. order[c] selects the source channel for output channel c. The output format must have the same channel count as the input so pixel data and semantic metadata cannot diverge.

Clamp every pixel to the inclusive interval.

Apply one shared FP32 [3, 4] affine color transform to RGB channels.

Composite two equal images with a clamped Gray or per-channel mask.

Convert between three-channel RGB and BGR semantics, including identity.

Apply same-shape 2D cross-correlation with an odd FP32 kernel up to 31×31.

Extract a region, clipping its right and bottom edges to the source extent.

Decode a JPEG, PNG, WebP, BMP, or TGA file and upload normalized planar FP32 pixels. The result is one NCHW image shaped [1, channels, height, width]. This is a synchronous host codec boundary followed by device upload. Errors Returns an error for an empty path, filesystem failure, unsupported or malformed media, unsupported requested format, checked-size overflow, or device allocation/upload failure.

Decode JPEG, PNG, WebP, BMP, or TGA bytes and upload normalized planar FP32 pixels. The requested output format may be Gray, Rgb, or Rgba. The result is one NCHW image shaped [1, channels, height, width]. Errors Returns an error for empty, unsupported, or malformed media, unsupported requested format, checked-size overflow, or device allocation/upload failure.

Dilate with an odd rectangular structuring element up to 31×31.

Read normalized planar FP32 pixels and encode one still-image bitstream. quality is used by the lossy JPEG and WebP encoders and must be in [1, 100] for those codecs. This is an explicit blocking host-observation boundary. Errors Returns an error for Auto, unsupported image metadata, invalid quality, readback failure, checked-size overflow, or codec failure.

Fill the intersection of a rectangle and the image with one finite value.

Erode with an odd rectangular structuring element up to 31×31.

Flip an image horizontally, vertically, or along both axes.

Apply max(value, 0)^gamma * gain.

Apply a normalized Gaussian blur with replicate borders.

Add explicit-seed Philox Gaussian noise to every pixel.

Convert an RGB or RGBA image to Rec.709 grayscale.

Produce true_value for values inside the inclusive interval and zero outside it.

Invert every value around max_value.

Apply the four-neighbor discrete Laplacian.

Select the median of an odd square neighborhood up to 15×15.

Return the morphological closing minus the source.

Dilate and then erode with the same rectangular element.

Return dilation minus erosion.

Erode and then dilate with the same rectangular element.

Return the source minus its morphological opening.

Apply (pixel - mean[channel]) / std[channel] to up to three channels.

Add asymmetric padding with an explicit border policy.

Quantize [low, high] to levels evenly spaced values using round-half-up.

Resample an image from an FP32 absolute-coordinate map shaped [1|batch, 2, output_height, output_width].

Resize an FP32 channel-first image while preserving its layout and format. NCHW and CHW storage are supported. Nearest sampling follows the donor OA floor-coordinate rule. Bilinear sampling uses half-pixel centers with align_corners = false. The call records work asynchronously and returns an Image immediately. Reading its backing matrix is an explicit submission and observation point. Errors Returns an error for a zero target extent, an empty source extent, a dtype or layout outside the admitted checkpoint, checked-size overflow, or a runtime recording failure.

Bilinearly resize and normalize in one semantic and executable operation.

Rotate clockwise by a multiple of 90 degrees.

Replace a seeded fraction of pixels with salt or pepper values.

Encode an Image using the codec inferred from the output extension and save it. Supported extensions are .jpg, .jpeg, .png, .webp, .bmp, and .tga. This is an explicit blocking device-readback and filesystem boundary. Errors Returns an error for an empty or unsupported path, invalid image or quality, readback or encoding failure, or a host filesystem failure.

Encode and save one exact packed RGBA8 host image. This is the host sink used by render sessions after their own explicit completion and readback protocol has produced packed bytes. Errors Returns an error for an empty or unsupported path, zero or overflowing dimensions, a mismatched byte count, invalid quality, codec failure, or a host filesystem failure.

Compute one first-order Scharr derivative.

Blend a class palette over an RGB/RGBA image using an I32 label map.

Apply horizontal and vertical odd FP32 vector kernels up to length 31.

Apply a nonnegative four-neighbor sharpening amount.

Compute one first-order Sobel derivative.

Invert values at or above threshold around max_value.

Replace values above threshold with max_value and all others with zero.

Replace values at or below threshold with max_value and all others with zero.

Replace values at or below threshold with zero.

Replace values above threshold with zero.

Truncate values above threshold.

Apply an unsharp mask using a Gaussian detail estimate.

Apply an inverse FP32 2x3 affine transform to an image.

Apply an inverse FP32 3x3 perspective transform to an image.

Emit borrowed NAL payloads as canonical four-byte-start-code Annex-B. Errors Returns an error when the output size overflows or host allocation fails.

Copy the first H.264 picture parameter set payload.

Copy the first H.265 picture parameter set payload.

Copy the first H.264 sequence parameter set payload.

Copy the first H.265 sequence parameter set payload.

Copy the first H.265 video parameter set payload.

Retain a render Texture as one timestamped video frame. The returned frame shares a buffer-backed Texture’s packed RGBA8 storage, semantic identity, and exact Matrix producer readiness. It performs no pixel copy, submission, wait, or Image conversion. Render targets require from_texture_with_ready so their producer completion is not lost. Errors Returns an error for a render-target-backed Texture.

Retain a texture and its exact producer completion in one video frame. This is the render-target path: it shares the texture and event without a readback, wait, or pixel copy. The event must belong to the texture’s Engine. Errors Returns an error when ready comes from a different Engine.

Inspect one raw access unit or the first frame of an IVF byte stream. Errors Returns an error for malformed IVF framing, invalid OBU structure, count overflow, or an empty frame payload.

Convert an MP4 length-prefixed H.264/H.265 sample into Annex-B NAL units. Errors Returns an error for a length width other than 1, 2, or 4, a truncated length field, a zero-sized NAL unit, arithmetic overflow, or payload overrun.

Parse one AV1 frame or frame-header OBU payload against explicit reference state. Errors Returns an error for truncated or inconsistent syntax and for donor features that are not yet represented: frame IDs, decoder-model removal timing, short reference signaling, non-uniform tiles, global motion, and film grain.

Split one raw AV1 access-unit payload into borrowed OBUs. Errors Returns an error for forbidden/reserved header bits, truncated extensions or LEB128 sizes, payload overruns, empty input, and arithmetic overflow.

Parse one AV1 sequence-header OBU payload. Pass Av1Obu::payload from an OBU whose type is Av1ObuType::SequenceHeader. The returned representation is independent of Vulkan’s standard-video C structures and retains the stream metadata needed to create exact backend profiles and session parameters. Errors Returns an error when the payload is empty or truncated, declares a reserved profile, carries an invalid frame-ID length, or encodes a chroma layout outside the AV1 profile rules.

Parse the tile ranges carried by one frame or tile-group OBU. Returned offsets are relative to the access-unit byte slice originally passed to super::parse_av1_obus, matching Vulkan Video’s source-buffer offset convention. Errors Returns an error for an incompatible OBU type, missing payload, invalid tile count/range/size width, truncated size fields, payload overruns, or offsets that do not fit Vulkan’s 32-bit tile arrays.

Parse one raw H.264 PPS NAL unit without an Annex-B start code.

Parse one raw H.264 coded-slice NAL using its resolved SPS and PPS. The returned picture-order-count field contains the coded LSB for POC type zero. POC types one and two require prior-picture state and therefore return None; the decoder session must derive the final POC rather than inventing a stateless approximation.

Parse one raw H.264 SPS NAL unit without an Annex-B start code.

Parse one raw H.265 PPS NAL unit without an Annex-B start code.

Parse one raw H.265 coded-slice NAL using its resolved SPS and PPS. Dependent slice segments and inline inter-predicted short-term reference sets require additional retained state and fail with MissingCapability.

Parse one raw H.265 SPS NAL unit without an Annex-B start code.

Parse one raw H.265 VPS NAL unit without an Annex-B start code.

Split an Annex-B byte stream into borrowed H.264 NAL payloads. Both 00 00 01 and 00 00 00 01 start codes are recognized. Bytes before the first start code and empty units are ignored, matching the donor oa::FnVideo::parseNalAnnexB contract.

Query the selected device’s exact limits for one decoder profile. Extension advertisement alone does not prove that a particular profile, chroma layout, bit depth, or codec option is usable. This query presents the complete profile to Vulkan and returns MissingCapability when rejected. Errors Returns an error when the codec extension is unavailable, the exact profile is unsupported, or the Vulkan capability query fails.

Enumerate decoded-output and DPB image formats for one exact profile. Errors Returns an error when the profile or its codec extension is unsupported, a Vulkan enumeration fails, or the driver reports an unreasonable result size.

Query Vulkan Video queues and codec extensions on the Engine’s selected device. This query reports both physical-device advertisement and whether Engine creation enabled video queues, and whether at least one qualified codec session path exists. Errors Returns an error when Vulkan device-extension or queue-family enumeration fails.

Query exact H.264 or H.265 encoder limits without creating a session.

Enumerate encode-input and DPB formats for one exact profile.